Dynamic storage management method and management platform for Internet of Things equipment data

Abstract

The invention relates to the technical field of big data storage and management, and provides a dynamic storage management method and management platform for Internet of Things equipment data. The method comprises the following steps: in response to a data query request, calculating a storage optimization index for target combined data to obtain a target storage optimization index; determining a first database for storing the target combined data based on the target storage optimization index; disassembling the target combined data to obtain multiple pieces of target single combined data, and calculating a storage optimization index of each piece of target single combined data to determine a storage database of each piece of target single combined data; migrating the target single combined data of which the storage optimization level of the database to be stored is higher than that of the first database to the corresponding database to be stored, and deleting the target single combined data of which the storage optimization level of the database to be stored is not higher than that of the first database; and migrating the target combined data to the first database to realize dynamic storage optimization of the target data combination.

Description

Technical Field

[0001] The present application relates to the field of big data storage and management technology, and in particular to a dynamic storage management method, management platform, electronic device and computer-readable storage medium for IoT device data. Background Art

[0002] With the continuous development of IoT technology, its application has gradually penetrated into all walks of life. Smart homes, digital parks, smart logistics, digital factories, etc. have all emerged based on IoT. In most application scenarios, the amount of data generated by IoT devices used to collect relevant monitoring data is increasing with the increase of investment time. Therefore, the amount of data stored in the background is also increasing. How to improve the data access rate under the premise of effectively controlling the data storage cost has become an important research direction in the industry. Summary of the invention

[0003] In order to achieve reasonable storage of IoT device data, thereby effectively controlling costs and ensuring data access rate, an embodiment of the present application provides a dynamic storage management method for IoT device data, which is applied to an IoT data management platform and includes the following steps: in response to a data query request, obtaining a query result corresponding to the data query request, wherein the query result includes at least one target combination data; calculating a storage optimization index for the target combination data to obtain a target storage optimization index; determining a first database for storing the target combination data based on the target storage optimization index; when it is determined that the target combination data is composite combination data, decomposing the target combination data to obtain multiple target single combination data, and calculating the storage optimization index of each target single combination data respectively to determine the database to be stored for each target single combination data; migrating the target single combination data whose storage optimization level of the database to be stored is higher than that of the first database to the corresponding database to be stored, and deleting the target single combination data whose storage optimization level of the database to be stored is not higher than that of the first database; migrating the target combination data to the first database.

[0004] Based on the above technical solution, during the data query request processing, the target combination data in the query results is stored and updated synchronously, the storage path is optimized, and the reasonable storage of the target combination data is ensured; and according to the optimized storage method, the storage method of the target single combination data is synchronously optimized in time to ensure that the target single combination data can be stored in a higher-level database, to ensure the access rate, and to release the storage space in time to save storage costs.

[0005] Based on the same inventive concept, an embodiment of the present application also provides an Internet of Things data management platform, which is used to implement the above method.

[0006] In addition, an embodiment of the present application also provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction implements the above method when executed by the processor.

[0007] The embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program implements the above method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The drawings constituting a part of the present application are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A schematic diagram of the structure of a management system for IoT device data provided in an embodiment of the present application is shown.

[0011] Figure 2 A flow chart of a device data storage method provided in an embodiment of the present application is shown.

[0012] Figure 3 A flow chart of a method for dynamic storage management of IoT device data provided in an embodiment of the present application is shown.

[0013] Figure 4 A flow chart of a target combination data migration method provided in an embodiment of the present application is shown.

[0014] Figure 5 A flow chart of a method for periodic storage dynamic optimization provided by an embodiment of the present application is shown.

[0015] Figure 6 A flow chart of a method for obtaining query results provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more, and "first", "second" and various digital numbers are only distinguished for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0018] The features, structures or characteristics in this application may be combined in one or more embodiments in any suitable manner. In various embodiments of this application, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0019] Some optional features in the embodiments of the present application may be implemented independently in some scenarios without relying on other features to solve corresponding technical problems and achieve corresponding effects. They may also be combined with other features in some scenarios as needed.

[0020] In this application, unless otherwise specified, the same or similar parts between the various embodiments can refer to each other. In the various embodiments of this application, if there is no special description and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships. The implementation methods of this application do not constitute a limitation on the scope of protection of this application.

[0021] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0022] Please refer to Figure 1 An embodiment of the present application provides a management system for IoT device data, including an IoT data management platform (referred to as the management platform) 11, an edge gateway 12, a monitoring device 13, a cloud cache database 141, a cloud database 142, an offline database 143 and an application client 15.

[0023] Among them, the monitoring device 13 is connected to the Internet of Things, also known as an Internet of Things device, and is deployed in a specific application environment to collect monitoring data of target objects, including but not limited to various sensors, industrial equipment, smart home devices, etc. The monitoring device 13 reports the collected monitoring data to the management platform 11 through the edge gateway 12.

[0024] The edge gateway 12 is a network transfer device used to realize data interaction between the monitoring device 13 and the management platform 11 . It can also realize preprocessing of monitoring data and send the preprocessed monitoring data to the management platform 11 .

[0025] The management platform 11 is used to manage the received device data, including storage management, data retrieval and data update. Storage management includes selecting a suitable database for storing the newly added device data and optimizing the storage for historical device data. In the embodiment of the present application, the management platform can select a reasonable storage database for the newly added device data from the cloud cache database 141, the cloud database 142, and the offline database 143.

[0026] Equipment data includes three types: monitoring data uploaded by monitoring equipment in real time, attribute data of monitoring equipment, and maintenance data.

[0027] In one example, attribute data includes data describing the device itself, such as the name, model, installation location, and operating status of the monitoring device, and maintenance data includes data recording the maintenance and sorting of the device or the monitoring data of the device by the staff, such as maintenance items, maintenance content, and maintenance time. It can be understood that the above data types include dynamic data that changes over time and relatively static data. The management platform 11 can implement classified storage management of dynamic data and static data respectively.

[0028] Please refer to Figure 2 , Figure 2 The device data storage method in an embodiment of the present application is illustrated, which specifically includes the following steps.

[0029] S201, receiving device data.

[0030] In practice, as mentioned above, equipment data includes three types, and different types of equipment data have different sources. For example, monitoring data comes from reports from monitoring equipment, while attribute data and maintenance data are obtained through the data update interface provided by the management platform.

[0031] S202, disassemble the device data to obtain combined data.

[0032] Specifically, the combined data is a specific piece of data determined based on the data combination.

[0033] In the embodiment of the present application, the data combination is determined based on the dynamic fields involved in the query scope provided by the management platform. The dynamic fields refer to data that are used to record dynamic changes in the device data, such as monitoring temperature and corresponding collection time, data maintenance content and time, device maintenance content and time, etc.

[0034] In one implementation, the dynamic fields involved in a query range are determined as a data combination. For example, the query range S1 is the monitoring data of a certain device in a certain time period. The query fields involved in obtaining the corresponding query results include the device identification, temperature and collection time, and the corresponding data combination includes the device identification, temperature and collection time.

[0035] In another implementation, the dynamic fields involved in a query range can be decomposed into multiple data combinations, that is, the query range involves dynamic fields of multiple devices. For example, query range S2 is the temperature and humidity monitoring data of a certain monitoring object in a certain time period. The query fields involved in obtaining the corresponding query results include the monitoring object identifier, temperature sensor information, temperature, temperature collection time, humidity sensor information, humidity, and humidity collection time. In this way, it can be decomposed into two data combinations, where data combination 1 includes: temperature sensor identifier, temperature, and temperature collection time; data combination 2 includes: humidity sensor identifier, humidity, and humidity collection time.

[0036] It can be understood that the identifier in the above data combination example is used to uniquely identify the source of the dynamic data, and therefore needs to be included in the data combination.

[0037] Technical personnel can determine the correspondence between all data combinations and device data types in advance based on the query scope involved in the query function provided by the management platform, and configure it in the management platform, so that the management platform can directly determine the corresponding data combination based on the device data type, and extract data based on the fields involved in the data combination to obtain combined data.

[0038] S203: Calculate a first storage optimization index of the combined data.

[0039] In implementation, the first storage optimization index is used to predict the access frequency of the combined data. In one example, the first storage optimization index I1 can be calculated based on the following formula: I1=(w1PC) p1 ×exp(w2CTR)×log(1+w3MW)×max(w4TF,0.5)×min(w5TI,1.0)×(w6DT) p2 ×(1-exp(-w7M)) Among them, PC stands for the heat coefficient, which can be calculated based on the frequency of access and the activity of the data. For example, data that is frequently accessed in the same period has a higher hot coefficient, and p1 and P2 are configurable power parameters.

[0040] CTR stands for click-through rate, which can be counted through historical visit records.

[0041] MW stands for Mark Weight, which can be assigned weights based on the importance of data marked by the user or system. For example, data marked as "important" can be assigned a higher weight.

[0042] TF stands for Time Factor, which decays based on how long the data has been stored. As time goes by, the importance of data may decrease, so data that has been stored for a long time has a lower Time Factor.

[0043] TI stands for timeliness factor, which is assigned different weights according to the real-time requirements of the data. Data with high real-time requirements have a high timeliness factor.

[0044] DT represents the weight of data requirement type. Data of different requirement types have different requirements for storage and access. For example, time series data requires higher storage efficiency, and NoSQL data requires higher query performance. Therefore, corresponding weights can be assigned to different data requirement types based on the design goals of the system.

[0045] M stands for the sorting time factor, which refers to the time when the data was last sorted or maintained. Data that is sorted regularly has a higher factor, while data that has not been sorted for a long time has a lower factor.

[0046] w1, w2, w3, …, w7 are the weights of each dimension, indicating the influence of the dimension on the optimization index, and the sum of the weights should be 1.

[0047] It is understandable that the parameter values ​​in the calculation formula of the first storage optimization index can be adjusted according to actual application requirements to meet different system design requirements.

[0048] It is worth noting that in the above calculation formula, the parameter values ​​used are determined according to the actual situation. Therefore, for the newly uploaded combined data, the heat coefficient and click rate are taken as 0 to more objectively calculate the first storage optimization index.

[0049] S204: Determine a storage database and a storage method for the combined data according to the first storage optimization index.

[0050] During implementation, the management platform pre-configures the corresponding storage optimization level according to the response rate of each access database. The faster the response rate, the higher the storage optimization level.

[0051] Different storage optimization levels correspond to different index ranges. By determining the range to which the first storage optimization index belongs, the storage database and storage method for storing the combined data can be determined.

[0052] For example, ultra-high-frequency access data with an optimization index higher than 80 is stored in the cloud cache database to achieve the fastest response; high-frequency access data with an optimization index higher than 70 is stored in the cloud database to ensure fast response and efficient access; medium and low-frequency access data with an optimization index between 70 and 50 will be stored in the offline database to balance cost and performance requirements; low-frequency short-term access data with an optimization index between the threshold of 50 and 20 will be stored in the offline database to provide reliable large-scale storage capabilities; long-term low-frequency access data with an optimization index lower than 20 will be compressed first and then stored in the offline database to save storage space.

[0053] Based on the above method, reasonable storage of combined data in device data can be achieved.

[0054] It is worth noting that the received device data may also involve static data that is not included in the combined data, that is, non-combined data, which is used to record fixed values ​​in the device data, such as device name, device model, monitoring object information, installation location, etc. Since static data does not change over time and the data volume is very small, in one implementation of the present application, the static data can be directly stored in the cloud database to ensure the access rate.

[0055] In another implementation of the present application, considering that some static data may also be frequently accessed, the static data may be first stored in a cloud cache database and then dynamically migrated according to historical access frequencies.

[0056] In one example, the popularity level of each static data can be determined according to the number of times the static data is accessed, and the corresponding storage database can be selected based on the popularity level. Static data with high access popularity is stored in a database with a good access response rate. For example, during the equipment inventory, the user requests the device name, model and other data many times. Then, as the access popularity increases, the static data may be migrated from the offline database to the cloud database, or some static data may be migrated from the cloud database to the offline database due to the low number of visits.

[0057] S205, storing the combined data in a storage database according to the storage method.

[0058] Based on the above method, the device data to be stored is disassembled and classified in the form of combined data, and the first storage optimization index is calculated for different combined data respectively, so as to determine a reasonable storage method. This can, to a certain extent, realize the advance storage of high-frequency access data in the cloud to ensure the response speed, while at the same time, it can realize the storage of low-frequency access data in the offline database, thereby saving data storage costs and reducing the impact on access speed.

[0059] Furthermore, the management platform can also trigger dynamic storage optimization based on data query requests received in real time to improve the response speed to actual user needs, thereby further improving the user experience.

[0060] Please refer to Figure 3 The dynamic storage management method of IoT device data provided in the embodiment of the present application includes the following steps.

[0061] S301, in response to a data query request, obtaining a query result corresponding to the data query request.

[0062] The query result includes an indication of a piece of combined data, and the target combined data is any piece of combined data in the query result.

[0063] S302, calculating the storage optimization index for the target combination data to obtain the target storage optimization index.

[0064] The calculation method of the target storage optimization index is the same as the calculation method of the first storage optimization index, which will not be described in detail.

[0065] S303: Determine a first database for storing target combination data based on the target storage optimization index.

[0066] S304, determining whether the first database is consistent with the second database.

[0067] The second database is a database that currently stores the target combination data.

[0068] If they are inconsistent, step S305 is executed; if they are consistent, the process ends.

[0069] S305: Migrate the target combination data to the first database.

[0070] In one implementation, the migration can be completed by storing the target combination data in the first database and deleting it from the second database.

[0071] In another implementation, in order to improve the response to data access requests and reduce the number of cross-database retrievals, the management platform can determine whether to create composite combination data for the query range based on the actual access to the data in the query range that includes multiple data combinations, for synchronously recording multiple combination data. The composite combination data is composed of at least two cross-dimensional and jointly queried single combination data. The so-called cross-dimensional and jointly queried means that the device dimensions or time dimensions corresponding to each single combination data output in a query result are different. In one example, the composite combination data may include real-time data collected by different monitoring devices for the same monitoring object, or may include real-time data collected by multiple devices of the same type in the same time dimension, or may be data collected by the same device in different time periods.

[0072] In one example, each time the management platform receives a data query request, it may first determine the query scope pointed to by the query request based on the sending page of the request. When the query scope pointed to by the request includes multiple combinations of data, the access data of the query scope is recorded, and the recorded access data is analyzed to determine whether the conditions for creating a composite combination are met. If so, a composite combination is created, and all target combination data in this query result are stored in the form of composite combination data, and the storage database of the composite combination data is determined at the same time.

[0073] In implementation, determining the storage database for the composite combination data may include selecting a database with the highest optimization index threshold level among the storage databases for the target combination data included therein, for storing the composite combination data, thereby ensuring the response efficiency of the composite combination data.

[0074] It is worth noting that after the composite combination data is stored in the corresponding database, the duplicate data in the database can be deleted, that is, the target combination data stored in the database is deleted to release storage space.

[0075] In one example, the creation condition includes that the access popularity of the composite combined data is higher than the single access popularity of any combined data in the composite combined data, wherein the single access popularity is obtained based on the number of times the combined data is output separately in the query result.

[0076] The query ranges S1 and S2 in the above example are taken as an example for explanation.

[0077] The management platform determines that the query range S1 only includes data combination 1 or data combination 2, that is, the query result only needs to output the temperature combination data or humidity combination data under the same time dimension; while the query range S2 includes data combination 1 and data combination 2, and the corresponding query results need to output the temperature combination data and humidity combination data simultaneously. When the access popularity of the query range S2 is higher than the access popularity of the query range S1 for the same target combination data according to the query request, it indicates that the user is more inclined to view the relevant monitoring data in the dimension of the monitoring object. Therefore, a composite combination including data combination 1 and data combination 2 can be constructed based on the query range S2 to store the target combination data that meets the creation conditions.

[0078] For example, users frequently request to query data records with abnormal temperature and humidity in the monitoring data of a monitored object. When the output number of temperature monitoring data or humidity monitoring data in a query result is greater than the number in the query output result of S1, the query result can be stored in the form of composite combined data.

[0079] After the composite combination data is stored in the database, the management platform can also dynamically optimize the storage method of the composite combination data according to data query requests.

[0080] Please refer to Figure 4 In a scenario where there is composite combined data, the method for migrating target combined data to the first database includes the following steps.

[0081] S401, determining whether the target combined data is composite combined data.

[0082] In one example, the management platform may set a tag for each piece of composite combined data for easy identification.

[0083] When it is determined that the target combined data is compound combined data, step S402 is executed; when it is determined that the target combined data is not compound combined data, step S405 is executed.

[0084] S402, decomposing the target combination data to obtain a plurality of target single combination data, and calculating the storage optimization index of each target single combination data respectively to determine the database where each target single combination data should be stored.

[0085] S403, migrating the target single combination data whose storage optimization level of the database to be stored is higher than that of the first database to the corresponding database to be stored, and deleting the target single combination data whose storage optimization level of the database to be stored is not higher than that of the first database.

[0086] It is understandable that when the current storage database of the target single combined data is consistent with the database to be stored, no migration operation is required.

[0087] S404: Migrate the target combination data to the first database.

[0088] In this way, while realizing dynamic storage optimization of the target combination data, the target single combination data can also be optimized synchronously, avoiding negative impact on the storage method of the target single combination data due to the migration of the composite combination data. For example, the target single combination data that should be stored in the cloud database is migrated to the offline database along with the composite combination data, thereby affecting the access rate of the target single combination data.

[0089] S405, determining whether the target combination data is stored in the target composite combination data.

[0090] If yes, execute step S406; otherwise, execute step S407.

[0091] S406: Determine the database where the target composite data should be stored based on the storage optimization index of the target composite data, and migrate the target composite data to the database where the target composite data should be stored.

[0092] In implementation, during the calculation of the storage optimization index of compound combination data, the statistics of the heat coefficient and click-through rate are determined based on the number of times the compound combination data is queried, and the method of determining other parameter values ​​includes but is not limited to selecting the highest value, lowest value or average value in a single combination data as the corresponding parameter value, which can be selected according to actual application requirements.

[0093] The method for determining the storage database based on the storage optimization index can be referred to above.

[0094] S407: If the target composite combination data is stored in the first database, the target combination data is deleted; otherwise, the target combination data is migrated to the first database.

[0095] Based on the above technical solution, the management platform can perform real-time dynamic storage optimization of target combination data, target single combination data, and target compound combination data in the process of processing data query requests. In this way, the storage path of the user's current access data can be dynamically adjusted and data migration can be performed in a timely manner, thereby improving the response rate when responding to users' frequent requests for access data.

[0096] In other embodiments of the present application, when the management platform determines that the query results contain multiple pages of data, it can cache other data in the process of displaying the homepage data to the user, that is, write it into the cloud cache database in advance, so that the user can respond quickly when requesting the next page, thereby improving the user experience.

[0097] In order to achieve dynamic storage optimization of stored data, further adaptively adjust storage costs and access rates, and ensure user experience, in an embodiment of the present application, the management platform can also periodically adjust the storage method of stored data to adapt to the ever-changing data requirements.

[0098] Please refer to Figure 5 The method for periodic storage dynamic optimization in the embodiment of the present application includes the following steps.

[0099] S501, periodically calculating a storage optimization index for combined data in each database to obtain a second storage optimization index.

[0100] In implementation, the second storage optimization index is calculated in the same manner as the first storage optimization index. During periodic calculation, it is only necessary to determine the latest value of each parameter and substitute it into the above calculation formula to obtain the corresponding index value.

[0101] It is worth noting that when there is complex combination data, the specific parameter values ​​can be determined based on the method described above.

[0102] S502: Determine a database where the combined data should be stored based on the second storage optimization index.

[0103] S503, migrating the combined data to a database to be stored.

[0104] For application scenarios where there is no composite data, the composite data can be directly transferred to the corresponding storage database.

[0105] For application scenarios that store composite data, all single combination data can be transferred to the corresponding storage database first, and then the composite data can be migrated to the storage database, and then the data deduplication strategy can be executed to free up storage space.

[0106] In one example, the data deduplication strategy includes performing the following operations in sequence: when compound combination data and the single combination data contained therein are stored in the same database at the same time, the single combination data is deleted; when a combination data is stored in different databases as single combination data and compound combination data respectively, and the storage optimization level of storing the compound combination data is higher than that of the single combination data, the single combination data is deleted; when all the single combination data in the compound combination data are separately stored in a database with a storage optimization level higher than that of the compound combination data, the compound combination data is deleted.

[0107] Based on the above dynamic storage optimization method, each piece of combined data can be migrated in different databases following the changes in the storage optimization index, and the data access efficiency can be improved by disassembling and assembling composite combined data, while releasing storage space in a timely manner.

[0108] It can be seen that in the dynamic storage management method of IoT device data provided in the embodiment of the present application, the management platform divides the newly received device data into combined data and non-combined data according to the provided query range and the dynamic characteristics of the data, so as to effectively distinguish the dynamic data and static data of the device, and then realize classified storage, thereby effectively reducing the number of repeated storage times of non-combined data and saving storage space; at the same time, combined with the storage optimization index of each combined data, suitable storage space is selected for the combined data, so that the combined data with a higher probability of being accessed is stored in a database with a faster access response, thereby optimizing data access efficiency and improving user experience.

[0109] Furthermore, the storage of target combination data is dynamically optimized in real time according to the data query request, and the related single combination data and compound combination data can be synchronously managed, so that the storage form of the related combination data is more reasonable.

[0110] In addition, by periodically recalculating the storage optimization index of all combined data in the database, determining the database that should be stored currently, and performing corresponding data migration and data deduplication operations, effective management of composite combined data can be achieved, freeing up storage space as much as possible while ensuring access speed, saving storage costs.

[0111] In the embodiments of this application, please refer to Figure 6 , the method for obtaining the query result specifically includes the following steps.

[0112] S601, determining a target query range pointed to by a data query request, and determining a query combination to be retrieved according to the target query range.

[0113] In a specific implementation, a user may submit a data query request based on a data query interface provided by an application client, and thus the management platform may quickly determine a target query scope based on a data query interface identifier.

[0114] In addition, the management platform can also determine the target query scope based on the query requirement information carried in the data query request.

[0115] The management platform can determine the query combinations that need to be retrieved during this query process based on the target query scope, including single combinations and compound combinations.

[0116] For example, the query combination corresponding to the query range S2 includes data combination 1, data combination 2, and a composite combination formed by combining data combinations 1 and 2.

[0117] It is worth noting that in the embodiment of the present application, device data is classified and stored in different databases, and migration may occur at any time. Therefore, in order to reduce the query statement generation burden of the front-end page, the application client can directly record the query requirements submitted by the user in the corresponding retrieval file and issue it along with the data query request.

[0118] The management platform can obtain specific query requirements, including the required query fields, filter conditions, etc. by parsing the search file in the request. The search file can be saved in Jason or XML format so that the management platform can accurately extract the query requirements.

[0119] S602: Predict a storage optimization index range corresponding to the query result based on the query requirement and the query combination.

[0120] In implementation, it is necessary to predict the corresponding storage optimization index range when the query results are stored in the form of various query combinations.

[0121] In one example, when the query combination includes a single combination and a compound combination, the management platform can collect statistics on the historical storage optimization index of each data combination, and obtain the highest and lowest values ​​of the historical storage optimization index under each data combination. Then, the maximum value of the highest values ​​and the minimum value of the lowest values ​​corresponding to each data combination are taken as the storage optimization indication range corresponding to the query result range.

[0122] In another example, if the query combination does not include a composite combination, the storage optimization indication ranges corresponding to different combination data in the query result range can be determined based on the highest and lowest values ​​of the historical storage optimization index of each single combination.

[0123] S603, searching the database according to the storage optimization index range match.

[0124] In implementation, the corresponding storage databases may be matched according to the maximum value and the minimum value in the storage optimization indication range, wherein the matching method of the storage database may refer to the above.

[0125] The matched storage databases and all databases whose storage optimization levels are between the matched storage databases are determined as retrieval databases.

[0126] S604, constructing a query statement based on the query requirement, query combination and retrieval database.

[0127] In one implementation, the management platform can create a data table based on the data combination for classifying and saving the combined data. In this way, the data table required for query can be determined based on the query combination, and the query statement corresponding to the query requirement can be constructed according to the grammatical requirements of the retrieval database.

[0128] S605: Execute the query statement to obtain the query result.

[0129] In one implementation, the query statement may be synchronously sent to a corresponding search database to perform a query, and the query result returned by the database may be received.

[0130] In another implementation, query results may be obtained in sequence from high to low according to the storage optimization level corresponding to the retrieval database, and when the total number of query results reaches the display upper limit, the query is stopped. In this way, query results of retrieval databases with higher storage optimization levels may be displayed preferentially.

[0131] It is understandable that in the process of executing the query statement, it may involve the query of non-combined data, for example, it is necessary to query the device identification according to the device name, or query the device identification according to the monitored object. In one example, the query of non-combined data can be directly queried from the cloud database. In another example, the current storage database can be determined according to the historical access times of the non-combined data, and the current storage database can be queried accordingly.

[0132] Based on the above technical solution, comprehensive and rapid acquisition of query results can be achieved.

[0133] In addition, an embodiment of the present application also provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements a method as in any one of the implementations in the embodiments of the present application; wherein the processor may adopt a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits, for executing relevant programs to implement the method in any one of the implementations in the embodiments of the present application.

[0134] The processor may also be an integrated circuit electronic device with signal processing capability. In the implementation process, each step of the method in any implementation of the embodiments of the present application may be completed by an integrated logic circuit of hardware in the processor or by instructions in software form.

[0135] The above-mentioned processor can also be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be combined and executed.

[0136] The software module may be located in a random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register or other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory, and combines its hardware to complete the functions required to be performed by the units included in the data processing device of the embodiment of the present application, or executes the method in any one of the implementation modes in the embodiment of the present application.

[0137] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method when executed by a processor.

[0138] Those skilled in the art can understand that all or part of the steps in the above-mentioned implementation method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of each implementation method of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0139] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for dynamic storage management of IoT device data, characterized in that: The method is applied to the Internet of Things data management platform and comprises the steps of: In response to a data query request, obtaining a query result corresponding to the data query request, wherein the query result includes at least one target combination data; Calculating a storage optimization index for the target combined data to obtain a target storage optimization index; Determining a first database for storing the target combination data based on the target storage optimization index; In the case where it is determined that the target combination data is composite combination data, the target combination data is disassembled to obtain a plurality of target single combination data, and the storage optimization index of each of the target single combination data is calculated respectively to determine the database in which each of the target single combination data should be stored; Migrating the target single combination data whose storage optimization level of the database to be stored is higher than that of the first database to the corresponding database to be stored, and deleting the target single combination data whose storage optimization level of the database to be stored is not higher than that of the first database; The target combination data is migrated to the first database.

2. The method according to claim 1, characterized in that: The storage method of the target combination data includes: Receive device data; Disassembling the device data to obtain the target combination data; Calculating a first storage optimization index of the target combination data; Determine the storage database and storage method of the target combination data according to the first storage optimization index; The target combination data is stored in the storage database according to the storage method.

3. The method according to claim 2, characterized in that The method further comprises: Disassembling the device data to obtain non-combined data; The non-combined data is stored in a cloud database.

4. The method according to claim 2, characterized in that: The method further comprises: Periodically calculating a storage optimization index for the target combination data to obtain a second storage optimization index; Determine a database in which the target combination data should be stored based on the second storage optimization index; The target combination data is migrated to the storage database.

5. The method according to claim 4, characterized in that When the target combination data is composite combination data, migrating the target combination data to the database to be stored includes: After all single combination data has been migrated; Migrating the target combination data to the database to be stored; Execute data deduplication strategy.

6. The method according to claim 5, characterized in that The execution data deduplication strategy includes: When compound combination data and the single combination data it contains are stored in the same database at the same time, the single combination data it contains shall be deleted; when the same combination data is stored in different databases as single combination data and compound combination data respectively, and the storage optimization level of the database storing the compound combination data is higher than that of the single combination data, the single combination data shall be deleted; when all the single combination data in the compound combination data are stored separately in the database with a storage optimization level higher than that of the compound combination data, the compound combination data shall be deleted.

7. The method according to claim 1, characterized in that The method for obtaining the query result corresponding to the data query request includes: Determine the target query scope pointed to by the data query request, and determine the query combination required to be retrieved according to the target query scope; Predicting a storage optimization index range corresponding to the query result based on the query requirement and the query combination; Predicting a storage optimization index range corresponding to the query result based on the data query requirement; Matching and searching a database according to the storage optimization index range; Constructing a query statement based on the query requirement, the query combination and the search database; Execute the query statement to obtain the query result.

8. An Internet of Things data management platform, characterized in that: The platform is used to implement the method according to any one of claims 1 to 7.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction implements the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images