A dynamic storage management method and management platform for internet of things device data

Abstract

The application relates to the technical field of big data storage and management, and provides a dynamic storage management method and a management platform for Internet of Things equipment data. The method comprises the following steps: in response to a data query request, calculating a target storage optimization index for target combined data to obtain a target storage optimization index; determining a first database for storing the target combined data based on the target storage optimization index; disassembling the target combined data to obtain a plurality of target single combined data, calculating a storage optimization index of each target single combined data to determine a storage database of each target single combined data; migrating the target single combined data with a storage optimization level higher than that of the first database to the corresponding storage database, and deleting the target single combined data with a storage optimization level not higher than that of the first database; and migrating the target combined data to the first database to realize dynamic storage optimization of the target data combination.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data storage and management, in particular to a dynamic storage management method for Internet of Things device data, a management platform, an electronic device and a computer readable storage medium. BACKGROUND

[0002] With the continuous development of Internet of Things technology, the application of Internet of Things technology has gradually penetrated into various industries. Intelligent home, digital park, smart logistics, digital factory and the like are all based on Internet of Things. In most application scenarios, the data generated by Internet of Things devices for collecting relevant monitoring data also increases with the growth of investment time, so the amount of data stored in the background is also increasing. How to improve the data access rate while effectively controlling the data storage cost has become an important research direction in the industry. SUMMARY

[0003] In order to realize the reasonable storage of Internet of Things device data, thereby effectively controlling the cost and ensuring the data access rate, the present application provides a dynamic storage management method for Internet of Things device data. The method is applied to an Internet of Things data management platform and includes the following steps: in response to a data query request, obtaining a query result corresponding to the data query request, the query result including at least one target combined data; calculating a storage optimization index for the target combined data to obtain a target storage optimization index; determining a first database for storing the target combined data based on the target storage optimization index; in the case where the target combined data is a composite combined data, disassembling the target combined data to obtain a plurality of target single combined data, calculating the storage optimization index of each target single combined data to determine the storage database of each target single combined data; migrating the target single combined data whose storage optimization level of the storage database is higher than that of the first database to the corresponding storage database, and deleting the target single combined data whose storage optimization level of the storage database is not higher than that of the first database; and migrating the target combined data to the first database.

[0004] Based on the above technical solution, by synchronously realizing the storage update of the target combined data in the query result in the data query request processing process, the storage path is optimized, and the reasonable storage of the target combined data is ensured. According to the optimized storage method, the storage mode of the target single combined data is timely optimized, so that the target single combined data can be stored in a higher level database, the access rate is ensured, the storage space is released in time, and the storage cost is saved.

[0005] Based on the same inventive concept, the application further provides an Internet of Things data management platform, which is used to implement the method.

[0006] In addition, the application further provides an electronic device, which comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement the method.

[0007] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method. BRIEF DESCRIPTION OF DRAWINGS

[0008] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and are used to interpret the present application together with the description. The accompanying drawings are not intended to limit the present application.

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings.

[0010] Figure 1 The structure schematic diagram of the Internet of Things device data management system provided by the embodiments of the present application is shown.

[0011] Figure 2 The device data storage method flow chart provided by the embodiments of the present application is shown.

[0012] Figure 3 The dynamic storage management method flow chart of the Internet of Things device data provided by the embodiments of the present application is shown.

[0013] Figure 4 The target combination data migration method flow chart provided by the embodiments of the present application is shown.

[0014] Figure 5 The method flow chart of periodic storage dynamic optimization provided by the embodiments of the present application is shown.

[0015] Figure 6 The method flow chart of query result acquisition provided by the embodiments of the present application is shown. DETAILED DESCRIPTION

[0016] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0017] In the description of the embodiments of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more, "first", "second", and various numerical numbers are only for the convenience of differentiation, and do not limit the scope of the embodiments of the present application.

[0018] The features, structures or characteristics in the present application can be combined in any suitable manner in one or more embodiments. In various embodiments of the present application, the size of the sequence of each process does not mean the order of execution, the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0019] Some optional features in the embodiments of the present application can be implemented independently in some scenarios without relying on other features, to solve the corresponding technical problems and achieve the corresponding effects, or can be combined with other features according to the needs in some scenarios.

[0020] In the present application, unless otherwise specified, the same or similar parts between various embodiments can be mutually referred. In various embodiments of the present application, if there is no special specification and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be mutually referred, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship. The implementation of the present application does not constitute a limitation on the protection scope of the present application.

[0021] The embodiments of the present application will be described in detail below in combination with the drawings.

[0022] Please refer to Figure 1 The embodiments of the present application provide a management system for Internet of Things device data, which comprises an Internet of Things data management platform (referred to as a management platform) 11, an edge gateway 12, a monitoring device 13, a cloud cache database 141, a cloud database 142, an offline database 143 and an application client 15.

[0023] Among them, the monitoring device 13 is connected to the Internet of Things, also known as an Internet of Things device, deployed in a specific application environment, used to collect monitoring data of target objects, including but not limited to various sensors, industrial devices, smart home devices, etc. The monitoring device 13 reports the collected monitoring data to the management platform 11 through the edge gateway 12.

[0024] The edge gateway 12 is a network relay device, which is configured to realize data interaction between the monitoring device 13 and the management platform 11, and also can realize preprocessing of the monitoring data and send the preprocessed monitoring data to the management platform 11.

[0025] The management platform 11 is configured to manage the received device data, including storage management, data retrieval and data update. The storage management includes selecting a suitable database for storing the newly added device data and optimizing the storage of the historical device data. In the embodiment of the present application, the management platform can select a reasonable storage database for the newly added device data from the cloud cache database 141, the cloud database 142 and the offline database 143.

[0026] The device data includes three types: monitoring data uploaded by the monitoring device in real time, attribute data of the monitoring device and maintenance data.

[0027] In an example, the attribute data includes the name, model, installation position and running state of the monitoring device, which describes the device itself, and the maintenance data includes the maintenance project, maintenance content and maintenance time, which records the maintenance and arrangement of the device or the monitoring data of the device by the staff. It can be understood that the above-mentioned data types include dynamic data changing over time and static data relatively static. The management platform 11 can realize classified storage management of the dynamic data and the static data respectively.

[0028] Please refer to Figure 2 , Figure 2 The device data storage method in the embodiment of the present application is shown, which specifically includes the following steps.

[0029] S201, receiving device data.

[0030] In the implementation, as described above, the device data includes three types, and the sources of the device data of different types are different, for example, the monitoring data comes from the report of the monitoring device, and the attribute data and the maintenance data are obtained through the data update interface provided by the management platform.

[0031] S202, decomposing the device data to obtain combined data.

[0032] Specifically, the combined data is a specific data determined based on data combination.

[0033] In the embodiment of the present application, the data combination is determined based on the dynamic fields involved in the query range provided by the management platform. The dynamic field refers to the data used to record the data in the device data which will change dynamically, for example, the monitoring temperature and the corresponding collection time, the data maintenance content and time, the device maintenance content and time, etc.

[0034] In one implementation, the dynamic fields involved in a query range are determined as a data combination. For example, the query range S1 is the monitoring data of a certain device in a certain time period. The query fields involved in obtaining the corresponding query results include device identification, temperature and collection time, and the corresponding data combination includes device identification, temperature and collection time.

[0035] In another implementation, the dynamic fields involved in a query range can be decomposed into multiple data combinations, that is, the query range involves dynamic fields of multiple devices. For example, the query range S2 is the temperature and humidity monitoring data of a certain monitoring object in a certain time period. The query fields involved in obtaining the corresponding query results include the monitoring object identifier, temperature sensor information, temperature, temperature acquisition time, humidity sensor information, humidity, and humidity acquisition time. In this way, it can be decomposed into two data combinations, where data combination 1 includes: temperature sensor identifier, temperature, and temperature acquisition time; data combination 2 includes: humidity sensor identifier, humidity, and humidity acquisition time.

[0036] It can be understood that the identifier in the above data combination example is used to uniquely identify the source of the dynamic data, and therefore needs to be included in the data combination.

[0037] Technical personnel can determine the correspondence between all data combinations and device data types in advance based on the query scope involved in the query function provided by the management platform, and configure it in the management platform so that the management platform can directly determine the corresponding data combination based on the device data type, and extract data based on the fields involved in the data combination to obtain combined data.

[0038] S203: Calculate a first storage optimization index of the combined data.

[0039] In implementation, the first storage optimization index is used to predict the access frequency of the combined data. In one example, the first storage optimization index I1 can be calculated based on the following formula:

[0040] I1=(w1PC) p1 ×exp(w2CTR)×log(1+w3MW)×max(w4TF,0.5)×min(w5TI,1.0)×(w6DT) p2 ×(1-exp(-w7M))

[0041] PC stands for the hot coefficient, which is calculated based on the frequency of access and the activity of the data. For example, data that is frequently accessed within the same time period has a higher hot coefficient. p1 and P2 are configurable power parameters.

[0042] CTR stands for click-through rate, which can be counted through historical visit records.

[0043] MW represents the marking weight, which is marked and assigned a weight based on the importance of the data to the user or the system. For example, data marked as "important" can be assigned a higher weight.

[0044] TF represents the time factor, which is attenuated based on the storage time of the data. Over time, the importance of the data may decrease, so the time factor of data stored for a longer time is lower.

[0045] TI represents the timeliness factor, which is assigned a different weight according to the real-time requirement of the data. The timeliness factor of data with high real-time requirement is high.

[0046] DT represents the data demand type weight, which is assigned a corresponding weight according to the design goal of the system, because different demand types of data have different storage and access requirements. For example, time series data requires high storage efficiency, and NoSQL data requires high query performance.

[0047] M represents the arrangement time factor, which refers to the last arrangement or maintenance time of the data. The factor of regularly arranged data is higher, and the factor of long-term unarranged data is lower.

[0048] w1, w2, w3, …, w7 are the weights of each dimension, representing the influence degree of the dimension on the optimization index, and the sum of the weights should be 1.

[0049] It can be understood that the parameter values in the calculation formula of the first storage optimization index can be adjusted according to the actual application requirements to meet different system design requirements.

[0050] It is worth noting that in the above calculation formula, the parameter values used are determined according to the actual situation, so the hotness coefficient and the click rate of the newly uploaded combined data are 0, so as to more objectively calculate the first storage optimization index.

[0051] S204, according to the first storage optimization index, determines the storage database and storage method for the combined data.

[0052] In implementation, the management platform pre-configures the corresponding storage optimization level according to the response rate of each access database. The faster the response rate, the higher the storage optimization level.

[0053] Different storage optimization levels correspond to different index ranges, and by judging the range to which the first storage optimization index belongs, the storage database and storage method for storing the combined data can be determined.

[0054] For example, the ultra-high frequency access data with an optimization index higher than 80 is stored in the cloud cache database to achieve the fastest response; the high frequency access data with an optimization index higher than 70 is stored in the cloud database to ensure fast response and efficient access; the medium-low frequency access data with an optimization index between 70 and 50 is stored in the offline database to balance the cost and performance requirements; the low frequency short-term access data with an optimization index between the threshold 50 and 20 is stored in the offline database to provide reliable large-scale storage capacity; the long-time low frequency access data with an optimization index lower than 20 is first compressed and then stored in the offline database to save storage space.

[0055] Based on the above method, the combined data in the device data can be reasonably stored.

[0056] It is worth noting that the received device data can also involve static data, i.e., non-combined data, which is not included in the combined data, and the non-combined data is used to record fixed values in the device data, such as device name, device model, monitoring object information, installation location, etc. Since the static data does not change over time and the data volume is small, in an embodiment of the present application, the static data can be directly stored in the cloud database to ensure the access rate.

[0057] In another embodiment of the present application, considering that part of the static data can also be frequently accessed, the static data can be first stored in the cloud cache database, and then dynamically migrated according to the historical access frequency.

[0058] In an example, the heat level of each static data can be determined according to the access times of the static data, and the corresponding storage database is selected based on the heat level. The static data with high access heat is stored in a database with good access response rate, for example, during the device inventory, the device name, model and other data are requested by the user many times, then with the increase of access heat, these static data can be migrated from the offline database to the cloud database, or part of the static data is migrated from the cloud database to the offline database due to less access volume.

[0059] S205, store the combined data in the storage database according to the storage mode.

[0060] Based on the above method, the device data to be stored is classified by means of combined data, and the first storage optimization index of different combined data is calculated, so as to determine a reasonable storage mode, which can to some extent realize the pre-storage of high frequency access data in the cloud to ensure the response speed, and at the same time realize the storage of low frequency access data in the offline database, thereby saving the data storage cost and reducing the influence on the access speed.

[0061] Further, the management platform can further trigger dynamic storage optimization based on the real-time received data query request to improve the response speed to actual user demand, thereby further improving the user experience.

[0062] Please refer to Figure 3 The dynamic storage management method of the Internet of Things device data provided by the embodiments of the present application comprises the following steps.

[0063] S301, in response to a data query request, obtaining a query result corresponding to the data query request.

[0064] The query result comprises an indication of a piece of combined data, and the target combined data is any piece of combined data in the query result.

[0065] S302, calculating a target storage optimization index for the target combined data, to obtain the target storage optimization index.

[0066] The calculation method of the target storage optimization index is the same as that of the first storage optimization index, and will not be repeated.

[0067] S303, determining a first database for storing the target combined data based on the target storage optimization index.

[0068] S304, judging whether the first database is consistent with a second database.

[0069] The second database is a database currently storing the target combined data.

[0070] If not, step S305 is executed; if yes, the process is ended.

[0071] S305, migrating the target combined data to the first database.

[0072] In an implementation, the migration can be completed by storing the target combined data into the first database and deleting it from the second database.

[0073] In another implementation, in order to improve the response to data access requests, reduce the number of retrievals across databases, the management platform can determine whether to create a composite combined data for a query range according to the actual access of the data in the query range containing multiple data combinations, for synchronously recording multiple combined data. The composite combined data is composed of at least two single combined data which are cross-dimension and can be jointly queried. Cross-dimension and jointly queried means that the device dimensions or time dimensions corresponding to each single combined data output in a query result are different. In an example, the composite combined data can include real-time data collected by different monitoring devices for the same monitoring object, or can include real-time collected data of multiple devices of the same type in the same time dimension, or is collected data of the same device in different time periods.

[0074] In an example, the management platform can first determine the query range pointed to by the query request according to the sending page of the request when receiving the data query request each time. When the query range pointed to by the request includes multiple combined data, the access data of the query range is recorded, and the recorded access data is analyzed to determine whether the creation condition of the composite combination is met. If it is met, the composite combination is created, and all target combined data in the query result is stored in the form of composite combined data, and the storage database of the composite combined data is determined.

[0075] In an implementation, determining the storage database of the composite combined data can include selecting the database with the highest optimization index threshold value among the storage databases of the target combined data contained, for storing the composite combined data, so as to ensure the response efficiency of the composite combined data.

[0076] It is worth noting that when the composite combined data is stored in the corresponding database, the repeated data in the database can be deleted, that is, the target combined data stored in the database is deleted to release the storage space.

[0077] In an example, the creation condition includes that the access popularity of the composite combined data is higher than the single access popularity of any combined data in the composite combined data, wherein the single access popularity is obtained based on the number of times of outputting the combined data in the query result.

[0078] The above query ranges S1 and S2 in the example are taken as examples for illustration.

[0079] The management platform determines that the query range S1 only contains data combination 1 or data combination 2, that is, the query result only needs to output temperature combination data or humidity combination data under the same time dimension; and the query range S2 includes data combination 1 and data combination 2, and the corresponding query result needs to synchronously output temperature combination data and humidity combination data. When the access heat of the query range S2 is higher than the access heat of the query range S1 according to the query request for the same target combination data, it indicates that the user is more inclined to view the related monitoring data in the dimension of the monitoring object, and therefore, a composite combination including data combination 1 and data combination 2 can be constructed based on the query range S2, for storing the target combination data meeting the creation condition.

[0080] For example, the user frequently requests to query data records in which the temperature and humidity of the monitoring data of a monitoring object are both abnormal. When the output times of the temperature monitoring data or the humidity monitoring data in a query result are greater than the times in the query output result of S1, the query result can be stored in the form of composite combination data.

[0081] After the composite combination data is stored in the database, the management platform can also dynamically optimize the storage mode of the composite combination data according to the data query request.

[0082] Please refer to Figure 4 In the scenario in which the composite combination data exists, the method for migrating the target combination data to the first database includes the following steps.

[0083] S401, determining whether the target combination data is composite combination data.

[0084] In an example, the management platform can set a mark for each piece of composite combination data, so as to facilitate identification.

[0085] In the case where it is determined that the target combination data is composite combination data, step S402 is performed; and in the case where it is determined that the target combination data is not composite combination data, step S405 is performed.

[0086] S402, disassembling the target combination data to obtain a plurality of target single combination data, respectively calculating storage optimization indexes of the target single combination data to determine the databases in which the target single combination data should be stored.

[0087] S403, migrating the target single combination data whose storage optimization level of the corresponding database in which the target single combination data should be stored is higher than that of the first database to the corresponding database, and deleting the target single combination data whose storage optimization level of the corresponding database in which the target single combination data should be stored is not higher than that of the first database.

[0088] It can be understood that when the current storage database of the target single combination data is consistent with the should storage database, the migration operation is not needed.

[0089] S404, migrating the target combination data to the first database.

[0090] In this way, the target single combination data can be synchronized optimized while the dynamic storage optimization of the target combination data is implemented, avoiding the negative impact on the storage mode of the target single combination data due to the migration of the compound combination data, for example, the target single combination data that should be stored in the cloud database is migrated to the offline database with the compound combination data, thereby affecting the access rate of the target single combination data.

[0091] S405, determining whether the target combination data is stored in the target compound combination data.

[0092] If yes, step S406 is executed; otherwise, step S407 is executed.

[0093] S406, determining the should storage database of the target compound combination data based on the storage optimization index of the target compound combination data, and migrating the target compound combination data to the should storage database.

[0094] In implementation, in the calculation process of the storage optimization index of the compound combination data, the statistics of the heat coefficient and the click rate are determined based on the number of times of querying the compound combination data, and the determination manner of other parameter values includes but is not limited to selecting the highest value, the lowest value or the average value in the single combination data as the corresponding parameter value, which can be selected according to actual application requirements.

[0095] The manner of determining the should storage database based on the storage optimization index can refer to the above.

[0096] S407, if the target compound combination data is stored in the first database, deleting the target combination data; otherwise, migrating the target combination data to the first database.

[0097] Based on the above technical solution, the management platform can perform real-time storage dynamic optimization on the target combination data, the target single combination data and the target compound combination data in the process of processing the data query request, so that the storage path of the current access data of the user can be dynamically adjusted and the data migration can be performed in time, thereby improving the response rate when the user frequently requests the access data.

[0098] In other embodiments of the present application, when the management platform determines that the query result contains multiple pages of data, the other data can be cached, i.e. written into the cloud cache database in advance, so that the user can be quickly responded when requesting the next page, thereby improving the user experience.

[0099] In order to enable dynamic storage optimization on the stored data, further adaptively adjust the storage cost and access rate, and ensure user experience, in the embodiments of the present application, the management platform can also periodically adjust the storage mode of the stored data, so as to adapt to the continuous change of data demand.

[0100] Please refer to Figure 5 The method of periodic storage dynamic optimization in the embodiments of the present application includes the following steps.

[0101] S501, periodically calculate the storage optimization index of the combined data in each database to obtain a second storage optimization index.

[0102] In implementation, the calculation method of the second storage optimization index is the same as that of the first storage optimization index. When periodically calculating, only the latest value of each parameter needs to be determined to be substituted into the above calculation formula to obtain the corresponding index value.

[0103] It is worth noting that when there is composite combined data, the specific parameter value can be determined based on the method described above.

[0104] S502, determine the database to which the combined data should be stored based on the second storage optimization index.

[0105] S503, migrate the combined data to the database to which the combined data should be stored.

[0106] For application scenarios where there is no composite combined data, the combined data can be directly transferred to the corresponding database to which the combined data should be stored.

[0107] For application scenarios where there is composite combined data, all single combined data can be first transferred to the corresponding database to which the combined data should be stored, then the composite combined data is migrated to the database to which the combined data should be stored, and then a data deduplication strategy is executed to release storage space.

[0108] In an example, the data deduplication strategy includes sequentially performing the following operations: when the same database simultaneously stores composite combined data and single combined data contained therein, the single combined data is deleted; when a combined data is stored in different databases as single combined data and composite combined data respectively, and the storage optimization level of the composite combined data is higher than that of the single combined data, the single combined data is deleted; when all single combined data in the composite combined data is stored in a database with a higher storage optimization level than the composite combined data, the composite combined data is deleted.

[0109] Based on the above dynamic storage optimization method, each combined data can be migrated in different databases following the change of the storage optimization index, and the data access efficiency can be improved by disassembling and assembling the composite combined data, and the storage space can be released in time.

[0110] Therefore, in the dynamic storage management method of the Internet of Things device data provided by the embodiment of the application, the management platform divides the newly received device data into combined data and non-combined data by the query range and the dynamic characteristics of the data, so as to effectively distinguish the dynamic data and the static data of the device, and then realize classified storage, thereby effectively reducing the number of repeated storage of non-combined data and saving storage space; in combination with the storage optimization index of each combined data, the management platform selects a suitable storage space for the combined data, so that the combined data with a larger access probability is stored in a database with a faster access response, thereby optimizing the data access efficiency and improving the user experience.

[0111] Furthermore, the storage dynamic optimization of the target combined data is triggered in real time according to the data query request, and the related single combined data and composite combined data can be synchronously managed, so that the storage form of the related combined data is more reasonable.

[0112] In addition, by periodically recalculating the storage optimization index of all combined data in the database, the current database to be stored is determined, and the corresponding data migration and data deduplication operations are performed, so that the composite combined data can be effectively managed, the storage space is released as much as possible under the premise of ensuring the access rate, and the storage cost is saved.

[0113] In the embodiment of the application, please refer to Figure 6 The method for obtaining the query result specifically includes the following steps.

[0114] S601, determine the target query range pointed by the data query request, and determine the query combination to be searched according to the target query range.

[0115] In specific implementation, the user can submit a data query request based on a data query interface provided by an application client, therefore, the management platform can quickly determine the target query range according to the data query interface identifier.

[0116] In addition, the management platform can also determine the target query range according to the query requirement information carried in the data query request.

[0117] The management platform can determine the query combination to be searched in the current query process according to the target query range, including single combination and composite combination.

[0118] For example, the query combination corresponding to the query range S2 includes the data combination 1, the data combination 2, and a composite combination formed by merging the data combination 1 and 2.

[0119] It is worth noting that in the embodiments of the present application, the device data is stored in different databases in a classified manner, and migration can occur at any time. Therefore, in order to reduce the burden of generating query statements of the front-end page, the application client can directly record the query requirements submitted by the user in the corresponding search file, and follow the data query request.

[0120] The management platform can know the specific query requirements, including the fields required to be queried and the filtering conditions, by analyzing the search file in the request. The search file can be saved in the format of Jason or XML, so that the management platform can accurately extract the query requirements.

[0121] S602, predicting a storage optimization index range corresponding to the query result based on the query requirements and the query combination.

[0122] In implementation, the storage optimization index range corresponding to the query result when the query result is stored in the form of each query combination needs to be predicted.

[0123] In an example, when the query combination includes single combinations and composite combinations, the management platform can statistically analyze the historical storage optimization indexes of each data combination, and obtain the maximum value and the minimum value of the historical storage optimization indexes under each data combination. Then, the maximum value in the maximum values corresponding to each data combination and the minimum value in the minimum values are taken as the storage optimization index range corresponding to the query result range.

[0124] In another example, if the query combination does not include composite combinations, the storage optimization index ranges corresponding to different combination data in the query result range can be determined according to the maximum values and the minimum values of the historical storage optimization indexes of each single combination.

[0125] S603, matching the search database according to the storage optimization index range.

[0126] In implementation, the maximum value and the minimum value in the storage optimization index range can be matched to the corresponding storage database, and the matching manner of the storage database can refer to the above.

[0127] The matched storage database and all databases whose storage optimization levels are located between the matched storage databases are determined as the search database.

[0128] S604, constructing a query statement based on the query requirements, the query combination, and the search database.

[0129] In an embodiment, the management platform can create a data table according to the data combination, for classified storage of the combined data, so that the data table required by the query can be determined based on the query combination, and a query statement corresponding to the query requirement can be constructed according to the syntax requirement of the search database.

[0130] S605, the query statement is executed to obtain a query result.

[0131] In an embodiment, the query statement can be synchronously sent to the corresponding search database for query, and the query result returned by the database can be received.

[0132] In another embodiment, the query result can be obtained in sequence from high to low according to the storage optimization level corresponding to the search database, and the query can be stopped when the total number of query results reaches the upper limit of display, so that the query result of the search database with a higher storage optimization level can be preferentially displayed.

[0133] It can be understood that in the process of executing the query statement, the query of non-combined data can be involved, for example, the device identifier needs to be queried according to the device name, or the device identifier needs to be queried according to the monitoring object. In an example, the query of non-combined data can be directly queried from the cloud database. In another example, the current storage database can be determined according to the historical access times of the non-combined data, and the current storage database is correspondingly queried.

[0134] Based on the above technical solutions, the comprehensive and rapid acquisition of the query result can be realized.

[0135] In addition, the electronic device provided in the embodiments of the present application includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. The program or instruction is executed by the processor to implement the method in any one of the embodiments of the present application. The processor can be a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits, for executing related programs to implement the method in any one of the embodiments of the present application.

[0136] The processor can also be an integrated circuit electronic device with signal processing capability. In the implementation process, each step of the method in any one of the embodiments of the present application can be completed by the integrated logic circuit of hardware in the processor or the instruction in the form of software.

[0137] The above processor can also be a general processor, a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as hardware code processor execution or executed by a combination of hardware and software modules in the code processor.

[0138] The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the storage memory, and the processor reads the information in the storage memory, and combines the functions required by the units included in the device for data processing of the embodiments of the present application to be executed by the hardware, or executes the method in any one of the implementation manners of the embodiments of the present application.

[0139] Another embodiment of the present application relates to a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to realize the above-mentioned method embodiments.

[0140] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by programs instructing related hardware. The programs are stored in a storage medium, and include a plurality of instructions for making a device (which can be a single chip microcomputer, a chip, etc.) or a processor execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk and various program code storage media.

[0141] The above are the preferred embodiments of the present application, not limited to the protection scope of the present application, therefore: all equivalent changes made on the basis of the structure, shape, principle of the present application should be covered in the protection scope of the present application.

Claims

1. A dynamic storage management method for Internet of Things device data, characterized in that, The method is applied to an Internet of Things data management platform and comprises the following steps: In response to a data query request, a query result corresponding to the data query request is obtained, and the query result comprises at least one target combined data; A storage optimization index of the target combined data is calculated to obtain a target storage optimization index; the storage optimization index is used to predict the access frequency of combined data; A first database for storing the target combined data is determined based on the target storage optimization index; In a case where the target combined data is composite combined data, the target combined data is disassembled to obtain a plurality of target single combined data, and a storage optimization index of each target single combined data is calculated to determine a storage database of each target single combined data; The target single combined data whose storage optimization level of the storage database is higher than that of the first database is migrated to the corresponding storage database, and the target single combined data whose storage optimization level of the storage database is not higher than that of the first database is deleted; The target combined data is migrated to the first database.

2. The method of claim 1, wherein, The storage method of the target combined data comprises the following steps: Receiving device data; Disassembling the device data to obtain the target combined data; Calculating a first storage optimization index of the target combined data; Determining a storage database and a storage mode of the target combined data according to the first storage optimization index; Storing the target combined data in the storage database according to the storage mode.

3. The method of claim 2, wherein, The method further comprises the following steps: Disassembling the device data to obtain non-combined data; Storing the non-combined data in a cloud database.

4. The method of claim 2, wherein, The method further comprises the following steps: Periodically calculating a second storage optimization index of the target combined data; Determining a storage database of the target combined data based on the second storage optimization index; Migrating the target combined data to the storage database.

5. The method of claim 4, wherein, When the target combined data is composite combined data, the migrating of the target combined data to the storage database comprises the following steps: After all single combined data are migrated; Migrating the target combined data to the storage database; Executing a data deduplication strategy.

6. The method of claim 5, wherein, The execution of the data deduplication strategy comprises the following steps: When composite combined data and single combined data contained in the composite combined data exist in the same database at the same time, the single combined data contained in the composite combined data is deleted; when the same combined data is stored in different databases as single combined data and composite combined data respectively, and the storage optimization level of the database storing the composite combined data is higher than that of the single combined data, the single combined data is deleted; when all single combined data in the composite combined data are stored in a database with a higher storage optimization level than the composite combined data, the composite combined data is deleted.

7. The method of claim 1, wherein, The method for obtaining the query result corresponding to the data query request comprises the following steps: A target query range pointed by the data query request is determined, and a query combination required to be searched is determined according to the target query range. predicting a storage optimization index range corresponding to the query result based on the query requirement and the query combination; predicting a storage optimization index range corresponding to the query result based on the data query requirement; matching a retrieval database according to the storage optimization index range; constructing a query statement based on the query requirement, the query combination and the retrieval database; executing the query statement to obtain the query result.

8. An Internet of Things data management platform, characterized by, The platform is used to implement the method in any one of claims 1 to 7.

9. An electronic device, comprising: comprising a processor, a memory, and a program or instructions stored on the memory and executable on the processor, the program or instructions being executed by the processor to implement the method in any one of claims 1 to 7.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the method in any one of claims 1 to 7.

Images