Data acquisition method, device and Internet of Things

Abstract

The present application discloses a data acquisition method, device and Internet of Things, which belong to the field of communications. The method can determine a first keyword based on a first data type and a first device identifier of a first Internet of Things device, and can determine at least one index block with the first keyword as an index, and then obtain a data block according to a storage position in the index block. Since there is no need to compare the device identifier of the first Internet of Things device with the device identifiers of multiple stored Internet of Things devices one by one, the efficiency of data acquisition is effectively improved.

Description

Technical Field

[0001] The present disclosure relates to the field of communications, and in particular to a data acquisition method, device and Internet of Things. Background Art

[0002] The Internet of Things usually includes multiple Internet of Things devices, data management devices and data analysis devices. The multiple Internet of Things devices are used to transmit the collected data to the data management device. The data management device is used to uniformly manage the data sent by multiple Internet of Things devices. The data analysis device is used to obtain data from the data management device and analyze the obtained data.

[0003] In the related art, the data analysis device can send a data acquisition request to the data management device, and the data acquisition request can carry the device identification of the specified IoT device. After receiving the data acquisition request, the data management device can compare the device identification with the device identifications of multiple IoT devices stored in it one by one, so as to determine the specified IoT device. After that, the data management device can transmit the data collected by the specified IoT device to the data analysis device.

[0004] However, the above data acquisition method is inefficient. Summary of the invention

[0005] The embodiments of the present disclosure provide a data acquisition method, device and Internet of Things, which can solve the problem of low efficiency of data acquisition methods in related technologies. The technical solution is as follows:

[0006] On the one hand, a data acquisition method is provided, which is applied to a data management device in the Internet of Things, wherein a plurality of data blocks are stored in the data management device, and each of the data blocks contains data of a data type collected by an Internet of Things device; the method comprises:

[0007] Obtain a data acquisition request for a first Internet of Things device, where the data acquisition request carries a first device identifier of the first Internet of Things device and a first data type of data to be acquired;

[0008] Based on the first data type and the first device identifier, determining a first keyword corresponding to the first data type and the first device identifier;

[0009] Determine at least one first index block corresponding to the first keyword, each of the first index blocks recording a storage position of a data block in a file;

[0010] At least one target data block is obtained from the file according to a storage position in the at least one first index block, each of the target data blocks storing data of the first data type collected by the first Internet of Things device.

[0011] Optionally, the data acquisition request further carries a target time period, and each of the first index blocks further records a collection time range of data stored in a data block; acquiring at least one target data block from the file according to a storage position in the at least one first index block includes:

[0012] Determine at least one target index block from the at least one first index block according to the target time period and the acquisition time range in each of the first index blocks, wherein the acquisition time range recorded in the target index block overlaps with the target time period;

[0013] At least one target data block is obtained from the file according to a storage position in the at least one target index block.

[0014] Optionally, the determining, based on the first data type and the first device identifier, a first keyword corresponding to the first data type and the first device identifier includes:

[0015] Determine, based on the first data type, from a measurement block device identification indexes of a plurality of candidate Internet of Things devices corresponding to the first data type, wherein the measurement block records a correspondence between the data type and the device identification indexes of the Internet of Things devices;

[0016] Determine, based on the first device identifier, a device identifier index of the first Internet of Things device corresponding to the first device identifier from among the device identifier indexes of the multiple candidate Internet of Things devices;

[0017] Determine a storage location of the first keyword in a sequence block based on a device identification index of the first Internet of Things device, wherein the sequence block records a plurality of keywords;

[0018] The first keyword is obtained from the sequence block based on a storage position of the first keyword in the sequence block.

[0019] Optionally, the method further includes:

[0020] Receive data information sent by a second Internet of Things device, where the data information includes data collected by the second Internet of Things device, a second device identifier of the second Internet of Things device, and a second data type of the data collected by the second Internet of Things device;

[0021] If the remaining storage capacity of a first data block among the multiple data blocks is greater than or equal to the storage capacity threshold, storing the data collected by the second IoT device in the first data block;

[0022] Generate a second keyword, and record a correspondence between the second keyword and the second index block, where the second index block records a storage position of the first data block in the file;

[0023] The second keyword is stored in the sequence block, and the storage position of the second keyword in the sequence block is determined by the device identification index corresponding to the second device identification.

[0024] Optionally, after acquiring the data in the at least one target data block, the method further includes:

[0025] If the length of the target linked list stored in the memory of the data management device is less than or equal to the length threshold, deleting the corresponding relationship at the tail of the target linked list;

[0026] The correspondence between the first data type, the first device identifier and the first keyword is stored at the head of the target linked list.

[0027] Optionally, obtaining a data acquisition request for the first IoT device includes:

[0028] Receiving a data acquisition request for the first Internet of Things device sent by a data analysis device;

[0029] After acquiring the data in the at least one target data block, the method further includes:

[0030] The acquired data is sent to the data analysis device.

[0031] On the other hand, a data acquisition device is provided, which is applied to a data management device in the Internet of Things, wherein a plurality of data blocks are stored in the data management device, each of which contains data of a data type collected by an Internet of Things device; the device comprises:

[0032] A first acquisition module, configured to acquire a data acquisition request for a first IoT device, wherein the data acquisition request carries a first device identifier of the first IoT device and a first data type of data to be acquired;

[0033] A first determining module, configured to determine, based on the first data type and the first device identifier, a first keyword corresponding to the first data type and the first device identifier;

[0034] A second determining module, configured to determine at least one first index block corresponding to the first keyword, each of which records a storage position of a data block in a file;

[0035] A second acquisition module is used to acquire at least one target data block from the file according to a storage position in the at least one first index block, each of the target data blocks storing data of the first data type collected by the first Internet of Things device.

[0036] On the other hand, a computer-readable storage medium is provided, in which a program code is stored. The program code is loaded and executed by a processor to implement the data acquisition method as described in the above aspects.

[0037] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores program code, and the program code is loaded and executed by the processor to implement the data acquisition method as described in the above aspects.

[0038] On the other hand, a computer program product comprising instructions is provided. When the computer program product is run on the computer, the computer is enabled to execute the data acquisition method described in the above aspects.

[0039] In yet another aspect, an Internet of Things is provided, the Internet of Things comprising: a data management device, at least one Internet of Things device, and a data analysis device;

[0040] The data management device comprises the data acquisition device as described in the above aspect;

[0041] Each of the IoT devices is used to send data information to the data management device, where the data information includes data collected by the IoT device, a device identifier of the IoT device, and a data type of the data collected by the IoT device;

[0042] The data analysis device is used to send a data acquisition request for the first Internet of Things device to the data management device, and receive data sent by the data management device.

[0043] The beneficial effects brought by the technical solution provided by the embodiments of the present disclosure include at least:

[0044] The disclosed embodiments provide a data acquisition method, device and Internet of Things, which can determine a first keyword based on a first data type and a first device identifier of a first Internet of Things device, and can determine at least one index block using the first keyword as an index, and then acquire a data block according to a storage position in the index block. Since there is no need to compare the device identifier of the first Internet of Things device with the device identifiers of multiple stored Internet of Things devices one by one, the efficiency of data acquisition is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 is a schematic diagram of an Internet of Things provided by an embodiment of the present disclosure;

[0047] Figure 2 is a flow chart of a data acquisition method provided by an embodiment of the present disclosure;

[0048] Figure 3 is a flow chart of another data acquisition method provided by an embodiment of the present disclosure;

[0049] Figure 4 is a schematic diagram of a file provided by an embodiment of the present disclosure;

[0050] Figure 5 is a schematic diagram of a data block provided by an embodiment of the present disclosure;

[0051] Figure 6 is a schematic diagram of an index block provided by an embodiment of the present disclosure;

[0052] Figure 7 is a schematic diagram of another index block provided by an embodiment of the present disclosure;

[0053] Figure 8 is a schematic diagram of data stored in a data management device provided by an embodiment of the present disclosure;

[0054] Fig. 9 is a schematic diagram of determining at least one target data block provided by an embodiment of the present disclosure;

[0055] Fig.10 is a block diagram of a data acquisition device provided by an embodiment of the present disclosure;

[0056] Fig.11 is a block diagram of another data acquisition device provided by an embodiment of the present disclosure;

[0057] Fig.12 It is a schematic diagram of a data management device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0059] Figure 1 is a schematic diagram of an Internet of Things provided by an embodiment of the present disclosure, such as Figure 1 As shown, the Internet of Things may include at least one Internet of Things device 10, a data management device 20 and a data analysis device 30. Figure 1 The example in which the Internet of Things includes three Internet of Things devices 10 is used for illustration.

[0060] Among them, a connection can be established between at least one IoT device 10 and the data management device 20, and between the data management device 20 and the data analysis device 30 via a wired network or a wireless network.

[0061] Each IoT device 10 may be a device capable of data collection, for example, a camera, a printer, or a sensor disposed on a wind turbine generator set. The IoT device 10 is used to transmit the real-time collected data to the data management device 20 .

[0062] The data management device 20 may be a terminal for operation by a management personnel. For example, it may be a computer. The data management device 20 may uniformly manage the data collected by at least one IoT device 10, and after receiving a data acquisition request sent by the data analysis device 30, send the data acquired based on the data acquisition request to the data analysis device 30.

[0063] After receiving the data sent by the IoT device, the data management device 20 can store the data in a data block. The data management device 20 stores a plurality of data blocks, each of which contains data of a data type collected by an IoT device.

[0064] The data analysis device 30 may be a device capable of performing data analysis on data. For example, it may be a server, or a server cluster consisting of several servers, or a cloud computing service center. For example, if the data received by the data analysis device 30 is the pitch angle of a wind turbine generator set and the active power of the wind turbine generator set, the data analysis device 30 may analyze the received multiple active powers and multiple pitch angles to predict the power generation of the wind turbine generator set in the future.

[0065] Figure 2 is a flow chart of a data acquisition method provided by an embodiment of the present disclosure. The method can be applied to Figure 1 The data management device 20 in the Internet of Things shown. Figure 2 As shown, the method may include:

[0066] Step 201: Obtain a data acquisition request for a first IoT device.

[0067] In the disclosed embodiment, the data acquisition request may be sent by a data analysis device or may be triggered by a user on a data management device. The data acquisition request carries a first device identifier of a first IoT device and a first data type of the data to be acquired. The first IoT device may be any IoT device among the at least one IoT device mentioned above.

[0068] Step 202: Based on the first data type and the first device identifier, determine a first keyword corresponding to the first data type and the first device identifier.

[0069] After obtaining the data acquisition request for the first IoT device, the data management device may determine a first keyword (key) corresponding to the first data type and the second device identifier based on the first data type and the first device identifier carried in the data acquisition request.

[0070] Step 203: Determine at least one first index block corresponding to the first keyword.

[0071] After acquiring the first keyword, the data management device may determine at least one index block corresponding to the first keyword, and each first index block records a storage position of a data block in the file.

[0072] Step 204: Obtain at least one target data block from the file according to a storage position in at least one first index block.

[0073] After acquiring at least one first index block, the data management device can acquire at least one target data block from the file according to the storage position in the at least one first index block, wherein each target data block stores data of the first data type collected by the first Internet of Things device.

[0074] In summary, the disclosed embodiment provides a data acquisition method, which can determine a first keyword based on a first data type and a first device identifier of a first IoT device, and can determine at least one index block using the first keyword as an index, and then acquire a data block according to a storage position in the index block. Since there is no need to compare the device identifier of the first IoT device with the device identifiers of multiple IoT devices stored one by one, the efficiency of data acquisition is effectively improved.

[0075] Figure 3 is a flow chart of another data acquisition method provided by an embodiment of the present disclosure. This method can be applied to Figure 1 As shown in the Internet of Things. Figure 3 As shown, the method may include:

[0076] Step 301: The second IoT device sends data information to the data management device.

[0077] In the embodiment of the present disclosure, the second Internet of Things device is any Internet of Things device among at least one Internet of Things device in the Internet of Things.

[0078] After collecting the data, the second IoT device may use the data, the second device identifier of the second IoT device, and the second data type of the data collected by the second IoT device as data information. Then, the second IoT device may send the data information to the data management device. The data information may include the data collected by the second IoT device, the second device identifier of the second IoT device, and the second data type of the data collected by the second IoT device.

[0079] Optionally, the second device identifier may include a second device number, and the second data type may be rotation speed, temperature, humidity, or the like.

[0080] For example, assuming that the second device identifier includes the second device number assetId, the second device number assetId can be UufSeRam, that is, UufSeRam is the content of the second device number assetId, that is, assetId = UufSeRam. If the second data type is rotation speed, the second data type can be represented by TSP@p1, and the data collected by the second IoT device can be 29729.

[0081] Step 302: The data management device stores the data information sent by the second IoT device.

[0082] In the embodiment of the present disclosure, at least one file, a plurality of data blocks and index blocks corresponding to the plurality of data blocks are stored in the data management device. The file may be a word file or an excel file, etc.

[0083] refer to Figure 4 , each file can store multiple data blocks and multiple index blocks. Optionally, each file can store at most a threshold number of data blocks and a threshold number of index blocks. Each data block corresponds to an index block, and each index block corresponds to a file. The storage capacity of different files can be the same, the storage capacity of different data blocks can also be the same, and the storage capacity of different index blocks can also be the same.

[0084] Each data block is used to store data of one data type of the same IoT device. Figure 5 , each data block contains the encoding type of data collected by an IoT device, a data set of multiple data collected by an IoT device at different times, the length of the data block, that is, the total amount of all data in the data block, and the collection time of each data, that is, the timestamp of each data. Optionally, the encoding type can be an integer (integer, int) type or a char (character) type, etc.

[0085] refer to Figure 6 Each index block may record the length of a corresponding data block, the acquisition time range of the data stored in the data block (eg, the minimum timestamp and maximum timestamp of the data stored in the data block), and the storage location of the data block in the file.

[0086] For example, if the current storage capacity of the data block is 10 megabytes (M), and the data block is the first data block in the file, the storage position of the data block in the file can be 0 to 10M. If the data block is the second data block in the file, and the storage position of the first data block is 0 to 20M, then the storage position of the data block in the file is 21M to 31M.

[0087] Optional, reference Figure 7 Each index block may include an index entry and index block meta information. The index entry may include: the length of the data block, the acquisition time range of the data stored in the data block, and the storage location of the data block in the file. The index block meta information may include the length of the data block, etc.

[0088] Optional, reference Figure 4 Each file also stores a first position block, which records the storage position of each index block in the file. The first position block can also be called a trailer block. Figure 8 , the data management device can record the correspondence between the first position block and each index block in a series sketch.

[0089] The following describes the process of storing the data information sent by the second IoT device by the data management device:

[0090] After receiving the data information sent by the second Internet of Things device, if the data management device determines that the remaining storage capacity of the first data block among the multiple stored data blocks is greater than or equal to the storage capacity threshold, the data management device can determine that the number of data blocks stored in the first file of at least one file is less than the number threshold, and the remaining storage capacity of the first data block in the first file is greater than or equal to the storage capacity threshold, that is, the first data block in the first file can store data of the second data type, so the data management device can execute the following steps A1 to A3.

[0091] Among them, the first data block is a data block for storing data of the first data type. The number threshold and the storage capacity threshold are numerical values ​​pre-stored in the data management device. The first file is a file in at least one file whose creation time is less than the creation time of other files, and the creation time refers to the time from the moment when the data management device creates the file to the current moment. The other files are files other than the first file in at least one file, and the number of data blocks stored in the other files is equal to the number threshold, and the remaining storage capacity of each data block stored in the other files is less than the storage capacity threshold. That is, no new data blocks can be created in the other files, and no new data can be stored in the data blocks that have been created in the other files.

[0092] If the remaining storage capacity of each of the multiple data blocks is less than the storage capacity threshold, the data management device can determine that the number of data blocks stored in each of the at least one file is equal to the number threshold, and the remaining storage capacity of each data block stored in the file is less than the storage capacity threshold. That is, no new data block can be created in each of the at least one file, and no new data can be stored in the multiple data blocks that have been created in the file. Therefore, the data management device can execute the following steps B1 to B3.

[0093] The process from step A1 to step A3 is described below:

[0094] A1. If the remaining storage capacity of a first data block among multiple data blocks stored in the data management device is greater than or equal to the storage capacity threshold, the data management device may store the data collected by the second IoT device in the first data block.

[0095] After the data management device receives the data information sent by the second Internet of Things device, if the remaining storage capacity of the first data block among the multiple data blocks stored in the data management device is greater than or equal to the storage capacity threshold, the data management device can store the data collected by the second Internet of Things device in the first data block.

[0096] A2. Generate a second keyword, and record the corresponding relationship between the second keyword and the second index block.

[0097] After storing the data collected by the second IoT device in the first data block, the data management device may generate a second keyword and record the corresponding relationship between the second keyword and the second index block, wherein the second index block records the storage location of the first data block in the file.

[0098] Optionally, the data management device may also randomly generate the second keyword. Alternatively, the data management device may generate the second keyword based on the second device identifier and the second data type. If the second device identifier includes the second device number, the data management device may generate the second keyword based on the second device number and the second data type.

[0099] For example, assuming that the second device number assetId is UufSeRam and the second data type is TSP@p1, the second keyword may be: TSP@p1, assetID=UufSeRam.

[0100] The data information sent by the second IoT device may further include the time at which the data is collected by the second IoT device, and the data management device may further store the time at which the data is collected in the first data block.

[0101] After storing the data collected by the second IoT device in the first data block, the data management device may also store the encoding type of the data in the first data block, and may update the length of the first data block stored in the first data block and the second index block.

[0102] After the data management device stores the data collected by the second Internet of Things device, the time when the data is collected, and the encoding type of the data in the first data block, and updates the length of the first data block stored, it can also update the storage position of the first data block recorded in the second index block in the first file based on the current storage capacity of the first data block.

[0103] For example, before the data management device receives the data information sent by the second IoT device, the storage position of the first data block recorded in the second index block in the first file is 0 to 10M. After the data management device updates the first data block based on the data information sent by the second IoT device, the storage capacity of the first data block is 12M, and the data management device updates the storage position of the first data block recorded in the second index block from 0 to 10M to 0 to 12M.

[0104] Furthermore, the data management device may also update the maximum timestamp of the first data block recorded in the second index block to the collection time of the data.

[0105] Optional, reference Figure 8, the data management device can record the correspondence between the second keyword and the second index block in the sequence overview.

[0106] A3. Store the second keyword in the sequence block.

[0107] refer to Figure 8 After generating the second keyword, the data management device may store the second keyword in a series block, and the storage position of the second keyword in the sequence block is determined by the device identification index corresponding to the second device identification.

[0108] The data management device may determine a first keyword index corresponding to the device identification index of the first IoT device using a hash algorithm based on the device identification index of the second device identification. The first keyword index is used to indicate a storage location of the first keyword in the sequence block.

[0109] Alternatively, after storing the second keyword in the sequence block, the data management device may generate a second keyword index for the second keyword based on the maximum value of the stored keyword indexes. Figure 8 , the data management device can record the correspondence between the device identification index corresponding to the second device identification and the second keyword index in a series identification set (series id set), that is, the correspondence between the device identification index and the keyword index is recorded in the series identification set. Each keyword index is used to indicate the storage position of the corresponding keyword in the sequence block, and the keyword index is a number. The series identification set can be an efficient compression bitmap, such as a roaring bitmap.

[0110] For example, if a sequence block has stored five keywords, namely, a first keyword, a second keyword, a third keyword, a fourth keyword, and a fifth keyword, the keyword indexes of the five keywords are 1, 2, 3, 4, and 5, respectively. The data management device may generate a second keyword index of the second keyword based on the maximum value 5 of the five keyword indexes, for example, the second keyword index 6 may be obtained by adding 1 to the maximum value 5 of the five keyword indexes.

[0111] Among them, the keyword index 1 is used to indicate that the storage position of the corresponding first keyword in the sequence block is 1, the keyword index 2 is used to indicate that the storage position of the corresponding second keyword in the sequence block is 2, the keyword index 3 is used to indicate that the storage position of the corresponding third keyword in the sequence block is 3, the keyword index 4 is used to indicate that the storage position of the corresponding fourth keyword in the sequence block is 4, and the keyword index 5 is used to indicate that the storage position of the corresponding fifth keyword in the sequence block is 5.

[0112] The process from step B1 to step B4 is described below:

[0113] B1. If the remaining storage capacity of each data block is less than the storage capacity threshold, the data management device can create a second data block and store the data collected by the second IoT device in the first data block.

[0114] After determining that the remaining storage capacity of each data block in at least one file is less than the storage capacity threshold, the data management device can create a second file and a second data block in the second file. The data management device can then store the data collected by the second IoT device in the second data block. The second data block is used to store data of the second data type collected by the second IoT device.

[0115] Optionally, after storing the data collected by the second Internet of Things device in the second data block, the data management device may store both the encoding type of the data and the length of the second data block in the second data block.

[0116] If the data information sent by the second Internet of Things device also includes the collection time of the data collected by the second Internet of Things device, the data management device may also store the collection time of the data in the second data block.

[0117] B2. Create a third index block for the second data block, and record the storage location of the second data block in the second file in the third index block.

[0118] After storing the data collected by the second IoT device in the second data block, the data management device may create a third index block of the second data block in the second file, and may record the length of the second data block in the third index block.

[0119] If the data information sent by the second IoT device also includes the collection time of the data collected by the second IoT device, the data management device can also record the collection time of the data as the minimum timestamp and maximum timestamp of the second data block in the third index block.

[0120] Furthermore, the data management device may also record the storage position of the second data block in the second file in the third index block based on the current storage capacity of the second data block.

[0121] For example, if the current storage capacity of the second data block is 1M, the storage position of the second data block recorded by the data management device in the third index block in the second file may be 0 to 1M.

[0122] B3. Generate a third keyword, and record the corresponding relationship between the third keyword and the third index block.

[0123] After recording the storage location of the second data block in the second file in the third index block, the data management device may generate a third keyword and may record the corresponding relationship between the third keyword and the third index block.

[0124] The specific implementation process of step B3 can refer to the above step A2, and the embodiment of the present disclosure will not be repeated here.

[0125] B4. Store the third keyword in the sequence block.

[0126] After generating the third keyword, the data management device may store the third keyword in a series block. The storage position of the third keyword in the sequence block is determined by the device identification index corresponding to the second device identification.

[0127] The specific implementation process of step B4 can refer to the above step A3, and the embodiment of the present disclosure will not be repeated here.

[0128] In the embodiments of the present disclosure, reference Figure 8 The data management device may also store a measurement block, a second position block and a tag block, and the second position block may also be called a trailer block.

[0129] The measurement block is used to record the correspondence between the data type and the device identification index of the IoT device, and the second location block is used to record the size of the measurement block, the size of the sequence block, the size of the label block, and the storage location of the measurement block, the sequence block and the label block in the data management device. The label block is used to record the device identification of the IoT device. Multiple keywords are recorded in the sequence block.

[0130] If the device identification of each IoT device includes a device number, the corresponding relationship recorded in the measurement block may be: a corresponding relationship between a data type, a device number index of the IoT device, and a content index of the device number.

[0131] The content index of the device number is the index of the content of the device number, and the device number index is the index of the device number. Both the content index and the device number index can be numbers.

[0132] Step 303: The data management device receives a data acquisition request for the first IoT device sent by the data analysis device.

[0133] In the disclosed embodiment, the data acquisition request may be sent by a data analysis device or may be triggered by a user on a data management device. The data acquisition request carries a first device identifier of a first IoT device and a first data type of data to be acquired. The first IoT device may be any IoT device among the at least one IoT device, and the first IoT device and the second IoT device may be the same device or different devices. The first device identifier may include a first device number.

[0134] For example, the first data type may be TS1@p1, assuming that the first device identifier includes a first device number, and the first device number assetId is UufSeRam.

[0135] Optionally, the code for the data acquisition request can be as follows:

[0136] select*from ot-Raw."TS1@p1"where assetId="UufSeRam"limit3.

[0137] Here, ot-Raw is the name of the database in the data management device, and the “*” indicates obtaining all parameters related to “TS1@p1”.

[0138] Step 304: The data management device determines a first keyword corresponding to the first data type and the first device identifier based on the first data type and the first device identifier.

[0139] After receiving the data acquisition request for the first Internet of Things device sent by the data analysis device, the data management device can determine the first keyword corresponding to the first data type and the first device identifier based on the first data type and the first device identifier carried in the data acquisition request.

[0140] The process of the data management device determining the first keyword corresponding to the first data type and the first device identifier may include the following steps C1 to C4:

[0141] C1. The data management device can determine the device identification indexes of multiple candidate Internet of Things devices corresponding to the first data type from the measurement block based on the first data type, and determine the device identification index of the first Internet of Things device corresponding to the first device identification from the device identification indexes of the multiple candidate Internet of Things based on the first device identification.

[0142] The data management device can determine the device identification indexes of multiple candidate Internet of Things devices corresponding to the first data type from the measurement block based on the first data type, and determine the device identification index of the first Internet of Things device corresponding to the first device identification from the device identification indexes of multiple candidate Internet of Things devices based on the first device identification.

[0143] Optionally, if the device identification of the first IoT device includes the first device number, the correspondence recorded in the measurement block may be: the correspondence between the data type, the device number index of the IoT device, and the content index of the device number. The data management device may determine multiple device number indexes corresponding to the first data type from the measurement block based on the first data type, and may determine multiple content indexes of candidate IoT devices corresponding to the target device number index from the measurement block based on the target device number index in the multiple device number indexes. The data management device may then determine the target content index in the content indexes of the multiple candidate IoT devices as the device identification index of the first IoT device. The target device number index is the index of the first device number, and the target content index is the index of the content of the first device number.

[0144] For example, if the first data type is TS1@p1 and the first device number assetId is UufSeRam, the data management device can first determine multiple content indexes corresponding to the first data type TS1@p1 from the measurement block, and can determine the content indexes of multiple candidate IoT devices corresponding to the target device number index from the measurement block based on the target device number index of the first device number assetId. Then, the data management device can determine the target content index among the content indexes of the multiple candidate IoT devices as the device identification index of the first IoT device. The target content index is the index of the content UufSeRam of the first device number assetId.

[0145] C2. The data management device determines a storage position of the first keyword in the sequence block based on the device identification index of the first Internet of Things device.

[0146] After determining the device identification index of the first Internet of Things device, the data management device may determine the storage position of the first keyword in the sequence block based on the device identification index of the first Internet of Things device.

[0147] Optionally, after determining the device identification index of the first IoT device, the data management device may determine, based on the device identification index of the first IoT device, a first keyword index corresponding to the device identification index of the first IoT device from the correspondence between the device identification index and the keyword index in the sequence identification set. The data management device may then determine the storage location of the first keyword in the sequence block based on the first keyword index. The first keyword index is used to indicate the storage location of the first keyword in the sequence block.

[0148] Alternatively, after determining the device identification index of the first IoT device, the data management device may determine the first keyword index corresponding to the device identification index of the first IoT device using a hash algorithm based on the device identification index of the first IoT device. The data management device may then determine the storage location of the first keyword in the sequence block based on the first keyword index.

[0149] In the disclosed embodiment, the data management device may determine multiple device identification indexes related to the first data type based on the first data type, and determine content indexes of multiple candidate IoT devices based on the target device identification index in the multiple device identification indexes. The data management device may then determine the content index of the first device number (i.e., the target content index) as the device identification index of the first IoT device. The data management device may further determine a first keyword index based on the device identification index of the first IoT device, and finally determine the storage location of the first keyword in the sequence block based on the first keyword index.

[0150] Since the first device number, the content of the first device number and the first keyword are all character strings, the content index, the device number index and the keyword index are all numbers. The method provided by the embodiment of the present disclosure determines the first keyword based on the device number index, the content index and the keyword index. Compared with comparing the first device number with the multiple device numbers stored therein one by one, comparing the content of the first device number with the content of the multiple device numbers stored therein one by one, and comparing the first keyword with the multiple pre-stored keywords one by one, the method provided by the embodiment of the present disclosure effectively improves the efficiency of keyword determination, thereby effectively improving the efficiency of data acquisition.

[0151] C3. The data management device obtains the first keyword from the sequence block based on the storage position of the first keyword in the sequence block.

[0152] After determining the first keyword, the data management device may obtain the first keyword from the sequence block based on the storage position of the first keyword in the sequence block.

[0153] For example, if the first keyword index is 1, the data management device may determine that the storage position of the first keyword in the sequence block is 1, and the data management device may determine the first keyword among the multiple keywords recorded in the sequence block as the first keyword.

[0154] Step 305: The data management device obtains at least one target data block from multiple data blocks based on the first keyword.

[0155] After acquiring the first keyword, the data management device may acquire at least one target data block from the plurality of data blocks based on the first keyword, wherein each target data block stores data of the first data type collected by the first Internet of Things device.

[0156] In the embodiment of the present disclosure, the data management device can determine at least one first index block corresponding to the first keyword, and can obtain the target data block from the file according to the storage position in the at least one first index block, wherein each first index block records the storage position of a data block in the file.

[0157] The data management device can determine at least one first index block corresponding to the first keyword from the sequence overview, and can determine the first position block corresponding to each first index block from the sequence overview. Then, the data management device can determine the storage position of the first index block in the corresponding file in the first position block corresponding to the first index block. Furthermore, the data management device can obtain the first index block from the corresponding file based on the storage position of the first index block, and can obtain the target data block from the file based on the storage position of the target data block recorded in the first index block in the file.

[0158] Furthermore, the data acquisition request also carries a target time period. Each first index block records a collection time range of the data stored in a data block. In the process of acquiring at least one target data block from a file according to a storage position in at least one first index block, the data management device can determine at least one target index block from at least one first index block according to the target time period and the collection time range in each first index block. And at least one target data block can be acquired from the file according to the storage position in at least one target index block. The collection time range recorded in the target index block overlaps with the target time period.

[0159] Optionally, the data management device may determine at least one target index block from at least one first index block based on the target time period and the acquisition time range in each candidate index block. Further, the data management device may determine the target data block corresponding to the target index block according to the storage position of the target data block recorded in each target index block in the corresponding file, thereby obtaining at least one target data block.

[0160] Example, reference Fig. 9 The data management device determines three first index blocks corresponding to the first keyword from the sequence overview, and determines two target index blocks from the three first index blocks based on the target time period and the acquisition time range in each candidate index block, and then the data management device determines the target data blocks corresponding to the two target index blocks.

[0161] Step 306: The data management device obtains data in at least one target data block.

[0162] After determining at least one target data block, the data management device may obtain data in the at least one data block. Optionally, the data management device may obtain all data in each target data block. Alternatively, the data management device may obtain data in the target data block whose collection time is within the target time period.

[0163] The data acquired by the data management device may be as shown in Table 1. Referring to Table 1, if the first data type is TS1@p1, TS1@p1 represents the rotation speed, and the three acquired data may include three rotation speed values: 45076, 45076, and 45078. The three data are all data collected by the IoT device with device number assetId UufSeRam at time 1597449603456000000, 1597449608379000000, and 1597449613337000000.

[0164] Table 1

[0165] time assetId value 1597449603456000000 UufSeRam 45076 1597449608379000000 UufSeRam 45077 1597449613337000000 UufSeRam 45078

[0166] In the embodiment of the present disclosure, if the code of the data acquisition request is: select * from ot-Raw. "TS1@p1" where assetId = "UufSeRam" limit3. Then the data management device can also obtain all data related to "TS1@p1". If TS1@p1 is the rotation speed, then the data management device can obtain the value of the rotation speed, the rate of change of the rotation speed, etc.

[0167] Step 307: The data management device sends the acquired data to the data analysis device.

[0168] After acquiring the data in at least one target data block, the data management device may send the acquired data to the data analysis device.

[0169] Step 308: The data analysis device performs data analysis on the data.

[0170] After acquiring the data sent by the data management device, the data analysis device can perform data analysis based on the data. By analyzing the data, the content of the data collected by the first IoT device can be understood in a timely manner, thereby realizing the understanding of the state of the object monitored by the first IoT device.

[0171] For example, the first IoT device monitors a wind turbine generator set. By analyzing the received data, the data analysis device can understand whether the operation of the wind turbine generator set has been abnormal over the past period of time, and / or predict the power generation of the wind turbine generator set in the future.

[0172] In the disclosed embodiment, after the data management device obtains the data in at least one target data block, if the length of the target linked list stored in the memory of the data management device is less than or equal to the length threshold, the data management device deletes the correspondence at the tail of the target linked list, and can store the correspondence between the first data type, the first device identifier, and the first keyword at the head of the target linked list. If the length of the target linked list stored in the memory of the data management device is greater than the length threshold, the data management device can store the correspondence between the first data type, the first device identifier, and the first keyword at the head of the target linked list.

[0173] Alternatively, the data management device may store the correspondence between the first data type, the first device identifier, and the first keyword index at the head of the target linked list.

[0174] This process can be implemented using the following code:

[0175] type TagValueSeriesIDCache struct{

[0176] sync.RWMutex / / read-write lock / /

[0177] cache map[string]map[string]map[string]*list.Element / / Store the correspondence between the first data type, the first device identifier and the first keyword / /

[0178] evictor*list.List / / If the length of the target list is less than or equal to the length threshold, delete the corresponding relationship at the end of the target list / /

[0179] capacity int / / The length of the target linked list / /

[0180] }

[0181] Among them, the first map in the map[string]map[string]map[string]*list.Element stores the data type, the second map stores the device number, the third map stores the content of the device number, and list.Element stores the keyword index or keyword.

[0182] The above implementation process is actually implemented using the least recently used (LRU) algorithm.

[0183] By storing the correspondence between the first data type, the first device identifier and the first keyword in the target linked list, or storing the correspondence between the first data type, the first device identifier and the first keyword index in the target linked list, the data management device can obtain the corresponding keyword (or the corresponding keyword index) from the target linked list after receiving the data acquisition request for the first Internet of Things device again, and then directly determine at least one target data block based on the keyword (or keyword index), thereby improving the efficiency of data acquisition.

[0184] It should be noted that the order of the steps of the data acquisition method provided in the embodiment of the present application can be appropriately adjusted, and the steps can also be deleted according to the situation, for example, step 301 and step 302 can be deleted according to the situation. Any technician familiar with the technical field can easily think of a change within the technical scope disclosed in this application, and it should be included in the protection scope of this application, so it will not be repeated.

[0185] In summary, the disclosed embodiment provides a data acquisition method, which can determine a first keyword based on a first data type and a first device identifier of a first IoT device, and can determine at least one index block using the first keyword as an index, and then acquire a data block according to a storage position in the index block. Since there is no need to compare the device identifier of the first IoT device with the device identifiers of multiple IoT devices stored one by one, the efficiency of data acquisition is effectively improved.

[0186] Fig.10 is a block diagram of a data acquisition device provided by an embodiment of the present disclosure. The data acquisition device 100 can be applied to Figure 1 The data management device 20 in the Internet of Things shown in the figure stores multiple data blocks, each of which contains data of a data type collected by an Internet of Things device. Fig.10 As shown, the device may include:

[0187] The first acquisition module 1001 is used to acquire a data acquisition request for a first IoT device, where the data acquisition request carries a first device identifier of the first IoT device and a first data type of data to be acquired.

[0188] The first determining module 1002 is configured to determine a first keyword corresponding to the first data type and the first device identifier based on the first data type and the first device identifier.

[0189] The first determination module 1003 is used to determine at least one first index block corresponding to the first keyword, each first index block records a storage position of a data block in the file.

[0190] The second acquisition module 1004 is used to acquire at least one target data block from the file according to a storage position in at least one first index block, each target data block storing data of the first data type collected by the first Internet of Things device.

[0191] In summary, the disclosed embodiment provides a data acquisition device, which can determine a first keyword based on a first data type and a first device identifier of a first IoT device, and can determine at least one index block using the first keyword as an index, and then acquire a data block according to a storage position in the index block. Since there is no need to compare the device identifier of the first IoT device with the device identifiers of multiple IoT devices stored one by one, the efficiency of data acquisition is effectively improved.

[0192] Optionally, the second acquisition module 1004 is used to:

[0193] At least one target index block is determined from at least one first index block according to the target time period and the collection time range in each first index block, and the collection time range recorded in the target index block overlaps with the target time period. The data acquisition request also carries the target time period, and each first index block records the collection time range of the data stored in a data block.

[0194] At least one target data block is obtained from the file according to a storage position in the at least one target index block.

[0195] Optionally, the first determining module 1003 is configured to:

[0196] Based on the first data type, device identification indexes of multiple candidate Internet of Things devices corresponding to the first data type are determined from the measurement block, and the measurement block records the corresponding relationship between the data type and the device identification index of the Internet of Things device.

[0197] Based on the first device identifier, a device identifier index of a first Internet of Things device corresponding to the first device identifier is determined from device identifier indexes of a plurality of candidate Internet of Things devices.

[0198] The storage position of the first keyword in the sequence block is determined based on the device identification index of the first Internet of Things device, and a plurality of keywords are recorded in the sequence block.

[0199] The first keyword is obtained from the sequence block based on the storage position of the first keyword in the sequence block.

[0200] Optional, reference Fig.11 , the device may also include:

[0201] The receiving module 1005 is used to receive data information sent by the second Internet of Things device, where the data information includes data collected by the second Internet of Things device, a second device identifier of the second Internet of Things device, and a second data type of the data collected by the second Internet of Things device.

[0202] The storage module 1006 is used to store the data collected by the second IoT device in the first data block if the remaining storage capacity of the first data block among the multiple data blocks is greater than or equal to the storage capacity threshold.

[0203] The generating module 1007 is used to generate a second keyword and record the corresponding relationship between the second keyword and the second index block, where the second index block records the storage position of the first data block in the file.

[0204] The storage module 1006 is further configured to store the second keyword in the sequence block, and the storage position of the second keyword in the sequence block is determined by the device identification index corresponding to the second device identification.

[0205] refer to Fig.11 , the device may also include:

[0206] The deletion module 1008 is used to delete the corresponding relationship at the end of the target linked list if the length of the target linked list stored in the memory of the data management device is less than or equal to the length threshold after acquiring the data in the at least one target data block.

[0207] The storage module 1006 is further used to store the correspondence between the first data type, the first device identifier and the first keyword at the head of the target linked list.

[0208] Optionally, the first acquisition module 1001 is used to:

[0209] Receiving a data acquisition request for a first Internet of Things device sent by a data analysis device;

[0210] refer to Fig.11 , the device may also include:

[0211] The sending module 1010 is used to send the acquired data to the data analysis device.

[0212] In summary, the disclosed embodiment provides a data acquisition device, which can determine a first keyword based on a first data type and a first device identifier of a first IoT device, and can determine at least one index block using the first keyword as an index, and then acquire a data block according to a storage position in the index block. Since there is no need to compare the device identifier of the first IoT device with the device identifiers of multiple IoT devices stored one by one, the efficiency of data acquisition is effectively improved.

[0213] The present disclosure provides a computer-readable storage medium in which a program code is stored. The program code is loaded and executed by a processor to implement the data acquisition method shown in the above embodiment, for example Figure 2 or Figure 3 The data acquisition method shown.

[0214] The present disclosure provides a computer program product including instructions. When the computer program product is run on a computer, the computer executes the data acquisition method shown in the above embodiment. For example, Figure 2 or Figure 3 The data acquisition method shown.

[0215] The present disclosure embodiment provides a data management device 20, referring to Fig.12 The data management device 20 may include a processor 21 and a memory 22. The memory 22 stores program codes, which are loaded and executed by the processor 21 to implement the data acquisition method shown in the above embodiment. For example, Figure 2 or Figure 3 The data acquisition method shown.

[0216] The present disclosure provides an Internet of Things, such as Figure 1 As shown, the Internet of Things may include a data management device 20, at least one Internet of Things device 10 and a data analysis device 30.

[0217] The data management device 20 includes Fig.10 or Fig.11 The data acquisition device 100 shown, or Fig.12 Device shown.

[0218] Each IoT device 10 is used to send data information to the data management device, where the data information includes data collected by the IoT device, a device identifier of the IoT device, and a data type of the data collected by the IoT device.

[0219] The data analysis device 30 is used to send a data acquisition request for the first IoT device to the data management device 20 , and receive data sent by the data management device 20 .

[0220] In the embodiments of the present disclosure, the terms "first", "second", "third", "fourth" and "fifth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. In the embodiments of the present disclosure, the term "plurality" means two or more. The term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previously associated objects are in an "or" relationship.

[0221] The above description is only an optional embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data acquisition method, characterized in that: A data management device applied to the Internet of Things, wherein a plurality of data blocks are stored in the data management device, each of the data blocks contains data of a data type collected by an Internet of Things device; the method comprises: Obtain a data acquisition request for a first Internet of Things device, where the data acquisition request carries a first device identifier of the first Internet of Things device and a first data type of data to be acquired; Based on the first data type and the first device identifier, determining a first keyword corresponding to the first data type and the first device identifier; Determine at least one first index block corresponding to the first keyword, each of the first index blocks recording a storage position of a data block in a file; According to the storage position in the at least one first index block, obtaining at least one target data block from the file, each of the target data blocks storing data of the first data type collected by the first Internet of Things device; The determining, based on the first data type and the first device identifier, a first keyword corresponding to the first data type and the first device identifier includes: Determine, based on the first data type, from a measurement block device identification indexes of a plurality of candidate Internet of Things devices corresponding to the first data type, wherein the measurement block records a correspondence between the data type and the device identification indexes of the Internet of Things devices; Determine, based on the first device identifier, a device identifier index of the first Internet of Things device corresponding to the first device identifier from among the device identifier indexes of the multiple candidate Internet of Things devices; Determine a first keyword index based on a device identification index of the first IoT device, where the first keyword index is used to indicate a storage position of the first keyword in a sequence block, where a plurality of keywords are recorded in the sequence block, and both the device identification index and the first keyword index are numbers; Based on a storage position of the first keyword in the sequence block, acquiring the first keyword from the sequence block; The first keyword index is determined based on the correspondence between the device identification index in the sequence identification set and the keyword index; or, the first keyword index is determined based on the device identification index of the first Internet of Things device using a hash algorithm.

2. The method according to claim 1, characterized in that The data acquisition request also carries a target time period, and each of the first index blocks also records a collection time range of data stored in a data block; acquiring at least one target data block from the file according to a storage position in the at least one first index block includes: Determine at least one target index block from the at least one first index block according to the target time period and the acquisition time range in each of the first index blocks, wherein the acquisition time range recorded in the target index block overlaps with the target time period; At least one target data block is obtained from the file according to a storage position in the at least one target index block.

3. The method according to claim 1, characterized in that The method further comprises: Receive data information sent by a second Internet of Things device, where the data information includes data collected by the second Internet of Things device, a second device identifier of the second Internet of Things device, and a second data type of the data collected by the second Internet of Things device; If the remaining storage capacity of a first data block among the multiple data blocks is greater than or equal to the storage capacity threshold, storing the data collected by the second IoT device in the first data block; Generate a second keyword, and record a correspondence between the second keyword and a second index block, where the second index block records a storage position of the first data block in a file; The second keyword is stored in the sequence block, and the storage position of the second keyword in the sequence block is determined by the device identification index corresponding to the second device identification.

4. The method according to any one of claims 1 to 3, characterized in that: After acquiring the data in the at least one target data block, the method further includes: If the length of the target linked list stored in the memory of the data management device is less than or equal to the length threshold, deleting the corresponding relationship at the tail of the target linked list; The correspondence between the first data type, the first device identifier and the first keyword is stored at the head of the target linked list.

5. The method according to any one of claims 1 to 3, characterized in that: The obtaining of a data acquisition request for the first IoT device includes: Receiving a data acquisition request for the first Internet of Things device sent by a data analysis device; After acquiring the data in the at least one target data block, the method further includes: The acquired data is sent to the data analysis device.

6. A data acquisition device, characterized in that: Applied to a data management device in the Internet of Things, the data management device stores a plurality of data blocks, each of which contains data of a data type collected by an Internet of Things device; the device comprises: A first acquisition module, configured to acquire a data acquisition request for a first IoT device, wherein the data acquisition request carries a first device identifier of the first IoT device and a first data type of data to be acquired; A first determining module, configured to determine, based on the first data type and the first device identifier, a first keyword corresponding to the first data type and the first device identifier; A second determining module, configured to determine at least one first index block corresponding to the first keyword, each of the first index blocks recording a storage position of a data block in a file; A second acquisition module, configured to acquire at least one target data block from the file according to a storage position in the at least one first index block, each of the target data blocks storing data of the first data type collected by the first Internet of Things device; Wherein, the first determining module is used to: Determine, based on the first data type, from a measurement block device identification indexes of a plurality of candidate Internet of Things devices corresponding to the first data type, wherein the measurement block records a correspondence between the data type and the device identification indexes of the Internet of Things devices; Determine, based on the first device identifier, a device identifier index of the first Internet of Things device corresponding to the first device identifier from among the device identifier indexes of the multiple candidate Internet of Things devices; Determine a first keyword index based on the device identification index of the first Internet of Things device, where the first keyword index is used to indicate a storage position of the first keyword in a sequence block, where a plurality of keywords are recorded; Based on a storage position of the first keyword in the sequence block, acquiring the first keyword from the sequence block; The first keyword index is determined based on the correspondence between the device identification index in the sequence identification set and the keyword index; or, the first keyword index is determined based on the device identification index of the first Internet of Things device using a hash algorithm.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes are loaded and executed by a processor to implement the data acquisition method according to any one of claims 1 to 5.

8. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores a program code, and the program code is loaded and executed by the processor to implement the data acquisition method according to any one of claims 1 to 5.

9. An Internet of Things, characterized in that: The Internet of Things includes: a data management device, at least one Internet of Things device and a data analysis device; The data management device comprises the data acquisition device as claimed in claim 6; Each of the IoT devices is used to send data information to the data management device, where the data information includes data collected by the IoT device, a device identifier of the IoT device, and a data type of the data collected by the IoT device; The data analysis device is used to send a data acquisition request for the first Internet of Things device to the data management device, and receive data sent by the data management device.

Images