Method and device for storing monitoring data of reservoir

Abstract

The present application provides a method and device for storing monitoring data of a reservoir. The method comprises: according to the equipment identification of each monitoring device that monitors the same target area of ​​the reservoir, obtaining each candidate data corresponding to one of the equipment identifications from each monitoring data stored in the cache of the target terminal; deduplicating each candidate data according to the net load data and timestamp of each candidate data to obtain each target monitoring data; storing each target monitoring data in the disk storage of the target terminal; wherein the net load data or timestamps between each target monitoring data are different. A method for storing monitoring data of a reservoir provided in an embodiment of the present application can reduce the storage pressure of the disk storage of the terminal.

Description

Technical Field

[0001] The present application relates to the field of data storage technology, and in particular to a method and device for storing monitoring data of a reservoir. Background Art

[0002] In order to improve the safety of the reservoir, various monitoring devices for monitoring water environment elements such as water quality, water level or water environment are usually deployed in a certain area of ​​the reservoir, so as to monitor various areas of the reservoir through various monitoring devices, and store the monitoring data in the disk storage of the target terminal through the underlying storage engine. Since the amount of data collected by the monitoring equipment is large, if the monitoring data is directly stored in the disk storage, the access frequency of the underlying storage engine will be too high, affecting the response speed and throughput of the target terminal. Therefore, in the related technology, the monitoring data can be stored in the cache of the terminal by setting a cache, and then stored in the terminal's disk storage after batch processing through an asynchronous task queue, so as to reduce the access frequency to the underlying storage engine.

[0003] However, although this method can reduce the frequency of access to the terminal's underlying storage engine, a large amount of monitoring data will still be stored in the disk storage, resulting in a large storage pressure on the terminal's disk storage. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a method for storing monitoring data of a reservoir, which can reduce the storage pressure of a disk storage of a terminal.

[0005] The application also proposes a monitoring data storage device for a reservoir.

[0006] The application also provides an electronic device.

[0007] The present application also proposes a computer-readable storage medium.

[0008] A method for storing monitoring data of a reservoir according to an embodiment of the first aspect of the present application includes:

[0009] According to the device identifiers of the monitoring devices monitoring the same target area of ​​the reservoir, the candidate data corresponding to one of the device identifiers are obtained from the monitoring data stored in the cache of the target terminal;

[0010] Deduplication of each candidate data is performed according to the payload data and timestamp of each candidate data to obtain each target monitoring data;

[0011] Storing each of the target monitoring data in a disk storage of the target terminal;

[0012] Among them, the payload data or timestamps between the target monitoring data are different.

[0013] By obtaining the candidate data corresponding to one of the device identifiers from the monitoring data stored in the cache of the target terminal according to the device identifiers of the monitoring devices that monitor the same target area of ​​the reservoir, and deduplicating the candidate data according to the net load data and timestamp of each candidate data, the target monitoring data is obtained, so as to store the target monitoring data in the disk storage of the target terminal. Since if the net load data and timestamp between the candidate data are the same, only retaining a single candidate data can also determine the monitoring result of the target area at the time point corresponding to the timestamp, and will not cause the monitoring result of the target area at the time point corresponding to the timestamp to be missing, thereby ensuring the integrity of the monitoring results stored in the disk storage while reducing the amount of data that needs to be stored in the disk storage, so as to reduce the storage pressure of the disk storage while reducing the access frequency to the underlying storage engine.

[0014] According to one embodiment of the present application, each of the monitoring devices is a monitoring device of the same type.

[0015] According to an embodiment of the present application, the candidate data are deduplicated according to the payload data and timestamp of each candidate data to obtain each target monitoring data, including:

[0016] Deduplication of each candidate data according to the payload data and timestamp of each candidate data to obtain deduplication data;

[0017] Determine that the timestamps of the deduplicated data are all different, and sort the deduplicated data according to the timestamps of the deduplicated data to obtain a first data sequence;

[0018] According to a segmentation point located between adjacent deduplicated data having different payload data in the first data sequence, the first data sequence is segmented to obtain at least one first subsequence;

[0019] The deduplicated data corresponding to the endpoints of each of the first subsequences are determined as the target monitoring data.

[0020] According to one embodiment of the present application, it also includes:

[0021] Determine that there is target deduplication data with the same timestamp among the deduplication data, and sort the remaining deduplication data according to the timestamps of the remaining deduplication data except the target deduplication data in the deduplication data to obtain a second data sequence;

[0022] The second data sequence is divided according to a first division point located between adjacent remaining deduplicated data that are not different payload data in the second data sequence, and a second division point determined according to a timestamp of each target deduplicated data, to obtain at least one second subsequence;

[0023] The remaining deduplicated data corresponding to each endpoint of the second subsequence and each target deduplicated data are determined as the target monitoring data.

[0024] According to one embodiment of the present application, storing each of the target monitoring data into a disk storage of the target terminal includes:

[0025] Determining the storage priority of the target monitoring data according to a comparison result between the payload data of any target monitoring data and the preset data;

[0026] According to the storage priority of each target monitoring data, each target monitoring data is stored in the disk storage in sequence;

[0027] The preset data is determined according to the device type of the monitoring device that sends the target monitoring data.

[0028] According to one embodiment of the present application, after storing the target monitoring data in the disk storage of the target terminal, the method further includes:

[0029] The target monitoring data is deleted from the cache.

[0030] According to one embodiment of the present application, it also includes:

[0031] It is determined that the access frequency of the target monitoring data in the preset time period is less than the preset frequency, and the target monitoring data is deleted from the disk storage.

[0032] A monitoring data storage device for a reservoir according to an embodiment of the second aspect of the present application includes:

[0033] A data acquisition module, for acquiring, according to the device identification of each monitoring device monitoring the same target area of ​​the reservoir, each candidate data corresponding to one of the device identifications from each monitoring data stored in the cache of the target terminal;

[0034] A data deduplication module, used to deduplicate each candidate data according to the payload data and timestamp of each candidate data to obtain each target monitoring data;

[0035] A data storage module, used for storing each of the target monitoring data into a disk storage of a target terminal;

[0036] Among them, the payload data or timestamps between the target monitoring data are different.

[0037] According to an electronic device of an embodiment of the third aspect of the present application, the electronic device includes a processor and a memory storing a computer program, and when the processor executes the computer program, the method for storing monitoring data of a reservoir described in any of the above embodiments is implemented.

[0038] According to the computer-readable storage medium of the fourth aspect of the present application, a computer program is stored thereon, and when the computer program is executed by a processor, the method for storing monitoring data of a reservoir described in any of the above embodiments is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 A first flow chart of a method for storing monitoring data of a reservoir according to some embodiments of the present application;

[0041] Figure 2 A second flow chart of a method for storing monitoring data of a reservoir according to some embodiments of the present application;

[0042] Figure 3 A third flow chart of a method for storing monitoring data of a reservoir in some embodiments of the present application;

[0043] Figure 4 It is a schematic diagram of the structure of a monitoring data storage device for a reservoir provided in an embodiment of the present application;

[0044] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] Below, the reservoir monitoring data storage method and device provided in the embodiments of the present application will be described and explained in detail through several specific embodiments.

[0047] In one embodiment, a method for storing monitoring data of a reservoir is provided, which is applied to a target terminal for storing monitoring data of the reservoir. The target terminal may be any target terminal such as a mobile terminal, a desktop terminal or a server. The server may be an independent server or a server cluster composed of multiple servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence sampling point devices.

[0048] like Figure 1 As shown, a reservoir monitoring data storage method provided in this embodiment includes:

[0049] Step 101, according to the device identifiers of the monitoring devices monitoring the same target area of ​​the reservoir, obtaining the candidate data corresponding to one of the device identifiers from the monitoring data stored in the cache of the target terminal;

[0050] Step 102, deduplicating each candidate data according to the payload data and timestamp of each candidate data to obtain each target monitoring data;

[0051] Step 103, storing each of the target monitoring data in the disk storage of the target terminal;

[0052] Among them, the payload data or timestamps between the target monitoring data are different.

[0053] In some embodiments, the target area may be any area among the areas obtained by dividing the reservoir. For any target area, various monitoring devices for monitoring water environment elements are uniformly arranged. For example, multiple water quality monitoring devices for monitoring the water quality of the target area, multiple water level monitoring devices for monitoring the water level of the target area, multiple rainfall monitoring devices for monitoring the rainfall in the target area, multiple humidity monitoring devices for monitoring the humidity in the target area, etc. may be arranged.

[0054] Each monitoring device in each target area of ​​the reservoir is used to monitor each target area regularly, generate corresponding monitoring data and upload it to the target terminal. The target terminal stores all received monitoring data in the cache. Among them, the monitoring data is uploaded to the target terminal in the form of a data packet, and the monitoring data records the device identification of the monitoring device that generates the monitoring data, the payload data representing the actual data content of the data packet transmission, and the timestamp. Among them, the device identification can be the IP address of the monitoring device.

[0055] In some embodiments, the device identification of each monitoring device monitoring the same target area in the reservoir can be obtained first. For example, if the target area is a water storage area in the reservoir for storing water, the device identification of all monitoring devices monitoring the water storage area can be obtained. Then, the device identification of any monitoring device monitoring the target area is matched with the device identification of all monitoring data stored in the cache, so as to obtain monitoring data with the same device identification as the device identification of the monitoring device from each monitoring data as candidate data.

[0056] After all the alternative data are obtained from the monitoring data stored in the cache according to the device identification of each monitoring device monitoring the same target area, the alternative data with the same net load data and timestamp can be deduplicated according to the net load data and timestamp of each alternative data, so that there is no alternative data with the same net load data and timestamp among the deduplicated alternative data, so that the deduplicated alternative data can be determined as the target monitoring data.

[0057] Exemplarily, assume that among the alternative data there is alternative data A uploaded by water quality monitoring device 1, and alternative data B uploaded by water quality monitoring device 2, the net load data recorded in alternative data A is PH6.5, and the timestamp is 8:00 a.m., and the net load data recorded in alternative data B is PH6.5, and the timestamp is 8:00 a.m. Since alternative data A and alternative data B are data obtained by monitoring the same target area, if the net load data and timestamp of alternative data A and alternative data B are the same, then only retaining a single alternative data can also determine the pH value of the target area at the time point corresponding to the timestamp, and will not cause the monitoring result of the target area at the time point corresponding to the timestamp to be missing, and the integrity of the monitoring result can still be guaranteed. Therefore, at this time, alternative data A and alternative data B can be deduplicated, and only alternative data A or alternative data B can be retained.

[0058] Since different types of monitoring equipment may be deployed in the same target area, and the net load data of the monitoring data uploaded by different monitoring equipment is different, using the net load data and timestamp to deduplicate data will not affect the monitoring data uploaded by different types of monitoring equipment.

[0059] After obtaining each target monitoring data, each target monitoring data can be stored in the disk storage of the target terminal through the underlying storage engine, such as recording each target monitoring data in an asynchronous task queue, and processing the asynchronous task queue through the underlying storage engine to store each target monitoring data in the asynchronous task queue in the disk storage of the target terminal. The disk storage can be a disk or a hard disk.

[0060] By obtaining the candidate data corresponding to one of the device identifiers from the monitoring data stored in the cache of the target terminal according to the device identifiers of the monitoring devices that monitor the same target area of ​​the reservoir, and deduplicating the candidate data according to the net load data and timestamp of each candidate data, the target monitoring data is obtained, so as to store the target monitoring data in the disk storage of the target terminal. Since if the net load data and timestamp between the candidate data are the same, only retaining a single candidate data can also determine the monitoring result of the target area at the time point corresponding to the timestamp, and will not cause the monitoring result of the target area at the time point corresponding to the timestamp to be missing, thereby ensuring the integrity of the monitoring results stored in the disk storage while reducing the amount of data that needs to be stored in the disk storage, so as to reduce the storage pressure of the disk storage while reducing the access frequency to the underlying storage engine.

[0061] Taking into account the different types of monitoring equipment, the net load data in the monitoring data uploaded by them must be different. For example, the net load data uploaded by water quality monitoring equipment and water level monitoring equipment are water quality and water level respectively. Therefore, in order to reduce the data that needs to be traversed during deduplication and improve the deduplication efficiency, in some embodiments, each monitoring equipment can be a monitoring equipment of the same type, that is, based on the equipment identifier of each monitoring equipment of the same type that monitors the same target area of ​​the reservoir, each alternative data can be obtained from each monitoring data, so that the data type of the net load data in each alternative data obtained is the same, so that the alternative data with the same data type of the net load data can be deduplicated.

[0062] To further reduce the storage pressure of the disk storage, when each monitoring device is of the same type, in some embodiments, such as Figure 2 As shown, according to the payload data and timestamp of each candidate data, each candidate data is deduplicated to obtain each target monitoring data, including:

[0063] Step 201, deduplicating each candidate data according to the payload data and timestamp of each candidate data to obtain deduplicated data;

[0064] Step 202, determining that the timestamps of the deduplicated data are all different, and sorting the deduplicated data according to the timestamps of the deduplicated data to obtain a first data sequence;

[0065] Step 203, dividing the first data sequence according to a division point located between adjacent deduplicated data having different payload data in the first data sequence to obtain at least one first subsequence;

[0066] Step 204: The deduplicated data corresponding to the endpoints of each of the first subsequences are determined as the target monitoring data.

[0067] In some embodiments, since each monitoring device is a monitoring device of the same type, the data type of the payload data of the acquired candidate data is the same.

[0068] After obtaining each candidate data, the candidate data with the same net load data and timestamp can be deduplicated first according to the net load data and timestamp of each candidate data, so that there is no candidate data with the same net load data and timestamp among the deduplicated candidate data, so that the deduplicated candidate data can be determined as deduplicated data.

[0069] After obtaining the deduplicated data, the timestamps of the deduplicated data can be compared. If the timestamps of the deduplicated data are different, it means that the monitoring results of the target area at the same time point are the same. At this time, the deduplicated data can be sorted from small to large or from large to small according to the timestamps of the deduplicated data to form a first data sequence.

[0070] Exemplarily, assuming that the deduplicated data include deduplicated data A, deduplicated data B, deduplicated data C, deduplicated data D and deduplicated data E, the timestamp of deduplicated data A is t1, the timestamp of deduplicated data B is t2, the timestamp of deduplicated data C is t3, the timestamp of deduplicated data D is t4, and the timestamp of deduplicated data E is t5, t1 is earlier than t2, earlier than t3, earlier than t4 and earlier than t5, then the first data sequence can be obtained: {A, B, C, D, E}.

[0071] After obtaining the first data sequence, it is possible to sequentially detect whether two adjacent deduplication data in the first data sequence have the same payload data. If two adjacent deduplication data have different payload data, the position between the two adjacent deduplication data can be used as a segmentation point. For example, assuming that the first data sequence is {A, B, C, D, E}, the payload data of deduplication data A is PH6.5, the payload data of deduplication data B is PH6.5, the payload data of deduplication data C is PH6.5, the payload data of deduplication data D is PH6.8, and the payload data of deduplication data E is PH7.0, then the position between deduplication data C and deduplication data D can be used as a segmentation point of the first data sequence, and the position between deduplication data D and deduplication data E can be used as another segmentation point of the first data sequence.

[0072] After determining each segmentation point of the first data sequence, the first data sequence can be segmented according to each segmentation point to obtain each first subsequence. For example, if the above-mentioned first data sequence {A, B, C, D, E} is segmented, each first subsequence can be obtained as {A, B, C}, {D}, {E}. After obtaining each first subsequence, the deduplication data corresponding to the endpoints of each first subsequence, that is, the first deduplication data and the last deduplication data in the first subsequence, can be determined as the target detection data. If each first subsequence is {A, B, C}, {D}, {E}, the target detection data can be determined as A, C, D, E.

[0073] Since there is a timestamp in the target detection data, only the endpoint data in the first subsequence is retained as the target detection data, and the detection result of the target area within a certain period of time can also be determined. For example, in the first subsequence {A, B, C}, the timestamp of the deduplicated data A is 8:00 a.m., the timestamp of the deduplicated data B is 8:05 a.m., and the timestamp of the deduplicated data C is 8:10 a.m. Then, only the deduplicated data A and the deduplicated data C are retained, and it can also be determined that the PH value of the target area in the period of 8:00 a.m-8:10 a.m is the same, that is, the PH value of the target area at 8:05 a.m is determined to be 6.8, which will not cause the loss of monitoring results. Therefore, while ensuring the integrity of the monitoring results stored in the disk storage, the amount of data that needs to be stored in the disk storage can be further reduced, so as to further reduce the storage pressure of the disk storage while further reducing the access frequency to the underlying storage engine.

[0074] Considering that different monitoring devices of the same type monitor the same target area, the payload data of the monitoring data uploaded at the same time may be different. For example, the water quality monitoring device 1 and the water quality monitoring device 2 that monitor the water quality of the same target area may monitor the water quality of 6.8 and 7.0 respectively at time t1. In order to avoid the loss of the monitoring results, in some embodiments, such as Figure 3 As shown, the method also includes:

[0075] Step 301, determining that there is target deduplication data with the same timestamp among the deduplication data, and sorting the remaining deduplication data according to the timestamps of the remaining deduplication data except the target deduplication data in the deduplication data to obtain a second data sequence;

[0076] Step 302, dividing the second data sequence according to a first dividing point located between adjacent remaining deduplicated data that are not different payload data in the second data sequence, and a second dividing point determined according to a timestamp of each target deduplicated data, to obtain at least one second subsequence;

[0077] Step 303: Determine the remaining deduplicated data corresponding to each endpoint of the second subsequence and each target deduplicated data as the target monitoring data.

[0078] In some embodiments, if there are deduplicated data with the same timestamp among the deduplicated data, it means that there are different monitoring results for the target area at a certain time point. At this time, the deduplicated data with the same timestamp can be determined as the target deduplicated data, and then the remaining deduplicated data are sorted from small to large or from large to small according to the timestamps of all the remaining deduplicated data except the target deduplicated data in each deduplicated data to form a second data sequence.

[0079] Exemplarily, assuming that each deduplicated data includes deduplicated data A, deduplicated data B, deduplicated data C, deduplicated data D, deduplicated data E, deduplicated data F and deduplicated data G, the timestamp of deduplicated data A is t1, the timestamp of deduplicated data B is t2, the timestamp of deduplicated data C is t2, the timestamp of deduplicated data D is t3, the timestamp of deduplicated data E is t4, the timestamp of deduplicated data F is t5, and the timestamp of deduplicated data G is t6, and t1 is earlier than t2, earlier than t3, earlier than t4, earlier than t5, and earlier than t6, then the deduplicated data B and deduplicated data C can be determined as the target deduplicated data, and the deduplicated data A, deduplicated data D, deduplicated data E, deduplicated data F and deduplicated data G can be determined as the remaining deduplicated data, and then the remaining deduplicated data are sorted to obtain a second data sequence: {A, D, E, F, G}.

[0080] After obtaining the second data sequence, it is possible to sequentially detect whether two adjacent deduplication data in the second data sequence have the same payload data. If two adjacent deduplication data have different payload data, the position between the two adjacent deduplication data can be used as the first segmentation point. For example, assuming that the second data sequence is {A, D, E, F, G}, the payload data of deduplication data A is PH6.5, the payload data of deduplication data D is PH6.5, the payload data of deduplication data E is PH6.8, the payload data of deduplication data F is PH6.8, and the payload data of deduplication data G is PH6.8, then the position between deduplication data D and deduplication data E can be used as the first segmentation point of the second data sequence.

[0081] At the same time, the timestamp of each target deduplication data can be determined as the second segmentation point. If the target deduplication data is duplicate data B and deduplication data C, and its timestamp is t2, then the timestamp t2 can be used as the second segmentation point of the second data sequence {A, D, E, F, G}. If the timestamp t1 of the deduplication data A is earlier than t2, and the timestamps of the deduplication data D, E, F, G are later than t2, then the position between the deduplication data A and the deduplication data D can be used as the second segmentation point.

[0082] After determining each first segmentation point and the second segmentation point, the second data sequence can be segmented according to the first segmentation point and the second segmentation point to obtain at least one second subsequence. For example, if the above second data sequence {A, D, E, F, G} is segmented, each second subsequence can be obtained as {A}, {D}, {E, F, G}. After obtaining each second subsequence, the remaining deduplication data corresponding to the endpoints of each second subsequence, that is, the first remaining deduplication data and the last remaining deduplication data in the second subsequence, and each target deduplication data can be determined as target detection data. If each second subsequence is {A}, {D}, {E, F, G}, and each target deduplication data is B, C, then the target detection data can be determined as A, B, C, D, E, G.

[0083] In this way, the amount of data that needs to be stored can be reduced while avoiding missing all monitoring results at a certain point in time and causing the loss of monitoring results. This can ensure the integrity of the monitoring results stored in the disk storage while further reducing the amount of data that needs to be stored in the disk storage, thereby further reducing the frequency of access to the underlying storage engine and further reducing the storage pressure on the disk storage.

[0084] After determining each target monitoring data, each target monitoring data can be stored in the disk storage of the target terminal through the underlying storage engine. In some embodiments, storing each target monitoring data in the disk storage of the target terminal includes:

[0085] Determining the storage priority of the target monitoring data according to a comparison result between the payload data of any target monitoring data and the preset data;

[0086] According to the storage priority of each target monitoring data, each target monitoring data is stored in the disk storage in sequence;

[0087] The preset data is determined according to the device type of the monitoring device that sends the target monitoring data.

[0088] In some embodiments, after obtaining each target monitoring data, for any target monitoring data, the device identification can be first obtained from the target monitoring data to determine the device type of the monitoring device that sends the target monitoring data based on the device identification. After determining the device type of the monitoring device, the preset data corresponding to the device type can be found from the record table that records the mapping relationship between the device type and the preset data. Among them, the preset data is used to indicate the critical value at which the data monitored by the monitoring device of a certain device type is abnormal. If the monitoring device is a water quality monitoring device, the corresponding preset data is the critical value indicating that the water quality is abnormal. If the water quality is less than the critical value, it means that the water quality is normal. If the water quality is greater than or equal to the critical value, it means that the water quality is abnormal.

[0089] After obtaining the corresponding preset data, the preset data can be compared with the net load data of the target monitoring data. If the net load data of the target monitoring data is greater than or equal to the preset data, it means that the target monitoring data is abnormal monitoring data and needs to be stored first. At this time, the storage priority of the target monitoring data can be determined as high priority; if the net load data of the target monitoring data is less than the preset data, it means that the target monitoring data is normal monitoring data and does not need to be stored first. At this time, the storage priority of the target monitoring data can be determined as low priority.

[0090] After determining the storage priority of each target monitoring data, each target monitoring data can be stored in the disk storage in descending order of storage priority to reduce the possibility of failure to store abnormal monitoring data in time due to system power outage.

[0091] In order to enable the cache to store data in time, in some embodiments, after storing the target monitoring data in the disk storage of the target terminal, the method further includes:

[0092] The target monitoring data is deleted from the cache.

[0093] In some embodiments, after detecting that the target detection data is stored in the disk storage of the target terminal, the target monitoring data can be deleted from the cache. After detecting that each target monitoring data is stored in the disk storage of the target terminal, all candidate data can be deleted to ensure that the cache can obtain the latest data in time when it is read next time.

[0094] In order to further reduce the storage pressure of the disk storage, in some embodiments, it also includes: determining that the access frequency of the target monitoring data in a preset time period is less than a preset frequency, and deleting the target monitoring data from the disk storage.

[0095] The preset time period and the preset frequency can be set according to actual conditions. After the target monitoring data is stored in the disk storage, the access frequency of the target monitoring data within the preset time period can be detected. Alternatively, it can be first monitored whether the remaining storage capacity of the disk storage is less than the preset storage capacity. If so, the access frequency of the target monitoring data within the preset time period can be detected.

[0096] If the access frequency of the target monitoring data within the preset time period is less than the preset frequency, it means that the target monitoring data is less likely to receive attention. At this time, the target monitoring data can be deleted from the disk storage.

[0097] The following is a description of the reservoir monitoring data storage device provided in the present application. The reservoir monitoring data storage device described below and the reservoir monitoring data storage method described above can be referenced to each other.

[0098] In one embodiment, if Figure 4 As shown, a monitoring data storage device for a reservoir is provided, comprising:

[0099] The data acquisition module 210 is used to acquire, according to the device identifiers of the monitoring devices monitoring the same target area of ​​the reservoir, the candidate data corresponding to one of the device identifiers from the monitoring data stored in the cache of the target terminal;

[0100] A data deduplication module 220, configured to deduplicate each candidate data according to the payload data and timestamp of each candidate data to obtain each target monitoring data;

[0101] The data storage module 230 is used to store each of the target monitoring data into the disk storage of the target terminal;

[0102] Among them, the payload data or timestamps between the target monitoring data are different.

[0103] By obtaining the candidate data corresponding to one of the device identifiers from the monitoring data stored in the cache of the target terminal according to the device identifiers of the monitoring devices that monitor the same target area of ​​the reservoir, and deduplicating the candidate data according to the net load data and timestamp of each candidate data, the target monitoring data is obtained, so as to store the target monitoring data in the disk storage of the target terminal. Since if the net load data and timestamp between the candidate data are the same, only retaining a single candidate data can also determine the monitoring result of the target area at the time point corresponding to the timestamp, and will not cause the monitoring result of the target area at the time point corresponding to the timestamp to be missing, thereby ensuring the integrity of the monitoring results stored in the disk storage while reducing the amount of data that needs to be stored in the disk storage, so as to reduce the storage pressure of the disk storage while reducing the access frequency to the underlying storage engine.

[0104] In one embodiment, each of the monitoring devices is a monitoring device of the same type.

[0105] In one embodiment, the data deduplication module 220 is specifically used for:

[0106] Deduplication of each candidate data according to the payload data and timestamp of each candidate data to obtain deduplication data;

[0107] Determine that the timestamps of the deduplicated data are all different, and sort the deduplicated data according to the timestamps of the deduplicated data to obtain a first data sequence;

[0108] According to a segmentation point located between adjacent deduplicated data having different payload data in the first data sequence, the first data sequence is segmented to obtain at least one first subsequence;

[0109] The deduplicated data corresponding to the endpoints of each of the first subsequences are determined as the target monitoring data.

[0110] In one embodiment, the data deduplication module 220 is further configured to:

[0111] Determine that there is target deduplication data with the same timestamp among the deduplication data, and sort the remaining deduplication data according to the timestamps of the remaining deduplication data except the target deduplication data in the deduplication data to obtain a second data sequence;

[0112] The second data sequence is divided according to a first division point located between adjacent remaining deduplicated data that are not different payload data in the second data sequence, and a second division point determined according to a timestamp of each target deduplicated data, to obtain at least one second subsequence;

[0113] The remaining deduplicated data corresponding to each endpoint of the second subsequence and each target deduplicated data are determined as the target monitoring data.

[0114] In one embodiment, the data storage module 230 is specifically used for:

[0115] Determining the storage priority of the target monitoring data according to a comparison result between the payload data of any target monitoring data and the preset data;

[0116] According to the storage priority of each target monitoring data, each target monitoring data is stored in the disk storage in sequence;

[0117] The preset data is determined according to the device type of the monitoring device that sends the target monitoring data.

[0118] In one embodiment, the data storage module 230 is further used for:

[0119] After storing the target monitoring data in the disk storage of the target terminal, the target monitoring data is deleted from the cache.

[0120] In one embodiment, the data storage module 230 is further used for:

[0121] It is determined that the access frequency of the target monitoring data in the preset time period is less than the preset frequency, and the target monitoring data is deleted from the disk storage.

[0122] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call a computer program in the memory 830 to execute a method for storing monitoring data of a reservoir, for example, including:

[0123] According to the device identifiers of the monitoring devices monitoring the same target area of ​​the reservoir, the candidate data corresponding to one of the device identifiers are obtained from the monitoring data stored in the cache of the target terminal;

[0124] Deduplication of each candidate data is performed according to the payload data and timestamp of each candidate data to obtain each target monitoring data;

[0125] Storing each of the target monitoring data in a disk storage of the target terminal;

[0126] Among them, the payload data or timestamps between the target monitoring data are different.

[0127] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0128] On the other hand, an embodiment of the present application further provides a storage medium, the storage medium includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the reservoir monitoring data storage method provided by the above embodiments, for example, including:

[0129] According to the device identifiers of the monitoring devices monitoring the same target area of ​​the reservoir, the candidate data corresponding to one of the device identifiers are obtained from the monitoring data stored in the cache of the target terminal;

[0130] Deduplication of each candidate data is performed according to the payload data and timestamp of each candidate data to obtain each target monitoring data;

[0131] Storing each of the target monitoring data in a disk storage of the target terminal;

[0132] Among them, the payload data or timestamps between the target monitoring data are different.

[0133] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for storing monitoring data of a reservoir, characterized in that: include: According to the device identifiers of the monitoring devices that monitor the same target area of ​​the reservoir, the candidate data corresponding to one of the device identifiers is obtained from the monitoring data stored in the cache of the target terminal; Deduplication of each candidate data is performed according to the payload data and timestamp of each candidate data to obtain each target monitoring data; Storing each of the target monitoring data in a disk storage of the target terminal; The payload data or timestamps of the target monitoring data are different, and the monitoring devices are monitoring devices of the same type; De-duplicate each candidate data according to the payload data and timestamp of each candidate data to obtain each target monitoring data, including: Deduplication of each candidate data according to the payload data and timestamp of each candidate data to obtain deduplication data; Determine that the timestamps of the deduplicated data are all different, and sort the deduplicated data from small to large or from large to small according to the timestamps of the deduplicated data to obtain a first data sequence; According to a segmentation point located between adjacent deduplicated data having different payload data in the first data sequence, the first data sequence is segmented to obtain at least one first subsequence; Determine the deduplicated data corresponding to the endpoints of each of the first subsequences as the target monitoring data; Determine that there is target deduplication data with the same timestamp among the deduplication data, and sort the remaining deduplication data according to the timestamps of the remaining deduplication data except the target deduplication data in the deduplication data to obtain a second data sequence; The second data sequence is divided according to a first division point located between adjacent remaining deduplicated data that are not different payload data in the second data sequence, and a second division point determined according to a timestamp of each target deduplicated data, to obtain at least one second subsequence; The remaining deduplicated data corresponding to each endpoint of the second subsequence and each target deduplicated data are determined as the target monitoring data.

2. The reservoir monitoring data storage method according to claim 1, characterized in that: Storing each of the target monitoring data to the disk storage of the target terminal includes: Determining the storage priority of the target monitoring data according to a comparison result between the payload data of any target monitoring data and the preset data; According to the storage priority of each target monitoring data, each target monitoring data is stored in the disk storage in sequence; The preset data is determined according to the device type of the monitoring device that sends the target monitoring data.

3. The reservoir monitoring data storage method according to claim 1, characterized in that: After storing the target monitoring data in the disk storage of the target terminal, the method further includes: The target monitoring data is deleted from the cache.

4. The reservoir monitoring data storage method according to claim 1, characterized in that: Also includes: It is determined that the access frequency of the target monitoring data in the preset time period is less than the preset frequency, and the target monitoring data is deleted from the disk storage.

5. A reservoir monitoring data storage device, characterized in that: include: A data acquisition module, for acquiring, according to the device identification of each monitoring device monitoring the same target area of ​​the reservoir, each candidate data corresponding to one of the device identifications from each monitoring data stored in the cache of the target terminal; A data deduplication module, used to deduplicate each candidate data according to the payload data and timestamp of each candidate data to obtain each target monitoring data; A data storage module, used for storing each of the target monitoring data into a disk storage of a target terminal; The payload data or timestamps of the target monitoring data are different, and the monitoring devices are monitoring devices of the same type; The data deduplication module is specifically used to include: Deduplication of each candidate data according to the payload data and timestamp of each candidate data to obtain deduplication data; Determine that the timestamps of the deduplicated data are all different, and sort the deduplicated data from small to large or from large to small according to the timestamps of the deduplicated data to obtain a first data sequence; According to a segmentation point located between adjacent deduplicated data having different payload data in the first data sequence, the first data sequence is segmented to obtain at least one first subsequence; Determine the deduplicated data corresponding to the endpoints of each of the first subsequences as the target monitoring data; Determine that there is target deduplication data with the same timestamp among the deduplication data, and sort the remaining deduplication data according to the timestamps of the remaining deduplication data except the target deduplication data in the deduplication data to obtain a second data sequence; The second data sequence is divided according to a first division point located between adjacent remaining deduplicated data that are not different payload data in the second data sequence, and a second division point determined according to a timestamp of each target deduplicated data, to obtain at least one second subsequence; The remaining deduplicated data corresponding to each endpoint of the second subsequence and each target deduplicated data are determined as the target monitoring data.

6. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the reservoir monitoring data storage method according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the reservoir monitoring data storage method according to any one of claims 1 to 4 is implemented.

Images