Data processing method and device, equipment, storage medium and computer program product

Abstract

The application discloses a data processing method, device and equipment, a storage medium and a computer program product. The method comprises the following steps: in the case that a first data processing request is received, a data maintenance table corresponding to a time sequence database is acquired, the data maintenance table maintaining update time of reference label data associated with each monitoring index in the time sequence database; target reference label data with a distance from the update time exceeding a first time period and a first monitoring index associated with the target reference label data are acquired from the data maintenance table, each monitoring index comprises the first monitoring index, and the reference label data comprises the target reference label data; and a time sequence in which the target reference label data and the first monitoring index are located in the time sequence database is deleted. The application can solve the problem that a large amount of expired data is stored in the time sequence database, thereby affecting performance.

Description

Technical Field

[0001] This invention belongs to the field of data processing, and particularly relates to a data processing method, apparatus, device, computer storage medium, and computer program product. Background Technology

[0002] Time-series databases primarily process and store time-stamped data, and are widely used in IoT device monitoring and operational data monitoring. For a specific monitoring object, the monitoring metrics parameters are typically defined in a time series format, which includes a combination of monitoring metrics and tags. However, factors such as the shutdown of monitoring equipment cause changes in the time series data within the database. Over time, this results in a large amount of expired data being stored in the time-series database, impacting its performance. Summary of the Invention

[0003] This invention provides a data processing method, apparatus, device, computer storage medium, and computer program product that can solve the problem of performance issues caused by storing a large amount of expired data in time-series databases.

[0004] Firstly, a data processing method is provided, including:

[0005] Upon receiving the first data processing request, the data maintenance table corresponding to the time series database is obtained. The data maintenance table maintains the update time of the reference tag data associated with each monitoring indicator in the time series database.

[0006] The target reference label data that is more than a first time period away from the update time is obtained from the data maintenance table, and the first monitoring indicator associated with the target reference label data is obtained. Each monitoring indicator includes the first monitoring indicator, and the reference label data includes the target reference label data.

[0007] Delete the target reference label data and the time series containing the first monitoring indicator from the time series database.

[0008] These examples ensure that the time-series database has sufficient space, thus solving the problem of large amounts of expired data stored in the time-series database, which affects database performance.

[0009] Optionally, before receiving the first data processing request, the method further includes:

[0010] If the system time reaches the preset timed update condition, the reference tag data associated with each monitoring indicator is obtained from the time series database. The reference tag data is the tag data recorded by the time series database in the second time period before the system time, and the second time period is shorter than the first time period.

[0011] Update the data maintenance table corresponding to the time series database based on each of the monitoring indicators and the associated reference tag data.

[0012] These examples demonstrate the update scheme for the data maintenance table corresponding to the time series database. This periodic update scheme is the basis for the timely deletion of expired time series in the time series database, thus ensuring that expired data can be identified in a timely and accurate manner.

[0013] Optionally, after updating the data maintenance table corresponding to the time-series database based on each of the monitoring indicators and the associated reference tag data, the method further includes:

[0014] Upon receiving a second data processing request, all target label data corresponding to the target monitoring indicator are obtained from the time series database, and reference label data corresponding to the target monitoring indicator are obtained from the data maintenance table. Each monitoring indicator includes the target monitoring indicator.

[0015] Filter out the difference label data that is not in the reference label data from all the target label data, and delete the difference label data from the time series database.

[0016] In the example above, by cleaning up historical time series data and combining it with the first data processing request, the space of the time series database was cleaned up to the maximum extent, ensuring the space utilization of the time series database and improving its performance.

[0017] Optionally, updating the data maintenance table corresponding to the time-series database based on each of the monitoring indicators and the associated reference tag data includes:

[0018] Determine whether any data field in the data maintenance table includes a second monitoring indicator and the reference tag data associated with the second monitoring indicator, wherein the second monitoring indicator is any of the monitoring indicators;

[0019] When none of the data fields in the data maintenance table include the second monitoring indicator and the reference tag data associated with the second monitoring indicator, a first data field is created. The first data field includes the second monitoring indicator, the reference tag data associated with the second monitoring indicator, and the creation time of the first data field.

[0020] When any data field in the data maintenance table includes a second monitoring indicator and the reference tag data associated with the second monitoring indicator, the update time of the reference tag data associated with the second monitoring indicator is set according to the current system time.

[0021] These examples demonstrate the update scheme for the data maintenance table corresponding to the time series database. This periodic update scheme is the basis for the timely deletion of expired time series in the time series database, thus ensuring that expired data can be identified in a timely and accurate manner.

[0022] Optionally, after deleting the target reference label data and the time series containing the first monitoring indicator from the time series database, the method further includes:

[0023] Set the running status of the second data field in the data maintenance table to the deleted status. The second data field includes the target reference label data and the first monitoring indicator.

[0024] In the example above, by updating the running status of the data fields in the data maintenance table in a timely manner, we can understand the running status of each time series maintained in the data maintenance table in a timely manner. When deleting time series, we can also remove these expired time series as early as possible, which helps to understand the timely status of each time series in the entire time series database.

[0025] Optionally, all data fields in the data maintenance table include a unique identifier calculated based on the monitoring metric and associated reference tag data; determining whether any data field in the data maintenance table includes the second monitoring metric and the associated reference tag data includes:

[0026] The first identifier is obtained by calculating the second monitoring indicator and the reference tag data associated with the second monitoring indicator;

[0027] Determine whether there exists a unique identifier in any data field of the data maintenance table that matches the first identifier; wherein when the unique identifiers in all data fields of the data maintenance table do not match the first identifier, then none of the data fields of the data maintenance table include the second monitoring indicator and the reference tag data associated with the second monitoring indicator.

[0028] In these optional examples, by setting and matching identifiers, it is possible to quickly retrieve whether the data maintenance table includes monitoring metrics and their associated tag data, improving the speed of information retrieval and making it convenient and fast.

[0029] Secondly, a data processing apparatus is provided, comprising:

[0030] The first acquisition module is used to acquire the data maintenance table corresponding to the time series database when a first data processing request is received. The data maintenance table maintains the update time of the reference tag data associated with each monitoring indicator in the time series database.

[0031] The second acquisition module is used to acquire target reference tag data that is more than a first time period away from the update time from the data maintenance table and a first monitoring indicator associated with the target reference tag data. Each monitoring indicator includes the first monitoring indicator, and the reference tag data includes the target reference tag data.

[0032] The deletion module is used to delete the target reference label data and the time series containing the first monitoring indicator from the time series database.

[0033] Thirdly, a data processing apparatus is provided, the data processing apparatus including a memory, a processor, and a data processing program stored in the memory and running on the processor, the data processing program implementing the data processing method of the first aspect.

[0034] Fourthly, a computer storage medium is provided that, when executed by a processor, implements the data processing method of the first aspect.

[0035] Fifthly, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the steps of the data processing method as described in the first aspect.

[0036] Compared with existing technologies, the data processing method, apparatus, device, storage medium, and computer program product provided in this application, upon receiving a first data processing request, obtains a data maintenance table corresponding to the time-series database. This data maintenance table maintains the update time of reference tag data associated with each monitoring indicator in the time-series database. Then, it retrieves target reference tag data whose update time exceeds a first time period and the first monitoring indicator associated with the target reference tag data from the data maintenance table. This deletes the time series containing the target reference tag data and the first monitoring indicator in the time-series database. Since the time series deleted from the time-series database is determined by the target reference tag data and the first monitoring indicator, which are the tag data and associated monitoring indicators maintained by the data maintenance table whose update time exceeds a first time period, what is deleted is expired data in the time-series database that has not been updated for a long time. This ensures the usable space of the time-series database and solves the problem of a large amount of expired data in the time-series database affecting database performance. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic flowchart of a data processing method according to an embodiment of the present invention.

[0039] Figure 2 This is a schematic flowchart of another embodiment of a data processing method.

[0040] Figure 3 This is a schematic block diagram of a data processing apparatus according to an embodiment of the present invention.

[0041] Figure 4 This is a schematic block diagram of a data processing device according to an embodiment of the present invention. Detailed Implementation

[0042] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of the invention by illustrating examples of the invention.

[0043] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0044] Time series data, also known as time-labeled data, is widely used in IoT device monitoring and operation and maintenance monitoring. Time series data can be data collected or generated by devices for real-time monitoring, inspection and analysis.

[0045] Traditional relational databases cannot meet the needs of efficient storage and processing of time series data. Therefore, there is an urgent need for a database system specifically optimized for time series data, which led to the creation of time series databases.

[0046] Time series databases, also known as time-series databases, are primarily used to process time-series data, which changes sequentially over time. As described in the background section, for monitoring indicators of a specific object, time series databases typically define them in the form of time series. Each time series represents a monitoring indicator parameter, and the time series includes a combination of monitoring indicators and tags.

[0047] The tag combination includes a tag key and a tag value, and the subsequent tag combination is also called tag data. The monitoring metrics, tags, and tag values ​​in the time series are serialized and stored in the monitoring metric mapping table of the time series database. For example, the monitoring metric mapping table of the time series database TSDB is the tsdb-uid table.

[0048] In relevant time-series database management, monitoring data and time series data are maintained separately. However, due to factors such as the shutdown of monitoring equipment, a large amount of expired data is stored in the monitoring indicator mapping table of the time-series database. Over time, the monitoring indicator mapping table that stores time series metadata becomes larger and larger, eventually reaching its storage limit and making it impossible to store new time series data. This seriously affects the performance of the time-series database.

[0049] Existing technologies also include time series metadata management functions in time series databases. When this function is enabled, time series information can be stored synchronously in another table in the time series database. However, this table mainly serves as a time series query function and cannot solve the above-mentioned problems.

[0050] To address the aforementioned issues, embodiments of this application provide a data processing method, apparatus, device, storage medium, and computer program product that periodically deletes expired data from a time-series database using a data maintenance table corresponding to the time-series database, thereby ensuring the usable space of the time-series database and resolving the problem of a large amount of expired data stored in the time-series database affecting database performance.

[0051] The data processing methods will be introduced below.

[0052] This application provides a data processing method, see below. Figure 1 In one alternative embodiment, the method includes:

[0053] S110, upon receiving the first data processing request, obtain the data maintenance table corresponding to the time series database, wherein the data maintenance table maintains the update time of the reference tag data associated with each monitoring indicator in the time series database.

[0054] S120, retrieve target reference tag data that is more than a first time period away from the update time from the data maintenance table and a first monitoring indicator associated with the target reference tag data. Each monitoring indicator includes the first monitoring indicator, and the reference tag data includes the target reference tag data.

[0055] S130, delete the target reference label data and the time series containing the first monitoring indicator from the time series database.

[0056] This embodiment of the application, upon receiving a first data processing request, retrieves the data maintenance table corresponding to the time-series database. The data maintenance table maintains the update time of reference tag data associated with each monitoring indicator in the time-series database. Then, it retrieves the target reference tag data and the first monitoring indicator associated with the target reference tag data that are more than a first time period since their update from the data maintenance table. This deletes the time series containing the target reference tag data and the first monitoring indicator from the time-series database. Since the time series deleted from the time-series database is determined by the target reference tag data and the first monitoring indicator, which are the tag data and associated monitoring indicators maintained in the data maintenance table that are more than a first time period since their update, what is deleted is expired data that has not been updated for a long time in the time-series database. This ensures the usable space of the time-series database and solves the problem of a large amount of expired data in the time-series database affecting database performance.

[0057] In some optional examples, in S110, the first data processing request may be issued by the user through the maintenance terminal. The first data processing request may also be received by the touch module of the data processing device, or it may be a pre-set periodic task. When the start conditions of the periodic task are met, the data processing device automatically initiates the first data processing request.

[0058] It should be noted that the first data processing request is a request to delete expired time series data in the time series database. The time series database involved in this application embodiment can be a database with the aforementioned technical problems. For example, the aforementioned time series database can be Alibaba Cloud Time Series Database, or it can be an open-source time series database such as Open TSDB.

[0059] The aforementioned data maintenance table can be maintained and updated periodically. The updated content includes the reference label data associated with each monitoring indicator in the time series database, the latest update time of the reference label data, and may also include the data status and data generation time of the reference label data.

[0060] It should also be noted that the data maintenance table can be built and maintained based on the Elasticsearch search engine, or it can be built and maintained using databases such as MySQL or PostgreSQL. When the data maintenance table is updated and maintained on a regular basis using the Elasticsearch search engine, its query processing speed is faster and its performance is better.

[0061] When retrieving the latest tag data associated with each monitoring indicator from the time series database, these monitoring indicators and the latest associated tag data can be stored in the data maintenance table for future comparison as reference data. In other words, the tag data stored in the periodically maintained table is the benchmark for comparison and is the reference tag data.

[0062] In some optional examples, please see Figure 2 Prior to S110, the periodic maintenance process for the data maintenance table could include:

[0063] S210, if the system time reaches the preset timed update condition, then the reference tag data associated with each monitoring indicator is obtained from the time series database. The reference tag data is the tag data recorded by the time series database in the second time period before the system time, and the second time period is shorter than the first time period.

[0064] S220, based on each of the monitoring indicators and the associated reference tag data, update the data maintenance table corresponding to the time series database.

[0065] It should be noted that the update process of the data maintenance table runs through the entire data processing logic, and it can also occur after S110.

[0066] Here, the system time is the real-time time of the data processing device, and the preset timed update condition can be that the timed update time conforms to a certain mathematical rule. For example, when the interval between the system time and the latest update time reaches 5 days, it indicates that the preset timed update condition is met.

[0067] In this example, the reference tag data is limited to the tag data recorded in the time series database within the second time period before the system time, obtained when the data maintenance table is updated. The value of the second time period is set to be relatively small, such as 1 hour or other values.

[0068] For example, when the number of access records of tag data corresponding to a certain monitoring indicator within 1 hour obtained from the time series database exceeds the limit, the query conditions can be modified to obtain tag data corresponding to the monitoring indicator within 30 minutes or 10 minutes.

[0069] Taking Alibaba Cloud Time Series Database as an example, the process of retrieving reference tag data from the Time Series Database can be as follows: By calling the "api / suggest" interface, all monitoring metrics can be obtained from the monitoring metric mapping table of the Time Series Database (i.e., the tsdb-uid table mentioned above), and these monitoring metrics can be cached in a list. Then, the individual monitoring metrics in the list are read in a loop, and the "api / query" interface is accessed to query the monitoring data of that monitoring metric in the second time period before the system time.

[0070] The aforementioned monitoring data includes monitoring metrics, tag data associated with the monitoring metrics (i.e., tag combinations), and monitoring metric data corresponding to the tag data. The monitoring metrics and tag data associated with the monitoring metrics can be extracted to update the data maintenance table.

[0071] Updating the data maintenance table based on the extracted monitoring metrics and the associated tag data can include:

[0072] Determine whether any data field in the data maintenance table includes a second monitoring indicator and the reference tag data associated with the second monitoring indicator, wherein the second monitoring indicator is any of the monitoring indicators.

[0073] When none of the data fields in the data maintenance table include the second monitoring indicator and the reference tag data associated with the second monitoring indicator, a first data field is created. The first data field includes the second monitoring indicator, the reference tag data associated with the second monitoring indicator, and the creation time of the first data field.

[0074] When any data field in the data maintenance table includes a second monitoring indicator and the reference tag data associated with the second monitoring indicator, the update time of the reference tag data associated with the second monitoring indicator is set according to the current system time.

[0075] In the example above, the monitoring metric and associated tag data are used as keywords to query whether the keyword exists in the data maintenance table. If the keyword exists, meaning that the monitoring metric and associated reference tag data were recorded in a previous update, the update time can be updated to the current system time, thus updating the data in the data maintenance table.

[0076] If the keyword does not exist in the data maintenance table, meaning that the monitoring metric and associated tag data have not been recorded in the previous update records of the data maintenance table, it can be stored in the data maintenance table (at this time, the associated tag data is the reference tag data), and the current system time can be set to its update time and creation time.

[0077] These examples demonstrate the update scheme for the data maintenance table corresponding to the time series database. This periodic update scheme is the basis for the timely deletion of expired time series in the time series database, thus ensuring that expired data can be identified in a timely and accurate manner.

[0078] In some optional examples, a unique identifier can be set in the data maintenance table for each keyword (i.e., a single monitoring metric and its associated reference tag data). This unique identifier can be used for identification and can be calculated based on the monitoring metric and the associated reference tag data. For example, an MD5 hash can be performed on a monitoring metric and its associated currently used tag data to obtain and store this unique identifier.

[0079] Therefore, when determining whether the data maintenance table includes the second monitoring indicator and its associated reference tag data, a first identifier can be obtained by calculating the second monitoring indicator and the reference tag data associated with the second monitoring indicator. Then, it is determined whether there is a unique identifier in any data field of the data maintenance table that matches the first identifier; wherein, when the unique identifiers in all data fields of the data maintenance table do not match the first identifier, then none of the data fields of the data maintenance table include the second monitoring indicator and the reference tag data associated with the second monitoring indicator; otherwise, the second monitoring indicator and its associated reference tag data were recorded during the creation and update process of the data maintenance table.

[0080] In these optional examples, by setting and matching identifiers, it is possible to quickly retrieve whether the data maintenance table includes monitoring metrics and their associated tag data, improving the speed of information retrieval and making it convenient and fast.

[0081] In some alternative examples, a running status field can also be created that corresponds to both the monitoring metric and the associated tag data; for example, the running status field could be "status". For instance, initially, the running status field could indicate a running status, such as "running".

[0082] Understandably, since the data maintenance table is updated periodically, after several updates and maintenance, the reference label data corresponding to each monitoring indicator maintained by the data maintenance table represents the usage of time series in the time database.

[0083] If the update time of the target reference tag data associated with the first monitoring indicator exceeds the first time period (the first time period is longer than the second time period, for example, the first time period can be 90 days), it means that the time series containing the first monitoring indicator and its associated target reference tag data has not been updated for too long. This may be caused by reasons such as the monitoring equipment being taken offline or the monitoring indicator being renamed. Therefore, the target reference tag data and the time series containing the first monitoring indicator in the time series database can be deleted. Based on the update time in the data maintenance table, the expired time series in the time series database are deleted in a timely manner, preserving the available space in the time series database and solving the problem of a large amount of expired data being stored in the time series database, which affects the database performance.

[0084] Taking Alibaba Cloud Time Series Database as an example, the process of deleting a time series in the time series database can include: generating a delete JSON field in the time series database, and deleting the target reference label data and the corresponding first monitoring indicator time series by accessing "api / delete_meta".

[0085] In some alternative examples, in addition to having a running status field set in the data maintenance table, after deleting the target reference label data and the corresponding time series of the first monitoring indicator in the time series database, it may also include:

[0086] Set the running status of the second data field in the data maintenance table to the deleted status. The second data field includes the target reference label data and the first monitoring indicator.

[0087] Using the "status" field as an example again, the "status" field can be set to "Deleted".

[0088] In the example above, by updating the running status of the data fields in the data maintenance table in a timely manner, we can understand the running status of each time series maintained in the data maintenance table in a timely manner. When deleting time series, we can also remove these expired time series as early as possible, which helps to understand the timely status of each time series in the entire time series database.

[0089] In other optional examples, please continue to see Figure 2 Following S220, it may also include:

[0090] S230, upon receiving the second data processing request, retrieve all target tag data corresponding to the target monitoring indicator from the time series database, and retrieve reference tag data corresponding to the target monitoring indicator from the data maintenance table, wherein each monitoring indicator includes the target monitoring indicator.

[0091] S240, filter out the difference label data that is not in the reference label data from all the target label data, and delete the difference label data in the time series database.

[0092] It should be noted that this request is applicable to scenarios where the creation times of the time-series database and the data maintenance table differ significantly. Therefore, this second data processing request is a request to delete historical time series data from the data maintenance table. The request can also be sent to the touch module of the data processing device or the maintenance terminal; the specific settings can be similar to those for the first data processing request.

[0093] Based on the foregoing analysis, since the data maintenance table is updated periodically, after several updates and maintenance cycles, the reference tag data corresponding to each monitoring indicator maintained in the data maintenance table represents the usage of time series data in the time database. However, there are other usage scenarios: some time series data did not appear in the data maintenance table before its creation due to reasons such as device shutdown or renaming. Therefore, all target tag data corresponding to the target monitoring indicator can be retrieved from the monitoring indicator mapping table of the time series database. The target monitoring indicator can be specified in the second data processing request, or it can be all or part of the monitoring indicators.

[0094] Continuing with the previous example, the "api / suggest" interface can be used to query and retrieve all target label data corresponding to the target monitoring metric. This target label data must include at least the label values ​​corresponding to each monitoring metric. Finally, using the reference label data associated with the target monitoring metric in the data maintenance table as a comparison benchmark, the label values ​​of the target monitoring metric not in the data maintenance table are deleted. This process addresses the legacy time series data collected before the data maintenance table was built.

[0095] In the example above, by cleaning up historical time series data and combining it with the first data processing request, the space of the time series database was cleaned up to the maximum extent, ensuring the space utilization of the time series database and improving its performance.

[0096] The above text combines Figures 1 to 2 The data processing method according to embodiments of the present invention is described in detail below, and will be combined with Figure 3 The data processing apparatus of the present invention will be described in detail in the embodiments thereof.

[0097] Please refer to Figure 3 This application also provides a data processing apparatus, which may include:

[0098] The first acquisition module 310 is used to acquire the data maintenance table corresponding to the time series database when a first data processing request is received. The data maintenance table maintains the update time of the reference tag data associated with each monitoring indicator in the time series database.

[0099] The second acquisition module 320 is used to acquire target reference tag data that is more than a first time period away from the update time from the data maintenance table and a first monitoring indicator associated with the target reference tag data. Each monitoring indicator includes the first monitoring indicator, and the reference tag data includes the target reference tag data.

[0100] The deletion module 330 is used to delete the target reference label data and the time series containing the first monitoring indicator from the time series database.

[0101] In an alternative example, the device may also include:

[0102] The third acquisition module is used to acquire reference tag data associated with each monitoring indicator from the time series database if the system time reaches the preset timed update condition. The reference tag data is the tag data recorded by the time series database in the second time period before the system time, and the second time period is shorter than the first time period.

[0103] The update module is used to update the data maintenance table corresponding to the time series database based on each of the monitoring indicators and the associated reference tag data.

[0104] In yet another alternative example, the device may also include:

[0105] The fourth acquisition module is used to, upon receiving the second data processing request, acquire all target tag data corresponding to the target monitoring indicator from the time series database, and acquire reference tag data corresponding to the target monitoring indicator from the data maintenance table, wherein each monitoring indicator includes the target monitoring indicator.

[0106] The filtering and deletion module is used to filter out the difference label data that is not in the reference label data from all the target label data, and delete the difference label data in the time series database.

[0107] In yet another alternative example, the update module includes:

[0108] The judgment unit is used to determine whether any data field of the data maintenance table includes a second monitoring indicator and the reference tag data associated with the second monitoring indicator, wherein the second monitoring indicator is any of the monitoring indicators.

[0109] A creation unit is used to create a first data field when none of the data fields in the data maintenance table include the second monitoring indicator and the reference tag data associated with the second monitoring indicator. The first data field includes the second monitoring indicator, the reference tag data associated with the second monitoring indicator, and the creation time of the first data field.

[0110] The setting unit is used to set the update time of the reference tag data associated with the second monitoring indicator according to the current system time when any data field of the data maintenance table includes the second monitoring indicator and the reference tag data associated with the second monitoring indicator.

[0111] In yet another alternative example, the device further includes:

[0112] Set the running status of the second data field in the data maintenance table to the deleted status. The second data field includes the target reference label data and the first monitoring indicator.

[0113] In another alternative example, all data fields in the data maintenance table include a unique identifier calculated based on monitoring metrics and associated reference tag data;

[0114] The judgment unit includes:

[0115] A calculation subunit is used to calculate the second monitoring indicator and the reference tag data associated with the second monitoring indicator to obtain a first identifier;

[0116] The judgment subunit is used to determine whether there is a unique identifier in any data field of the data maintenance table that matches the first identifier; wherein when the unique identifiers in all data fields of the data maintenance table do not match the first identifier, then none of the data fields of the data maintenance table include the second monitoring indicator and the reference tag data associated with the second monitoring indicator.

[0117] Figure 4 A schematic diagram of the hardware structure of a data processing device provided in an embodiment of this application is shown. The data processing device may include a processor 401 and a memory 402 storing computer program instructions.

[0118] Specifically, the processor 401 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0119] Memory 402 may include mass storage for data or instructions. For example, and not limitingly, memory 402 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 402 may include removable or non-removable (or fixed) media. Where appropriate, memory 402 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 402 is non-volatile solid-state memory.

[0120] In certain embodiments, memory 402 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory 402 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described in the data processing method according to the foregoing aspects of this disclosure.

[0121] The processor 401 implements any of the data processing methods described in the above embodiments by reading and executing computer program instructions stored in the memory 402.

[0122] In one example, the data processing device may further include a communication interface 403 and a bus 410. Wherein, as... Figure 4 As shown, the processor 401, memory 402, and communication interface 403 are connected through bus 410 and complete communication with each other.

[0123] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0124] Bus 410 includes hardware, software, or both, that couples components of a data processing device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 410 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0125] The data processing device can execute the data processing method described in the embodiments of this application, thereby achieving the combination Figures 1 to 3 The data processing methods and apparatus described.

[0126] Furthermore, in conjunction with the data processing methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the data processing methods in the above embodiments.

[0127] In addition, this application also provides a computer program product, including a computer program, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0128] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0129] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0130] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0131] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0132] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A data processing method, characterized in that, include: Upon receiving the first data processing request, the data maintenance table corresponding to the time series database is obtained. The data maintenance table maintains the update time of the reference tag data associated with each monitoring indicator in the time series database. The target reference label data that is more than a first time period away from the update time is obtained from the data maintenance table, and the first monitoring indicator associated with the target reference label data is obtained. Each monitoring indicator includes the first monitoring indicator, and the reference label data includes the target reference label data. Delete the target reference label data and the time series containing the first monitoring indicator from the time series database; Before receiving the first data processing request, the method further includes: If the system time reaches the preset timed update condition, the reference tag data associated with each monitoring indicator is obtained from the time series database. The reference tag data is the tag data recorded by the time series database in the second time period before the system time, and the second time period is shorter than the first time period. Update the data maintenance table corresponding to the time series database based on each of the monitoring indicators and the associated reference tag data; After updating the data maintenance table corresponding to the time-series database based on each of the monitoring indicators and the associated reference tag data, the method further includes: Upon receiving a second data processing request, all target label data corresponding to the target monitoring indicator are obtained from the time series database, and reference label data corresponding to the target monitoring indicator are obtained from the data maintenance table. Each monitoring indicator includes the target monitoring indicator. Filter out the difference label data that is not in the reference label data from all the target label data, and delete the difference label data from the time series database.

2. The method according to claim 1, characterized in that, The step of updating the data maintenance table corresponding to the time-series database based on each of the monitoring indicators and the associated reference tag data includes: Determine whether any data field in the data maintenance table includes a second monitoring indicator and the reference tag data associated with the second monitoring indicator, wherein the second monitoring indicator is any of the monitoring indicators; When none of the data fields in the data maintenance table include the second monitoring indicator and the reference tag data associated with the second monitoring indicator, a first data field is created. The first data field includes the second monitoring indicator, the reference tag data associated with the second monitoring indicator, and the creation time of the first data field. When any data field in the data maintenance table includes a second monitoring indicator and the reference tag data associated with the second monitoring indicator, the update time of the reference tag data associated with the second monitoring indicator is set according to the current system time.

3. The method according to claim 2, characterized in that, After deleting the target reference label data and the time series containing the first monitoring indicator from the time series database, the method further includes: Set the running status of the second data field in the data maintenance table to the deleted status. The second data field includes the target reference label data and the first monitoring indicator.

4. The method according to claim 2, characterized in that, All data fields in the data maintenance table include a unique identifier calculated based on the monitoring metric and associated reference tag data; determining whether any data field in the data maintenance table includes the second monitoring metric and the associated reference tag data includes: The first identifier is obtained by calculating the second monitoring indicator and the reference tag data associated with the second monitoring indicator; Determine whether there exists a unique identifier in any data field of the data maintenance table that matches the first identifier; wherein when the unique identifiers in all data fields of the data maintenance table do not match the first identifier, then none of the data fields of the data maintenance table include the second monitoring indicator and the reference tag data associated with the second monitoring indicator.

5. A data processing apparatus, characterized in that, include: The first acquisition module is used to acquire the data maintenance table corresponding to the time series database when a first data processing request is received. The data maintenance table maintains the update time of the reference tag data associated with each monitoring indicator in the time series database. The second acquisition module is used to acquire target reference tag data that is more than a first time period away from the update time from the data maintenance table and a first monitoring indicator associated with the target reference tag data. Each monitoring indicator includes the first monitoring indicator, and the reference tag data includes the target reference tag data. The deletion module is used to delete the target reference label data and the time series containing the first monitoring indicator from the time series database; The third acquisition module is used to acquire reference tag data associated with each monitoring indicator from the time series database if the system time reaches the preset timed update condition. The reference tag data is the tag data recorded by the time series database in a second time period before the system time, and the second time period is shorter than the first time period. The update module is used to update the data maintenance table corresponding to the time series database based on each of the monitoring indicators and the associated reference tag data. The fourth acquisition module is used to, upon receiving the second data processing request, acquire all target tag data corresponding to the target monitoring indicator from the time series database, and acquire reference tag data corresponding to the target monitoring indicator from the data maintenance table, wherein each monitoring indicator includes the target monitoring indicator; The filtering and deletion module is used to filter out the difference label data that is not in the reference label data from all the target label data, and delete the difference label data in the time series database.

6. A data processing device, characterized in that, The data processing device includes a memory, a processor, and a data processing program stored in the memory and running on the processor, the data processing program performing the data processing method as described in any one of claims 1 to 4.

7. A computer storage medium, characterized in that, When the computer storage medium is executed by the processor, it implements the data processing method according to any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the data processing method as described in any one of 1 to 4.

Images