Cache data monitoring management method and device, equipment and storage medium

Abstract

The application discloses a kind of monitoring management methods, devices and equipment of cache data and storage medium.The method comprises: the cache data generated in the process of database executing database statement is written in preset cache space;The number of preset cache space is at least one;When the cache data write-in quantity of each preset cache space meets disk write-in condition, the cache data in its cache space is written into disk;Determine the data accumulation amount of the cache data written into disk by each preset cache space under the current disk write-in period;According to the data accumulation amount corresponding to each preset cache space, it is determined whether to terminate the cache data generated by database, to obtain termination judgment result;According to termination judgment result, control database to continue generating or terminate generating cache data, to realize the monitoring management of cache data.The embodiment of the application realizes the automatic monitoring management of cache data, solves the business production safety problem caused by temporary file being too large.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and storage medium for monitoring and managing cached data. Background Technology

[0002] In the field of database technology, temporary files in databases are often used to cache data generated during the execution of database statements, such as `ORDER BY` and `GROUP BY` statements. In certain database execution scenarios, the temporary files created by the database can reach tens or even hundreds of gigabytes in size. Excessively large temporary files can lead to insufficient disk space, causing a series of serious problems such as transaction commit failures, master-slave failover, and business interruptions, which greatly affect business production security.

[0003] Currently, the main approach to monitoring, alerting, and repairing disk status is a combination of deploying external monitoring tools and manual intervention. However, this method has significant drawbacks. When the database is instantly written to a large number of temporary files, real-time monitoring and intervention are not possible, and the problem of excessively large temporary files causing business production security issues cannot be fundamentally resolved. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for monitoring and managing cached data, so as to realize automated monitoring and management of cached data and solve the business production security problem caused by excessively large temporary files.

[0005] According to one aspect of the present invention, a method for monitoring and managing cached data is provided, the method comprising:

[0006] The cached data generated during the execution of database statements is written into a preset cache space; the number of preset cache spaces is at least one.

[0007] When the amount of cached data written in each of the preset cache spaces meets the disk write conditions, the cached data in its own cache space is written to the disk.

[0008] Determine the cumulative amount of cached data written to the disk in each of the preset cache spaces during the current disk write cycle;

[0009] Based on the accumulated data corresponding to each preset cache space, determine whether to terminate the generation of cache data in the database, and obtain a termination judgment result;

[0010] Based on the termination judgment result, the database is controlled to continue generating or terminate the generation of cached data, so as to realize the monitoring and management of the cached data.

[0011] According to another aspect of the present invention, a monitoring and management device for cached data is provided, comprising:

[0012] A cache data writing module is used to write cache data generated during the execution of database statements into a preset cache space; the number of preset cache spaces is at least one.

[0013] The disk writing module is used to write the cached data in its own cache space to the disk when the cached data write amount in each of the preset cache spaces meets the disk writing conditions.

[0014] The data accumulation determination module is used to determine the data accumulation of cached data written to the disk in each of the preset cache spaces during the current disk write cycle;

[0015] The judgment result determination module is used to determine whether to terminate the generation of cached data in the database based on the data accumulation corresponding to each preset cache space, and to obtain a termination judgment result;

[0016] The monitoring and management module is used to control the database to continue generating or terminate the generation of cached data based on the termination judgment result, so as to realize the monitoring and management of the cached data.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the monitoring and management method for cached data according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the monitoring and management method for cached data as described in any embodiment of the present invention.

[0022] The technical solution of this invention writes cached data generated during the execution of database statements into a preset cache space; when the cached data write volume of each preset cache space meets the disk write condition, the cached data in its own cache space is written to the disk; the cumulative amount of cached data written to the disk in each preset cache space during the current disk write cycle is determined; based on the cumulative amount of data corresponding to each preset cache space, it is determined whether to terminate the generation of cached data by the database, and a termination judgment result is obtained; based on the termination judgment result, the database is controlled to continue generating or terminate the generation of cached data, so as to achieve monitoring and management of cached data. The above technical solution, by writing the cached data generated by the database to the disk in a temporary manner through preset cache spaces, and determining whether to control the continued generation of cached data based on the cumulative amount of data, realizes automated monitoring and management of cached data, and solves the security problem of business production process caused by excessively large temporary files.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a method for monitoring and managing cached data according to Embodiment 1 of the present invention;

[0026] Figure 2 This is a flowchart of a method for monitoring and managing cached data according to Embodiment 2 of the present invention;

[0027] Figure 3 This is a flowchart of a method for monitoring and managing cached data according to Embodiment 3 of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of a monitoring and management device for cached data provided in Embodiment 4 of the present invention;

[0029] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the monitoring and management method for cached data according to an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] Example 1

[0033] Figure 1 This is a flowchart illustrating a method for monitoring and managing cached data according to Embodiment 1 of the present invention. This embodiment is applicable to monitoring and managing cached data generated during the execution of database statements. The method can be executed by a cached data monitoring and management device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0034] S110. Write the cached data generated during the execution of database statements into a preset cache space; the number of preset cache spaces is at least one.

[0035] The database statements executed by the database can be, for example, order by or group by statements. In addition to generating cached data during the execution of database statements, the database may also generate cached data when executing its internal database methods.

[0036] The preset cache space can be a memory space used for temporary storage of cached data. The preset cache space can be created in advance; for example, the preset cache space can be an IO cache.

[0037] It should be noted that there can be at least one preset cache space. During the process of writing cache data to each preset cache space, each generation of cache data is written to only one preset cache space. The second generation of cache data can be written to other preset cache spaces, but again, it is only written to one preset cache space. Specifically, a round-robin method can be used to write the generated cache data to each preset cache space in sequence.

[0038] It's important to note that the cache space is essentially the I / O cache in memory. Each I / O cache corresponds to a temporary file. The cache is created when needed and destroyed when no longer required. When the temporary file no longer needs to be deleted, the corresponding cache is destroyed.

[0039] For example, if there are preset cache spaces IO CACHE A, IO CACHE B, and IO CACHE C; and 50K of cached data is generated during the execution of a database statement, then the 50K of cached data can be written to preset cache space IO CACHE A; if 75K of cached data is generated during the subsequent execution of a database statement, then according to the round-robin rule, the generated 75K of cached data can be written to preset cache space IO CACHE B; similarly, a certain amount of cached data generated during the next execution of a database statement can be written to preset cache space IO CACHE C according to the round-robin rule.

[0040] S120. When the amount of cached data written in each preset cache space meets the disk write conditions, write the cached data in its own cache space to the disk.

[0041] The memory size of each preset cache space may be the same or different, and correspondingly, the disk write conditions for each cache space may be the same or different.

[0042] For example, for any preset cache space, if the space size of the preset cache space is 64K, the corresponding disk write condition can be that when the amount of cache data written to the preset cache space reaches 64K, 64K of cache data is written to the disk. Here, the cumulative data amount of any preset cache space can represent the size of a single cache temporary file written to.

[0043] It should be noted that if the amount of cached data written is less than the size of the preset cache space during the process of writing cached data to the preset cache space, the system can wait for the next cached data to be written until the amount of cached data written reaches the preset disk write condition, that is, after the space memory is filled to 64K, the cached data in its own cache space is written to the disk.

[0044] It should be noted that cached data is typically written to disk sequentially, with a small amount of data written per operation but a large number of total writes. Furthermore, the amount of cached data written to disk each time is not fixed, and obtaining the size of the temporary file on disk consumes performance. Therefore, this application transforms the sequential write operation into an aligned write operation by constructing a preset cache space to temporarily write cached data, and writing a certain amount of cached data to disk when certain write conditions are met. Moreover, if the preset cache space is an IO cache, it can itself record the amount of cached data written and the number of writes, without consuming performance to determine the size of the temporary file on disk.

[0045] S130. Determine the cumulative amount of cached data written to the disk in each preset cache space during the current disk write cycle.

[0046] The cumulative data amount can be the cumulative amount of cached data written to the disk in each disk write cycle in the preset cache space.

[0047] For example, if the preset cache space writes 64K of cached data to the disk in each disk write cycle, the total number of writes in the current disk write cycle can be determined. If the total number of writes is 3, the cumulative amount of cached data written to the disk by the preset cache space is 192K.

[0048] S140. Based on the accumulated data amount corresponding to each preset cache space, determine whether to terminate the generation of cache data in the database and obtain the termination judgment result.

[0049] For example, if the accumulated data amount corresponding to each preset cache space meets the preset termination condition, then it is determined to terminate the generation of cache data in the database, and a judgment result of terminating cache data is obtained; if the accumulated data amount corresponding to each preset cache space does not meet the preset termination condition, then it is determined to continue generating cache data in the database, and a judgment result of continuing to generate cache data is obtained.

[0050] The termination condition can be preset by relevant technical personnel. For example, the termination condition could be that the accumulated data in a preset number of preset cache spaces exceeds a set cache data volume threshold. The preset number and cache data volume threshold can be preset by relevant technical personnel. The preset number can be set based on the total number of preset cache spaces; for example, the preset number could be 30% of the total number. If the total number of preset cache spaces is 20, then the preset number could be 6. For example, the cache data volume threshold could be set to 16M.

[0051] S150. Based on the termination judgment result, control the database to continue generating or terminate the generation of cached data, so as to realize the monitoring and management of cached data.

[0052] For example, if the termination judgment result is to stop the database from generating cached data, a corresponding termination command can be generated and sent to the database to control the database to stop generating cached data, thereby achieving monitoring and management of cached data. If the termination judgment result is that the database can continue to generate cached data, no response processing is required, and the database can continue to generate cached data, thereby achieving monitoring and management of cached data.

[0053] The technical solution of this invention writes cached data generated during the execution of database statements into a preset cache space; when the cached data write volume of each preset cache space meets the disk write condition, the cached data in its own cache space is written to the disk; the cumulative amount of cached data written to the disk in each preset cache space during the current disk write cycle is determined; based on the cumulative amount of data corresponding to each preset cache space, it is determined whether to terminate the generation of cached data by the database, and a termination judgment result is obtained; based on the termination judgment result, the database is controlled to continue generating or terminate the generation of cached data, so as to achieve monitoring and management of cached data. The above technical solution, by writing the cached data generated by the database to the disk in a temporary manner through preset cache spaces, and determining whether to control the continued generation of cached data based on the cumulative amount of data, realizes automated monitoring and management of cached data, and solves the security problem of business production process caused by excessively large temporary files.

[0054] Example 2

[0055] Figure 2 This is a flowchart of a method for monitoring and managing cached data provided in Embodiment 2 of the present invention. This embodiment is an optimization and improvement based on the above technical solutions.

[0056] Furthermore, the step "determine whether to terminate the generation of cached data in the database based on the accumulated data amount corresponding to each preset cache space, and obtain a termination judgment result" is refined to "determine the total number of times the cached data is written to the disk; if the accumulated data amount of any preset cache space meets the preset accumulated amount judgment condition, or the total number of writes meets the preset total number of writes judgment condition, then determine the total amount of cached data written to the disk based on the accumulated data amount corresponding to each preset cache space; based on the total amount of data, determine whether to terminate the generation of cached data in the database, and obtain a termination judgment result." This improves the method for determining the termination judgment result. It should be noted that parts not described in detail in this embodiment of the invention can be referred to in the descriptions of other embodiments.

[0057] like Figure 2 As shown, the method includes the following specific steps:

[0058] S210. Write the cached data generated during the execution of database statements into a preset cache space; the number of preset cache spaces is at least one.

[0059] S220. When the amount of cached data written in each preset cache space meets the disk write conditions, write the cached data in its own cache space to the disk.

[0060] S230. Determine the cumulative amount of cached data written to the disk in each preset cache space during the current disk write cycle.

[0061] S240. Determine the total number of writes of cached data to the disk.

[0062] The total number of writes can be the sum of all cached data written to disk from the current write cycle up to the current preset cache space. The total number of writes can also represent the number of times the cache writes to temporary files; this value is continuously increasing.

[0063] S250. If the data accumulation in any preset cache space meets the preset accumulation judgment condition, or the total number of writes meets the preset total number of write judgment condition, then the total amount of cached data written to the disk is determined according to the data accumulation corresponding to each preset cache space.

[0064] The criteria for determining the cumulative amount can be preset by relevant technical personnel. For example, the criteria could be that the cumulative amount of data in the preset cache space is not less than a preset cache data size threshold. This cache data size threshold can also be preset by relevant technical personnel. For example, the cache data size threshold could be 16MB.

[0065] The total amount of cached data can be the sum of the accumulated data in all preset cache spaces. That is, in the current write cycle, the total amount of cached data written to disk from all preset cache spaces. The total data amount can also represent the total size of all temporary files. When a temporary file does not need to be deleted, its size needs to be subtracted from the total data amount; this value can be increased or decreased.

[0066] For example, if the accumulated data in any preset cache space meets the preset accumulated data condition, or the total number of writes meets the preset total number of writes condition, then the total amount of cached data written to the disk is determined; if the accumulated data in any preset cache space does not meet the preset accumulated data condition, and the total number of writes does not meet the preset total number of writes condition, then data continues to be cached in the preset cache space.

[0067] S260. Based on the total amount of data, determine whether to terminate the generation of cached data in the database, and obtain the termination judgment result.

[0068] For example, if the total amount of data is not less than a preset total threshold, the generation of cached data in the database will be terminated; if the total amount of data is less than the preset total threshold, the generation of cached data will continue.

[0069] S270. Based on the termination judgment result, control the database to continue generating or terminate the generation of cached data, so as to realize the monitoring and management of cached data.

[0070] In one optional embodiment, determining whether to terminate the generation of cached data in the database based on the total amount of data, and obtaining a termination judgment result, includes: if the total amount of data does not meet a preset total amount judgment condition, then determining to terminate the generation of cached data in the database; correspondingly, according to the termination judgment result, controlling the database to continue generating or terminate the generation of cached data, so as to realize the monitoring and management of cached data, including: if the termination judgment result is to determine to terminate the generation of cached data in the database, then sending a generation statement termination instruction to the database, controlling the database to terminate the generation of cached data according to the statement termination instruction, so as to realize the monitoring and management of cached data.

[0071] The total quantity judgment criteria can be preset by relevant technical personnel. For example, the total quantity judgment criteria could be that the total data volume is less than a preset total quantity threshold. The total quantity threshold and the total number of times threshold can also be preset by relevant technical personnel. For example, the total quantity threshold could be 32M.

[0072] For example, if the total amount of data does not meet the preset total amount judgment condition, that is, the total amount of data is not less than the preset total amount threshold, then it is determined to terminate the generation of cached data in the database. If it is determined to terminate the generation of cached data in the database, a statement termination instruction is generated and sent to the database to control the database to terminate the generation of cached data according to the statement termination instruction, thereby realizing the monitoring and management of cached data.

[0073] In one optional embodiment, determining whether to terminate the generation of cached data in the database based on the total amount of data, and obtaining a termination judgment result, includes: if the total amount of data meets a preset total amount judgment condition, then determining that the database continues to generate cached data; correspondingly, according to the termination judgment result, controlling the database to continue generating or terminate the generation of cached data, so as to realize the monitoring and management of cached data, including: if the termination judgment result is that the database continues to generate cached data, then controlling the database to continue generating cached data, so as to realize the monitoring and management of cached data.

[0074] For example, if the total amount of data meets a preset total amount judgment condition, that is, the total amount of data is less than a preset total amount threshold, then it can be determined that the database can continue to generate cached data. If the termination judgment result is that the database continues to generate cached data, then the database is controlled to continue generating cached data, so as to realize the monitoring and management of cached data.

[0075] It should be noted that, in order to further improve the effective management of cached data and enhance the security of the cached data management process, the performance conditions of the disk can also be fully considered during the monitoring and management of cached data.

[0076] In an optional embodiment, if the total amount of data does not meet the preset total amount judgment condition, the available space capacity and available space ratio of the disk are determined under the current disk write cycle; if the available space capacity meets the preset capacity judgment condition and the available space ratio meets the preset ratio judgment condition, then the database is determined to continue generating cached data.

[0077] The available space capacity can be the amount of space that the disk can use for cached data storage during the current disk write cycle; the available space percentage can be the ratio between the available space capacity of the disk during the current disk write cycle and the total disk space capacity.

[0078] The capacity judgment condition and the percentage judgment condition can be preset by relevant technical personnel. For example, the capacity judgment condition can be that the available space capacity is less than a preset capacity threshold; the percentage judgment condition can be that the available space percentage is less than a preset percentage threshold.

[0079] For example, if the available space capacity meets the preset capacity judgment condition (i.e., the available space capacity is less than the preset capacity threshold), and the available space percentage meets the preset percentage judgment condition (i.e., the available space percentage is less than the preset percentage threshold), then it is determined that the database will continue to generate cached data.

[0080] Optionally, if the available space capacity does not meet the preset capacity judgment condition, or the available space percentage does not meet the preset percentage judgment condition, then the generation of cached data in the database will be terminated.

[0081] For example, if the available space capacity does not meet the preset capacity judgment condition, that is, the available space capacity is not less than the preset capacity threshold, or the available space percentage does not meet the preset percentage judgment condition, that is, the available space percentage is not less than the preset percentage threshold, then the database will stop generating cached data.

[0082] In one specific embodiment, it should be noted that the amount of temporary data generated at one time is usually small. Therefore, it can be cached in the cache first. If the capacity of a single cache is 64K, the above process is repeated. When the cache capacity is full, the temporary data cached in the cache is written to a temporary file on the disk. One of the main functions of the cache is to merge small amounts of I / O, reduce the number of disk writes, and reduce the performance impact to some extent. The first time 30K of temporary data is generated, it is all cached in the cache; the second time 32K of temporary data is generated, it is also all cached in the cache; the third time 50K of temporary data is generated, the cache capacity has already used 62K, at this time 2K of temporary data is cached in the cache, the cache capacity is full, and the 64K of temporary data cached in the cache is written to the temporary file on the disk (the first disk write). 48K of the original 50K of temporary data remains, which is cached in the cache. The fourth time 500K of temporary data is generated (the case of a large amount of data generated at one time is rare, but it does exist). The cache has already used 48KB of data. At this point, 16KB of temporary data is cached in the cache. When the cache is full, the remaining 64KB of temporary data is written to a temporary disk file (second disk write). Of the original 500KB of temporary data, 484KB remains. Since 484KB / 64KB = 7, another 64KB * 7 = 448KB of temporary data is written directly to the temporary disk file (third disk write). Of the original 500KB of temporary data, 500KB - 16KB - 448KB = 36KB remains and continues to be cached in the cache.

[0083] The disk temporary file write volume for the single cache mentioned above is 64K + 64K + 448K = 576K, with 3 disk writes. There isn't a strict correlation between the write volume and the write count. The significance of counting the write count is that in concurrent scenarios, each thread may create temporary files, but each individual temporary file is very small and deleted after use. In this scenario, the condition for judging the cumulative data size of a single cache might not be met. Therefore, adding a write count check ensures that subsequent checks on whether the total size of temporary files exceeds the limit are triggered.

[0084] This embodiment's technical solution determines the total amount of cached data written to the disk and the total number of writes. When it is determined that the accumulated data in any preset cache space meets a preset accumulation judgment condition, it determines whether to terminate the generation of cached data in the database based on the accumulated data corresponding to each preset cache space or the total number of writes meeting a preset total number of write judgment condition, thus obtaining a termination judgment result. This achieves accurate judgment on whether to terminate the generation of cached data in the database, improves the accuracy of determining the termination judgment result, realizes automated monitoring and management of cached data, and solves the security problem in the business production process caused by excessively large temporary files.

[0085] Example 3

[0086] Figure 3 This is a flowchart illustrating a method for monitoring and managing cached data according to Embodiment 3 of the present invention. This embodiment provides a preferred example based on the above embodiments.

[0087] like Figure 3 As shown, the method includes the following specific steps:

[0088] S301. Write the cached data generated during the execution of database statements into the preset cache space.

[0089] S302. When the amount of cached data written in each preset cache space meets the disk write conditions, write the cached data in its own cache space to the disk.

[0090] S303. Determine the cumulative amount of cached data written to the disk in each preset cache space during the current disk write cycle.

[0091] S304. Determine the total number of writes of cached data to the disk.

[0092] S305. If the data accumulation in any preset cache space meets the preset accumulation judgment condition, or the total number of writes meets the preset total number of write judgment condition, then the total amount of cached data written to the disk is determined according to the data accumulation corresponding to each preset cache space.

[0093] S306. If the total amount of data does not meet the preset total amount judgment condition, then determine to terminate the generation of cached data in the database.

[0094] S307. If it is determined to terminate the generation of cached data in the database, a termination command is sent to the database to control the database to terminate the generation of cached data according to the termination command, so as to realize the monitoring and management of cached data.

[0095] S308. If the total amount of data meets the preset total amount judgment condition, then determine the available space capacity and available space ratio of the disk under the current disk write cycle.

[0096] S309. If the available space capacity meets the preset capacity judgment condition and the available space ratio meets the preset ratio judgment condition, then the database is determined to continue generating cached data.

[0097] S310. If the available space capacity does not meet the preset capacity judgment condition, or the available space percentage does not meet the preset percentage judgment condition, then the generation of cached data in the database will be terminated.

[0098] Example 4

[0099] Figure 4 This is a schematic diagram of a cache data monitoring and management device provided in Embodiment 4 of the present invention. The cache data monitoring and management device provided in this embodiment of the present invention is applicable to situations where cache data generated during the execution of database statements is monitored and managed. This cache data monitoring and management device can be implemented in hardware and / or software, such as… Figure 4 As shown, the device specifically includes: a cache data writing module 401, a disk writing module 402, a data accumulation determination module 403, a judgment result determination module 404, and a monitoring and management module 405. Among them,

[0100] The cache data writing module 401 is used to write cache data generated during the execution of database statements into a preset cache space; the number of preset cache spaces is at least one.

[0101] The disk writing module 402 is used to write the cached data in its own cache space to the disk when the cached data write amount in each of the preset cache spaces meets the disk writing conditions.

[0102] The data accumulation determination module 403 is used to determine the data accumulation of cached data written to the disk in each of the preset cache spaces during the current disk write cycle;

[0103] The judgment result determination module 404 is used to determine whether to terminate the generation of cached data in the database based on the data accumulation corresponding to each preset cache space, and obtain a termination judgment result;

[0104] The monitoring and management module 405 is used to control the database to continue generating or terminate the generation of cached data based on the termination judgment result, so as to realize the monitoring and management of the cached data.

[0105] The technical solution of this invention writes cached data generated during the execution of database statements into a preset cache space; when the cached data write volume of each preset cache space meets the disk write condition, the cached data in its own cache space is written to the disk; the cumulative amount of cached data written to the disk in each preset cache space during the current disk write cycle is determined; based on the cumulative amount of data corresponding to each preset cache space, it is determined whether to terminate the generation of cached data by the database, and a termination judgment result is obtained; based on the termination judgment result, the database is controlled to continue generating or terminate the generation of cached data, so as to achieve monitoring and management of cached data. The above technical solution, by writing the cached data generated by the database to the disk in a temporary manner through preset cache spaces, and determining whether to control the continued generation of cached data based on the cumulative amount of data, realizes automated monitoring and management of cached data, and solves the security problem of business production process caused by excessively large temporary files.

[0106] Optionally, the judgment result determination module 404 includes:

[0107] The total number of writes is determined by the total number of writes of cached data to the disk.

[0108] The write total amount determination unit is used to determine the total amount of cached data written to the disk based on the data accumulation amount corresponding to each preset cache space if the data accumulation amount of any preset cache space meets the preset accumulation amount judgment condition, or the total number of writes meets the preset total number of writes judgment condition.

[0109] The judgment result determination unit is used to determine whether to terminate the generation of cached data in the database based on the total amount of data, and obtain a termination judgment result.

[0110] Optionally, the determination result unit includes:

[0111] The data termination generation subunit is used to determine to terminate the generation of cached data in the database if the total amount of data does not meet the preset total amount judgment condition.

[0112] Accordingly, the monitoring and management module 405 includes:

[0113] The termination instruction generation unit is used to generate a statement termination instruction and send it to the database if the termination judgment result is to determine to terminate the generation of data in the database, thereby controlling the database to terminate the generation of cached data according to the statement termination instruction, so as to realize the monitoring and management of the cached data.

[0114] Optionally, the determination result unit includes:

[0115] The cache data generation subunit is used to determine that the database continues to generate cache data if the total amount of data meets a preset total amount judgment condition.

[0116] Accordingly, the monitoring and management module 405 includes:

[0117] The cached data generation unit is used to control the database to continue generating cached data if the termination judgment result is that the database continues to generate data, so as to realize the monitoring and management of the cached data.

[0118] Optionally, the judgment result determination unit further includes:

[0119] The space capacity determination subunit is used to determine the available space capacity and available space percentage of the disk under the current disk write cycle after the total amount of data does not meet the preset total amount judgment condition and the total number of writes does not meet the preset total number of writes judgment condition.

[0120] The first capacity condition judgment subunit is used to determine that the database continues to generate cached data if the available space capacity meets a preset capacity judgment condition and the available space percentage meets a preset percentage judgment condition.

[0121] Optionally, the judgment result determination unit further includes:

[0122] The second capacity condition judgment subunit is used to terminate the generation of cached data in the database if the available space capacity does not meet the preset capacity judgment condition or the available space percentage does not meet the preset percentage judgment condition.

[0123] The cache data monitoring and management device provided in this embodiment of the invention can execute the cache data monitoring and management method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0124] Example 5

[0125] Figure 5 A schematic diagram of an electronic device 50 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0126] like Figure 5 As shown, the electronic device 50 includes at least one processor 51 and a memory, such as a read-only memory (ROM) 52 and a random access memory (RAM) 53, communicatively connected to the at least one processor 51. The memory stores computer programs executable by the at least one processor. The processor 51 can perform various appropriate actions and processes based on the computer program stored in the ROM 52 or loaded into the RAM 53 from storage unit 58. The RAM 53 can also store various programs and data required for the operation of the electronic device 50. The processor 51, ROM 52, and RAM 53 are interconnected via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0127] Multiple components in electronic device 50 are connected to I / O interface 55, including: input unit 56, such as keyboard, mouse, etc.; output unit 57, such as various types of monitors, speakers, etc.; storage unit 58, such as disk, optical disk, etc.; and communication unit 59, such as network card, modem, wireless transceiver, etc. Communication unit 59 allows electronic device 50 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0128] Processor 51 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 51 performs the various methods and processes described above, such as methods for monitoring and managing cached data.

[0129] In some embodiments, the method for monitoring and managing cached data may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 50 via ROM 52 and / or communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the method for monitoring and managing cached data described above may be performed. Alternatively, in other embodiments, processor 51 may be configured to perform the method for monitoring and managing cached data by any other suitable means (e.g., by means of firmware).

[0130] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0131] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0132] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0133] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0134] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0135] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0136] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0137] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for monitoring and managing cached data, characterized in that, include: The polling method is used to sequentially write the cached data generated during the execution of database statements into the preset cache space; The preset cache space is a high-speed cache and there is at least one such cache. When the amount of cached data written in each of the preset cache spaces meets the disk write conditions, the cached data in its own cache space is aligned and written to the disk. Determine the cumulative amount of cached data written to the disk in each of the preset cache spaces during the current disk write cycle; Determine the total number of writes of cached data to disk; If the data accumulation in any preset cache space meets the preset accumulation judgment condition, or the total number of writes meets the preset total number of write judgment condition, then the total amount of cached data written to the disk is determined according to the data accumulation corresponding to each preset cache space. Based on the total amount of data, determine whether to terminate the generation of cached data in the database, and obtain a termination judgment result; Based on the termination judgment result, the database is controlled to continue generating or terminate the generation of cached data, so as to realize the monitoring and management of the cached data.

2. The method according to claim 1, characterized in that, The step of determining whether to terminate the generation of cached data in the database based on the total amount of data, and obtaining a termination judgment result, includes: If the total amount of data does not meet the preset total amount judgment condition, then it is determined to terminate the generation of cached data in the database; Accordingly, based on the termination judgment result, controlling the database to continue generating or terminate the generation of cached data, in order to achieve monitoring and management of the cached data, includes: If the termination determination result is to terminate the generation of cached data in the database, a termination command is sent to the database to control the database to terminate the generation of cached data according to the termination command, so as to realize the monitoring and management of the cached data.

3. The method according to claim 1, characterized in that, The step of determining whether to terminate the generation of cached data in the database based on the total amount of data, and obtaining a termination judgment result, includes: If the total amount of data meets the preset total amount judgment condition, then it is determined that the database will continue to generate cached data; Accordingly, based on the termination judgment result, controlling the database to continue generating or terminate the generation of cached data, in order to achieve monitoring and management of the cached data, includes: If the termination judgment result is that the database continues to generate cached data, then the database is controlled to continue generating cached data in order to achieve monitoring and management of the cached data.

4. The method according to claim 3, characterized in that, After the total amount of data meets the preset total amount judgment condition, the method further includes: Determine the available space capacity and percentage of available space of the disk under the current disk write cycle; If the available space capacity meets the preset capacity judgment condition, and the available space percentage meets the preset percentage judgment condition, then it is determined that the database will continue to generate cached data.

5. The method according to claim 4, characterized in that, The method further includes: If the available space capacity does not meet the preset capacity judgment condition, or the available space percentage does not meet the preset percentage judgment condition, then the generation of cached data in the database will be terminated.

6. A monitoring and management device for cached data, characterized in that, include: The cache data writing module is used to sequentially write the cache data generated during the execution of database statements into a preset cache space using a polling method; The preset cache space is a high-speed cache and there is at least one such cache. The disk writing module is used to write the cached data in its own cache space to the disk when the amount of cached data written in each of the preset cache spaces meets the disk writing conditions. The data accumulation determination module is used to determine the data accumulation of cached data written to the disk in each of the preset cache spaces during the current disk write cycle; The judgment result determination module includes: a total number of times determination unit, a total amount of writing determination unit, and a judgment result determination unit; The total number of writes unit is used to determine the total number of writes of cached data to the disk; The write total determination unit is used to determine the total amount of cached data written to the disk based on the data accumulation amount corresponding to each preset cache space if the data accumulation amount of any preset cache space meets the preset accumulation amount judgment condition, or the total number of writes meets the preset total number of writes judgment condition. The judgment result determination unit is used to determine whether to terminate the generation of cached data in the database based on the total amount of data, and obtain a termination judgment result; The monitoring and management module is used to control the database to continue generating or terminate the generation of cached data based on the termination judgment result, so as to realize the monitoring and management of the cached data.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the monitoring and management method for cached data according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the method for monitoring and managing cached data as described in any one of claims 1-5.

Images