Database cache management method, database, equipment and storage medium

Abstract

The embodiment of the invention provides a database cache management method, a database, equipment and a storage medium, and the method comprises the steps: in a current brushing period, obtaining a first data block and a second data block in a buffer data space; obtaining a first data quantity of the first data block and a second data quantity of the second data block, and determining a first dirty brushing parameter based on the first data quantity and the second data quantity; obtaining a historical dirt brushing number of a plurality of historical cycles, and determining a second dirt brushing parameter based on the historical dirt brushing number; acquiring a data serial number of each first data block, and determining a third dirty brushing parameter based on the data serial number; determining a target dirt brushing parameter according to the first dirt brushing parameter, the second dirt brushing parameter and the third dirt brushing parameter; determining a target data block according to the target dirt brushing quantity, and writing the target data block into a local data space; according to the embodiment of the invention, the memory space and the read-write performance can be optimized, and the file is prevented from being damaged.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of database management technology, and in particular to a database cache management method, database, device and storage medium. Background Art

[0002] The database uses the file system to store user data. In order to improve the read and write performance, the current mainstream databases all load the data blocks into the memory first. A data block in the memory is called a page. When the database performs a write operation, the modified data block is called a dirty page. The dirty page is considered to be written successfully only when it is persisted in the data file. Data will not be lost due to downtime. The process of persisting dirty pages is called flushing.

[0003] In related technologies, most databases adopt an incremental dirty flushing mode, which avoids the read and write peaks caused by dirty flushing and reduces the impact of background dirty flushing on other database services to a certain extent. However, the database usually determines the dirty flushing speed based on the number of dirty pages currently in memory. The read and write speeds with large fluctuations are not conducive to the stable operation of other database services. Summary of the invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The embodiments of the present application provide a database cache management method, database, device and storage medium, which can optimize memory space and read and write performance to avoid file damage.

[0006] To achieve the above-mentioned purpose, the first aspect of an embodiment of the present application proposes a database cache management method, including: in the current dirty flushing cycle, obtaining a first data block and a second data block in a buffer data space, wherein the first data block represents data that has not been written to the local data space, and the second data block represents data that has been written to the local data space; obtaining a first data quantity of the first data block and a second data quantity of the second data block, and determining a first dirty flushing parameter based on the first data quantity and the second data quantity; obtaining a historical dirty flushing quantity of multiple historical cycles, and determining a second dirty flushing parameter based on the historical dirty flushing quantity; obtaining a data sequence number of each of the first data blocks, and determining a third dirty flushing parameter based on the data sequence number; determining a target dirty flushing parameter based on the first dirty flushing parameter, the second dirty flushing parameter and the third dirty flushing parameter; determining a target data block based on the target dirty flushing quantity, and writing the target data block into the local data space.

[0007] In some embodiments, the obtaining of the first data quantity of the first data block and the second data quantity of the second data block, and determining a first dirty flushing parameter based on the first data quantity and the second data quantity, includes: obtaining the first data quantity of the first data block, and the second data quantity of the second data block, and determining a proportion of dirty space to be flushed of the buffer data space according to the first data quantity and the second data quantity; and determining a first dirty flushing parameter based on the proportion of dirty space to be flushed and the first data quantity.

[0008] In some embodiments, determining the first dirty brush parameter based on the proportion of data to be brushed and the first amount of data includes: obtaining the historical dirty brush speed of the previous dirty brush cycle, and determining the first dirty brush limit based on the historical dirty brush speed; when the first amount of data is greater than or equal to the first dirty brush limit, or the dirty brush ratio is greater than or equal to a preset second dirty brush limit, obtaining a third dirty brush limit, and determining the first dirty brush parameter according to the third dirty brush limit, wherein the third dirty brush limit represents the maximum number of data blocks to be brushed dirty by the database within a single dirty brush cycle; when the first amount of data is less than the first dirty brush limit, and the dirty brush ratio is less than the second dirty brush limit, determining the first dirty brush parameter according to the preset second dirty brush limit.

[0009] In some embodiments, obtaining the historical dirty brush counts of multiple historical cycles and determining the second dirty brush parameter based on all the historical dirty brush counts includes: obtaining the historical dirty brush counts of multiple historical cycles and determining the average dirty brush count based on all the historical dirty brush counts; determining the second dirty brush parameter based on the historical dirty brush counts.

[0010] In some embodiments, obtaining the data sequence number of each of the first data blocks and determining the third dirty flushing parameter based on the data sequence number includes: obtaining the data sequence number of each of the first data blocks, determining the first minimum sequence number and the first maximum sequence number in the first data blocks, and determining the sequence difference based on the first minimum sequence number and the first maximum sequence number; determining the third dirty flushing parameter according to the sequence difference.

[0011] In one embodiment, determining the third dirty brush parameter based on the sequence difference includes: when the sequence difference is greater than or equal to a preset difference threshold, obtaining a first difference factor, and determining the third dirty brush parameter based on the first difference factor and the historical dirty brush number; when the sequence difference is less than the difference threshold, obtaining a second difference factor, and determining the third dirty brush parameter based on the second difference factor and the historical dirty brush number, wherein the first difference factor is greater than the second difference factor.

[0012] In some embodiments, determining the target data block according to the target dirty flushing number and writing the target data block into the local data space includes: determining the target data block according to the target dirty flushing number, obtaining the second minimum sequence number and the second maximum sequence number in the target data block; based on the data series number, starting from the target data block corresponding to the second minimum sequence number, writing the target data blocks into the local data space in sequence; when the first data number is greater than or equal to a preset fourth dirty flushing limit, and when the sequence difference is greater than or equal to the difference threshold, continuously obtaining the dirty flushing running time of the current dirty flushing cycle; obtaining the cycle dirty flushing time, when the target data block corresponding to the second maximum sequence number is written into the local data space, and the dirty flushing running time is less than the cycle dirty flushing time, re-obtaining the first data block and the second data block in the current buffer data space to enter the next dirty flushing cycle.

[0013] To achieve the above-mentioned purpose, the second aspect of the present application proposes a database, which is used to execute the database cache management method described in the first aspect.

[0014] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the database cache management method described in the first aspect when executing the computer program.

[0015] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, it implements the database cache management method described in the first aspect above.

[0016] The embodiments of the present application include at least the following beneficial effects: in the current cycle, by distinguishing the first data block and the second data block and counting their numbers, it is possible to more accurately evaluate which data needs to be flushed first, thereby reducing unnecessary data writing operations and improving the flushing efficiency; the target flushing number represents the number of first data blocks written to the local data space in the current cycle, and the target flushing number is affected by the density of data in the database buffer data space and the database write-in and write-out capabilities. When calculating the target flushing number of the current cycle, first, the capacity of the buffer data space is constant, the space occupied by the second data block can be flushed when needed, and the first data block needs to be persisted By obtaining the first data quantity of the first data block and the second data quantity of the second data block, the congestion degree of the buffer data space at the beginning of the current cycle can be known. When the first data quantity is greater than a certain threshold, it means that the first data block occupies a large capacity in the buffer data space, and there are many files in the buffer data space that have not been written into the local data space. New files cannot be written, and the first data block in the buffer data space is also prone to loss. Therefore, the first dirty flushing parameter is determined by the first data quantity and the second data quantity, so that the first data block can be written into the local data block as soon as possible, thereby improving the writing capacity of the buffer data space; then, when the number of target dirty flushing users fluctuates greatly in adjacent cycles, it may cause the database to The performance drops instantaneously. Therefore, by obtaining the historical dirty flushing numbers of multiple historical cycles, the historical dirty flushing numbers represent the target dirty flushing numbers of the corresponding cycles, and determining the second dirty flushing parameter according to the historical dirty flushing numbers, and determining the target dirty flushing number of the current cycle according to the second dirty flushing parameter, the target dirty flushing numbers of adjacent cycles can be smoothed, reducing the competition for simultaneous write operations of different threads to the local data space, improving the overall write efficiency, and reducing the fluctuation of the target dirty flushing number in adjacent cycles. More stable database performance can be provided; finally, in the buffered data space, the data sequence number of the first data block is determined based on the write time. The data sequence numbers of the first data block of the same file are continuous, but the continuous numbers The first data block with the data sequence number is discontinuous in the physical space of the buffer data space. When writing the first data block into the local data space, for the sake of read and write efficiency, the first data blocks adjacent to the physical space are usually read and written in one cycle. Therefore, by obtaining the data sequence number of the first data block, when the data sequence number of the first data block differs too much, it indicates that there is a first data block with a longer storage time. The third dirty flushing parameter is determined based on the data sequence number, so that the first data block that needs the most persistence can be written into the local data space first according to the importance of the first data block data, thereby avoiding file damage, and optimizing the buffer data space and the local data space to improve the write-in and write-out efficiency of the database;In summary, the database cache management method proposed in this application determines the target dirty flushing number in the above manner, and determines the target data block according to the target dirty flushing number, which can optimize the memory space and read and write performance, and also can avoid file damage. ;

[0017] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0019] Figure 1 An optional flowchart of a database cache management method provided in an embodiment of the present application; Figure 2 A schematic diagram of an optional flow chart for calculating the first dirty brushing parameter provided in an embodiment of the present application; Figure 3 A schematic diagram of an optional specific process for calculating the first dirty flushing parameter provided in an embodiment of the present application; Figure 4 A schematic diagram of an optional flow chart for calculating the second dirty brushing parameter provided in an embodiment of the present application; Figure 5 A schematic diagram of an optional flow chart for calculating the third dirty brushing parameter provided in an embodiment of the present application; Figure 6 A schematic diagram of an optional specific process for calculating the third dirty flushing parameter provided in an embodiment of the present application; Figure 7 A schematic diagram of an optional process for entering the next dirty-cleaning cycle provided in an embodiment of the present application; Figure 8 An optional flow chart of determining a target dirty flushing quantity according to database capability provided in an embodiment of the present application; Fig. 9 A schematic diagram of an optional hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings.

[0021] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0022] In the description of this application, “several” means one or more, “more” means more than two, “greater than”, “less than”, “exceed”, etc. are understood to exclude the number, and “above”, “below”, “within”, etc. are understood to include the number.

[0023] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0024] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here: Flushing Dirty Pages is an operation in database memory management, which refers to writing dirty pages in memory (data pages that have been modified but not yet written to disk) to disk to ensure data consistency. This process is part of database buffer pool management and is usually triggered by the database's checkpoint mechanism, or automatically performed when the number of dirty pages in memory reaches a certain threshold. Flushing dirty pages ensures that data changes in memory are synchronized to disk, but it does not guarantee the persistence of transactions itself.

[0025] Persistence means that data is stored on a persistent storage medium, such as a hard disk or solid-state drive, so that the data will not be lost even in the event of a system crash or power outage. In a database system, persistence is usually implemented by relying on transaction logs (such as redo logs). When a transaction is committed, its changes are recorded in the log, which can be used to recover data and ensure the "D" (Durability) of the ACID properties of the database service. Persistence is a broader concept that includes not only flushing operations, but also logging, transaction management, and other mechanisms to ensure that data is not lost.

[0026] In related technologies, most databases adopt an incremental dirty flushing mode, which avoids the read and write peaks caused by dirty flushing and reduces the impact of background dirty flushing on other database services to a certain extent. However, the database usually determines the dirty flushing speed based on the number of dirty pages currently in memory. The read and write speeds with large fluctuations are not conducive to the stable operation of other database services.

[0027] Based on this, the embodiments of the present application provide a database cache management method, database, device and storage medium, which can optimize memory space and read and write performance to avoid file damage.

[0028] The database cache management method, database, device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the database cache management method in the embodiments of the present application is described.

[0029] The database cache management method provided in the embodiment of the present application relates to the field of computer technology. The database cache management method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the database cache management method, etc., but is not limited to the above forms.

[0030] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0031] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0032] like Figure 1 As shown, Figure 1 An optional flow chart of a database cache management method provided in an embodiment of the present application, the database cache management method can be executed by a server, or can also be executed by a terminal, or can also be executed by a server in cooperation with a terminal, the database cache management method includes but is not limited to the following steps S110 to S160: Step S110, in the current dirty flushing cycle, obtaining a first data block and a second data block in the buffer data space, wherein the first data block represents data that has not been written into the local data space, and the second data block represents data that has been written into the local data space; Step S120, obtaining a first data quantity of the first data block and a second data quantity of the second data block, and determining a first dirty flushing parameter based on the first data quantity and the second data quantity; Step S130, obtaining the historical dirty flushing numbers of multiple historical periods, and determining a second dirty flushing parameter based on the historical dirty flushing numbers; Step S140, obtaining a data sequence number of each first data block, and determining a third dirty flushing parameter based on the data sequence number; Step S150, determining a target dirty brushing parameter according to the first dirty brushing parameter, the second dirty brushing parameter and the third dirty brushing parameter; Step S160, determining a target data block according to a target dirty flushing quantity, and writing the target data block into a local data space.

[0033] It can be understood that, in the current cycle, by distinguishing the first data block and the second data block and counting their numbers, it is possible to more accurately evaluate which data needs to be flushed first, thereby reducing unnecessary data writing operations and improving the flushing efficiency; the target flushing number represents the number of first data blocks written to the local data space in the current cycle. The target flushing number is affected by the density of data in the database buffer data space and the database write-in and write-out capabilities. When calculating the target flushing number of the current cycle, first, the capacity of the buffer data space is constant, the space occupied by the second data block can be flushed when needed, and the first data block needs to be persisted. By obtaining The first data quantity of the first data block and the second data quantity of the second data block can be used to know the congestion level of the buffer data space at the beginning of the current cycle. When the first data quantity is greater than a certain threshold, it means that the first data block occupies a large capacity in the buffer data space, and there are many files in the buffer data space that have not been written into the local data space. New files cannot be written, and the first data block in the buffer data space is also prone to loss. Therefore, the first dirty flushing parameter is determined by the first data quantity and the second data quantity, so that the first data block can be written into the local data block as soon as possible, thereby improving the writing capacity of the buffer data space. Then, when the number of target dirty flushing users fluctuates greatly in adjacent cycles, it may cause instantaneous database performance. Therefore, by obtaining the historical dirty flushing numbers of multiple historical cycles, the historical dirty flushing numbers represent the target dirty flushing numbers of the corresponding cycles, determining the second dirty flushing parameter according to the historical dirty flushing numbers, and determining the target dirty flushing number of the current cycle according to the second dirty flushing parameter, the target dirty flushing numbers of adjacent cycles can be smoothed, reducing the competition for simultaneous write operations of different threads to the local data space, improving the overall write efficiency, and reducing the fluctuation of the target dirty flushing number in adjacent cycles. More stable database performance can be provided; finally, in the buffer data space, the data sequence number of the first data block is determined based on the write time. The data sequence numbers of the first data blocks of the same file are continuous, but the continuous data sequence numbers are The first data block with the first data block number is discontinuous in the physical space of the buffer data space. When writing the first data block into the local data space, for the sake of read and write efficiency, the first data blocks adjacent to the physical space are usually read and written in one cycle. Therefore, by obtaining the data sequence number of the first data block, when the data sequence number of the first data block differs too much, it indicates that there is a first data block with a longer storage time. The third dirty flushing parameter is determined based on the data sequence number, so that the first data block that needs the most persistence can be written into the local data space first according to the importance of the first data block data, thereby avoiding file damage, and optimizing the buffer data space and the local data space to improve the write-in and write-out efficiency of the database;In summary, the database cache management method proposed in this application determines the target dirty flushing number in the above manner, and determines the target data block according to the target dirty flushing number, which can optimize the memory space and read and write performance, and also can avoid file damage. ;

[0034] It should be noted that the first data block can be understood as a dirty page, the second data block can be understood as a clean page, the buffer data space can be understood as the memory in the database, the local data space can be understood as the local hard disk in the database, and the operation of writing the first data block from the buffer data space to the local data space can be understood as flushing or persistence. In the buffer data space, the file is divided into multiple data blocks of 8kb in size for storage, that is, the size of the first data block and the second data block are both 8kb.

[0035] In a specific implementation, the target brush dirty number is equal to an average of the first brush dirty parameter, the second brush dirty parameter, and the third brush dirty parameter.

[0036] In a specific embodiment, referring to Figure 8 As shown, when the target dirty flushing number is greater than the maximum read-write capacity of the database, the target dirty flushing number is taken as the maximum read-write capacity of the database; when the target dirty flushing number is less than the single minimum read-write number of the database, the target dirty flushing number is taken as the single minimum read-write number.

[0037] In addition, refer to Figure 2 As shown, some embodiments of the present application, Figure 1 Step S120 in the embodiment includes but is not limited to the following steps S210 to S220: Step S210, obtaining a first data quantity of a first data block and a second data quantity of a second data block, and determining a dirty ratio to be flushed of a buffer data space according to the first data quantity and the second data quantity; Step S220, determining a first dirty flushing parameter based on the proportion of dirty data to be flushed and the first data quantity.

[0038] It can be understood that by calculating the proportion of dirty blocks to be flushed, it is possible to avoid excessively flushing data blocks that have not been modified a lot, thereby reducing unnecessary read and write operations and improving the overall efficiency of the system. Under different workloads, the calculation of the proportion of dirty blocks to be flushed and the determination of the dirty flushing parameters can adapt to different data access patterns, so that the database system can maintain good performance when facing different workloads.

[0039] In addition, refer to Figure 3 As shown, some embodiments of the present application, Figure 2Step S220 in the embodiment includes but is not limited to the following steps S310 to S330: Step S310, obtaining a historical dirty-cleaning speed of a previous dirty-cleaning cycle, and determining a first dirty-cleaning limit value based on the historical dirty-cleaning speed; Step S320: when the first data quantity is greater than or equal to the first dirty flushing limit, or the dirty flushing ratio is greater than or equal to the preset second dirty flushing limit, obtaining a third dirty flushing limit, and determining a first dirty flushing parameter according to the third dirty flushing limit, wherein the third dirty flushing limit represents the maximum number of data blocks that the database performs dirty flushing in a single dirty flushing cycle; Step S330: when the first data quantity is less than the first dirty brush limit value, and the dirty brush ratio is less than the second dirty brush limit value, determine the first dirty brush parameter according to the preset second dirty brush limit value.

[0040] In a specific implementation, the first dirty flush limit is equal to the target dirty flush number corresponding to the previous dirty flush cycle, the dirty flush ratio is equal to the ratio of the first data number to the sum of the first data number and the second data number, the second dirty flush limit is equal to 75%, and the third dirty flush limit represents the maximum read and write capacity of the database.

[0041] The value of the first dirty brush parameter is a multi-level step function. When the first data quantity is greater than or equal to the first dirty brush limit, or the dirty brush ratio is greater than or equal to the preset second dirty brush limit, the first dirty brush parameter is 75% of the third dirty brush limit; when the first data quantity is less than the first dirty brush limit, and the dirty brush ratio is less than the second dirty brush limit, the first dirty brush parameter is a preset low value.

[0042] In a specific implementation, when the first data quantity is less than the first dirty brush limit and the dirty brush ratio is less than the second dirty brush limit, the first dirty brush parameter is equal to the target dirty brush quantity corresponding to the previous dirty brush cycle.

[0043] In addition, refer to Figure 4 As shown, some embodiments of the present application, Figure 1 Step S130 in the embodiment includes but is not limited to the following steps S410 to S420: Step S410, obtaining the historical dirty flushing numbers of multiple historical periods, and determining the average dirty flushing number based on all the historical dirty flushing numbers; Step S420, determining a second dirty flushing parameter based on the historical dirty flushing quantity.

[0044] It is understandable that when the target number of dirty flushing users fluctuates greatly in adjacent cycles, it may cause an instantaneous drop in database performance. Therefore, by obtaining the historical dirty flushing numbers of multiple historical cycles, the historical dirty flushing numbers represent the target dirty flushing numbers of the corresponding cycles, and determining the second dirty flushing parameter based on the historical dirty flushing numbers, and determining the target dirty flushing number of the current cycle based on the second dirty flushing parameter, the target dirty flushing numbers of adjacent cycles can be smoothed, reducing the competition for simultaneous write operations on the local data space by different threads, improving the overall write efficiency, and providing more stable database performance by reducing the fluctuation of the target dirty flushing number in adjacent cycles.

[0045] In a specific implementation, the second dirty brush parameter is equal to an average value of the historical dirty brush quantities of 30 historical cycles.

[0046] In addition, refer to Figure 5 As shown, some embodiments of the present application, Figure 1 Step S140 in the embodiment includes but is not limited to the following steps S510 to S520: Step S510, obtaining a data sequence number of each first data block, determining a first minimum sequence number and a first maximum sequence number in the first data block, and determining a sequence difference based on the first minimum sequence number and the first maximum sequence number; Step S520: determining a third dirty flushing parameter according to the sequence difference.

[0047] It can be understood that in the buffer data space, that is, in the memory of the database, the data sequence number of the first data block is determined based on the write time, and the data sequence numbers of the first data blocks of the same file are continuous, but the first data blocks with continuous data sequence numbers are discontinuous in the physical space of the buffer data space. When the first data block is written to the local data space, for the sake of read and write efficiency, within one cycle, the first data blocks adjacent to each other in the physical space are usually read and written. Therefore, by obtaining the data sequence number of the first data block, when the data sequence numbers of the first data blocks differ too much, it means that there is a first data block with a longer storage time. The third dirty flushing parameter is determined based on the data sequence number, so that the first data block that needs to be persisted the most can be written to the local data space in priority according to the importance of the first data block data, thereby avoiding file damage, and optimizing the buffer data space and the local data space to improve the write and write efficiency of the database.

[0048] In addition, refer to Figure 6 As shown, some embodiments of the present application, Figure 5 Step S520 in the embodiment includes but is not limited to the following steps S610 to S620: Step S610, when the sequence difference is greater than or equal to a preset difference threshold, a first difference factor is obtained, and a third dirty flushing parameter is determined according to the first difference factor and the number of historical dirty flushing; Step S620: when the sequence difference is less than the difference threshold, obtain a second difference factor, and determine a third dirty flushing parameter according to the second difference factor and the number of historical dirty flushing, wherein the first difference factor is greater than the second difference factor.

[0049] In a specific embodiment, the first minimum sequence number represents the first data block written to the buffer data space earliest, and the second maximum sequence number represents the first data block written to the buffer data space most recently. When the sequence difference is greater than or equal to a preset difference threshold, the third dirty flush parameter is equal to the product of the second dirty flush parameter and the first difference factor. When the sequence difference is less than the difference threshold, the third difference factor is equal to the product of the second dirty flush parameter and the second difference factor.

[0050] In a specific implementation, when the sequence difference is greater than or equal to a preset difference threshold, the first difference factor is equal to 3, and when the sequence difference is less than the difference threshold, the second difference factor is equal to 1.

[0051] It can be understood that when the sequence difference is large, it means that there are data blocks in the buffer data space that were written earlier but have not yet been written to the local data space for persistence. In the related technology, when writing new data blocks to the buffer data space, the database prioritizes clearing data blocks with smaller data sequence numbers, that is, data blocks written earlier. Therefore, when the sequence difference is large, a larger first difference factor is given, and the third dirty flushing parameter is increased to increase the target dirty flushing number, which is conducive to timely writing the first data block with a longer storage time into the local data space to avoid file data damage; at the same time, the third dirty flushing parameter is determined based on the first difference factor or the second difference factor and the average of multiple historical dirty flushing numbers, and the target dirty flushing numbers of adjacent cycles can also be smoothed, thereby reducing the competition for different threads to perform write operations on the local data space at the same time, improving the overall write efficiency, and reducing the fluctuation of the target dirty flushing number in adjacent cycles. More stable database performance can be provided.

[0052] In addition, refer to Figure 7 As shown, some embodiments of the present application, Figure 1 Step S150 in the embodiment includes but is not limited to the following steps S710 to S740: Step S710, determining a target data block according to a target dirty flushing quantity, and obtaining a second minimum sequence number and a second maximum sequence number in the target data block; Step S720, based on the data sequence number, starting from the target data block corresponding to the second smallest sequence number, the target data blocks are sequentially written into the local data space; Step S730, when the first data quantity is greater than or equal to a preset fourth dirty flushing limit value, and when the sequence difference is greater than or equal to the difference threshold value, continuously obtaining the dirty flushing running time of the current dirty flushing cycle; Step S740, obtain the periodic dirty flushing time. When the target data block corresponding to the second maximum sequence number is written into the local data space and the dirty flushing operation time is less than the periodic dirty flushing time, re-acquire the first data block and the second data block in the current buffer data space to enter the next dirty flushing cycle.

[0053] Specifically, after determining the target number of dirty flushes, based on the database persistence strategy in the relevant technology, the database determines the target number of dirty flushes and the target data blocks that need to be written to the local data space in the current cycle and their reading and writing order. The second smallest sequence number represents the target data block to be read and written first, and the second largest sequence number represents the second data block to be read and written last.

[0054] In the related art, when the database implements a persistence strategy, the target duration of each flushing cycle is usually fixed. When the first data quantity is less than the preset fourth flushing limit, or when the sequence difference is less than the difference threshold, the database flushes according to the preset target time. Even when the target data block corresponding to the second maximum sequence number is written to the local data space and the flushing operation time is less than the periodic flushing time, the database still waits for the flushing time to be equal to the target time before performing the next flushing cycle.

[0055] When the target data block corresponding to the second largest sequence number is written into the local data space and the dirty flushing operation time is less than the periodic dirty flushing time, when the database completes reading and writing the target data block corresponding to the second largest sequence number, the database automatically completes and stops the current dirty flushing cycle, and re-acquires the target data block and the second data block to enter the next dirty flushing cycle. That is, when the target data block corresponding to the second largest sequence number is persisted, regardless of whether the dirty flushing time is equal to the target time specified by the persistence policy, the database exits the current dirty flushing cycle and enters the next dirty flushing cycle.

[0056] In a specific implementation, the fourth dirty brush limit is equal to the first dirty brush limit, that is, the fourth dirty brush limit is equal to the target dirty brush number corresponding to the previous dirty brush cycle.

[0057] It can be understood that when the first data quantity is greater than or equal to the preset fourth dirty flushing limit, and when the sequence difference is greater than or equal to the difference threshold, it can directly indicate that there are more target data blocks in the buffer data space, and indirectly indicate that the target dirty flushing quantity is large and the persistence task is heavy. At this time, when the dirty flushing task in the current dirty flushing cycle is completed, the current dirty flushing cycle is completed and stopped in time and the next dirty flushing cycle is quickly entered, which can reduce the time that the database is idle, effectively improve the reading and writing efficiency so that the target data blocks can be written into the local data space in time to ensure the integrity of the file.

[0058] In addition, the present application also proposes a database, which is used to execute the database cache management method in the above embodiment.

[0059] It can be understood that the specific implementation of the database is basically the same as the specific implementation of the above-mentioned database cache management method, and will not be repeated here.

[0060] In addition, refer to Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes: The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application; The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the database cache management method of the embodiment of the present application, for example, executing the above-described Figure 1 Steps S110 to S160 of the method, Figure 2 Steps S210 to S220 of the method, Figure 3 Steps S310 to S330 of the method, Figure 4 Steps S410 to S420 of the method, Figure 5 Steps S510 to S520 of the method, Figure 6 Steps S610 to S620 of the method, Figure 7Steps S710 to S740 of the method; Input / output interface 903, used to implement information input and output; Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.); A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904); The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0061] The embodiment of the present application also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned database cache management method, for example, to execute the above-described Figure 1 Steps S110 to S160 of the method, Figure 2 Steps S210 to S220 of the method, Figure 3 Steps S310 to S330 of the method, Figure 4 Steps S410 to S420 of the method, Figure 5 Steps S510 to S520 of the method, Figure 6 Steps S610 to S620 of the method, Figure 7 The method comprises steps S710 to S740.

[0062] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0063] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0064] It can be understood by those skilled in the art that Figures 1 to 8 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0065] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0066] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0067] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0068] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0069] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0070] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0071] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0072] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0073] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A database cache management method, characterized in that: include: In the current dirty flushing cycle, a first data block and a second data block in the buffer data space are obtained, wherein the first data block represents data that has not been written into the local data space, and the second data block represents data that has been written into the local data space; Acquire a first data quantity of the first data block and a second data quantity of the second data block, and determine a first dirty flushing parameter based on the first data quantity and the second data quantity; Obtaining historical dirty flushing numbers of multiple historical periods, and determining a second dirty flushing parameter based on the historical dirty flushing numbers; Obtaining a data sequence number of each of the first data blocks, and determining a third dirty flushing parameter based on the data sequence number; Determine a target dirty brushing parameter according to the first dirty brushing parameter, the second dirty brushing parameter and the third dirty brushing parameter; A target data block is determined according to the target dirty flushing quantity, and the target data block is written into the local data space.

2. The database cache management method according to claim 1, characterized in that: The obtaining a first data quantity of the first data block and a second data quantity of the second data block, and determining a first dirty flushing parameter based on the first data quantity and the second data quantity, includes: Acquire a first data quantity of the first data block and a second data quantity of the second data block, and determine a dirty ratio to be flushed of the buffer data space according to the first data quantity and the second data quantity; A first dirty flushing parameter is determined based on the proportion of dirty items to be flushed and the first data quantity.

3. The database cache management method according to claim 2, characterized in that: The determining a first dirty flushing parameter based on the dirty flushing ratio and the first data quantity includes: Acquire a historical dirty-cleaning speed of a previous dirty-cleaning cycle, and determine a first dirty-cleaning limit value based on the historical dirty-cleaning speed; When the first data quantity is greater than or equal to the first dirty flushing limit, or the dirty flushing ratio is greater than or equal to a preset second dirty flushing limit, obtaining a third dirty flushing limit, and determining a first dirty flushing parameter according to the third dirty flushing limit, wherein the third dirty flushing limit represents the maximum number of data blocks that the database performs dirty flushing in a single dirty flushing cycle; When the first data quantity is less than the first dirty brush limit, and the dirty brush ratio is less than the second dirty brush limit, a first dirty brush parameter is determined according to a preset second dirty brush limit.

4. The database cache management method according to claim 1, characterized in that: The obtaining of historical dirty flushing numbers of multiple historical periods and determining a second dirty flushing parameter based on all the historical dirty flushing numbers includes: Obtain historical dirty flushing numbers of multiple historical periods, and determine an average dirty flushing number based on all the historical dirty flushing numbers; A second dirty brushing parameter is determined based on the historical dirty brushing quantity.

5. The database cache management method according to claim 4, characterized in that: The obtaining of the data sequence number of each of the first data blocks and determining the third dirty flushing parameter based on the data sequence number includes: Obtaining a data sequence number of each of the first data blocks, determining a first minimum sequence number and a first maximum sequence number in the first data blocks, and determining a sequence difference based on the first minimum sequence number and the first maximum sequence number; A third dirty flushing parameter is determined according to the sequence difference.

6. The database cache management method according to claim 5, characterized in that: The determining a third dirty flushing parameter according to the sequence difference includes: When the sequence difference is greater than or equal to a preset difference threshold, a first difference factor is obtained, and a third dirty flushing parameter is determined according to the first difference factor and the historical dirty flushing quantity; When the sequence difference is less than the difference threshold, a second difference factor is obtained, and the third dirty flushing parameter is determined according to the second difference factor and the historical dirty flushing quantity, wherein the first difference factor is greater than the second difference factor.

7. The database cache management method according to claim 6, characterized in that: The step of determining a target data block according to the target dirty flushing quantity and writing the target data block into the local data space includes: Determine a target data block according to the target dirty flushing quantity, and obtain a second minimum sequence number and a second maximum sequence number in the target data block; Based on the data serial number, starting from the target data block corresponding to the second smallest serial number, the target data blocks are sequentially written into the local data space; When the first data quantity is greater than or equal to a preset fourth dirty flushing limit value, and when the sequence difference value is greater than or equal to the difference threshold value, continuously acquiring the dirty flushing running time of the current dirty flushing cycle; Obtain the periodic dirty flushing time. When the target data block corresponding to the second maximum sequence number is written into the local data space and the dirty flushing running time is less than the periodic dirty flushing time, re-acquire the first data block and the second data block in the current buffer data space to enter the next dirty flushing cycle.

8. A database, characterized in that: The database is used to execute the database cache management method described in any one of claims 1 to 7.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the database cache management method according to any one of claims 1 to 7 when executing the computer program.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the database cache management method according to any one of claims 1 to 7 is implemented.

Images