Method and System for Coordinating and Optimizing Multi-Embedded Database Buffer Sizes

Abstract

Method and system for coordinated optimization of multi-embedded database buffer sizes, relating to the field of database technologies. In view of the problem that when optimizing embedded databases in the prior art, optimization is often carried out one by one, and this process is repeatedly executed multiple times in the same system, which not only wastes time but also causes a significant increase in the cache of each database, resulting in the inability of other programs to run due to insufficient memory when other programs are running. In response to the lightweight characteristics of embedded databases, this application performs unified optimization on all embedded databases instead of optimizing each database separately, improving the optimization efficiency and avoiding the time waste caused by individual optimization. Moreover, this application helps the system tide over the peak period of memory usage by shrinking sub-functions and restoring the buffer to the default size in a sorted manner, thereby solving the problem that the significant increase in the cache of the database causes other programs to be unable to run due to insufficient memory when other programs are running.

Description

Technical Field

[0001] The present invention relates to the technical field of databases, and specifically to a method and system for coordinating and optimizing the buffer size of multiple embedded databases. Background Art

[0002] Optimization algorithms for the operation of databases have been widely studied since the birth of databases. However, the research on optimization algorithms for embedded databases is still not sufficient. Due to many limitations existing in embedded databases themselves, the main ways to optimize them are to optimize queries and read / writes, such as cardinality estimation optimization, query plan tree optimization, etc. These techniques are carried over from relational databases, assuming an embedded database as a small relational database. On the other hand, some research aims to integrate machine learning techniques into embedded databases to optimize the databases, such as using deep learning algorithms to automatically adjust various configurations of embedded databases to reach the optimal configuration state. The characteristics of embedded databases increase the difficulty of optimizing them.

[0003] Existing embedded databases such as SQLite, H2, etc. often rarely have optimizations for their performance. This is determined by various characteristics of embedded databases. Being lightweight, different from server-side databases such as MySQL, the purpose of an embedded database is not its performance level, but to enable programs, system services, etc. to store data conveniently. It is embedded in a program and plays a role weaker than a conventional database. This requires the size of the embedded database to be light enough and not redundant. However, this also limits the optimization of it. On the other hand, in order to achieve its lightweight characteristics, an embedded database removes some functions, such as its concurrency performance is usually very poor and it does not support multiple user operations, etc.

[0004] As shown above, an embedded database is embedded in a program. Multiple programs or system services may be running on the same device, and each program or service has its own database, which means that multiple embedded databases are running simultaneously on the same device. On the other hand, an embedded database cannot occupy all the computing resources of the entire device. It must cooperate with other programs to run, which is different from server-side databases that can monopolize all the resources of the entire device. Therefore, when optimizing an embedded database, it is also impossible to assume the use of all the resources of the device. Only the optimization can be carried out as much as possible within limited resources.

[0005] When existing methods optimize an embedded database, they often optimize them one by one. This process is repeatedly executed multiple times in the same system, which not only wastes time but also causes the cache of each database to increase significantly. When other programs run, due to insufficient memory, other programs cannot run. Summary of the Invention

[0006] The object of the present invention is to propose a method and system for coordinated optimization of the buffer sizes of multiple embedded databases to address the problem in the prior art that when optimizing an embedded database, optimization is often carried out one by one, and this process is repeatedly executed multiple times in the same system, which not only wastes time but also causes a significant increase in the cache of each database. As a result, when other programs run, due to insufficient memory, other programs cannot run.

[0007] The technical solution adopted by the present invention to solve the above technical problems is as follows:

[0008] A method for coordinated optimization of the buffer sizes of multiple embedded databases includes the following steps:

[0009] Step 1: Connect to the embedded database and listen for the transaction status signals sent by the embedded database in the connected embedded database. The transaction status includes active embedded databases and inactive embedded databases. The signal of an active embedded database refers to an embedded database where a read or write transaction is in progress or a read or write transaction has not been completed. The signal of an inactive embedded database refers to an embedded database where no transaction is currently in progress.

[0010] Step 2: After collecting the transaction status signals sent by all the embedded databases, according to the transaction status, input the active embedded databases into the increasing sub-function and the inactive embedded databases into the decreasing sub-function, and execute Step 5. The increasing sub-function and the decreasing sub-function are respectively used for increasing and decreasing the buffer of the embedded database.

[0011] In the increasing sub-function, first determine whether the current memory usage rate is greater than or equal to 60%. If so, stop increasing the buffer of the active embedded database, input the active embedded database into the decreasing sub-function, and execute Step 5. Otherwise, continue the operation of increasing the buffer of the active embedded database.

[0012] If the increasing operation continues, first calculate the difference between the current memory usage rate and 60%, and based on the actual memory size and in combination with the difference, obtain the maximum memory size allowed for increasing the buffer. Within the allowable range, for each embedded database input into the increasing sub-function according to the order, perform the following steps:

[0013] Gradually increase the size of the memory in the embedded database buffer and record the throughput rate during the operation of the embedded database. If the throughput rate increases monotonically with the size of the memory in the buffer, then the buffer size at this time is the locally optimal record. Then continue to increase the size of the memory in the buffer and update the locally optimal record. If the throughput rate does not increase monotonically with the size of the memory in the buffer, then the buffer size at this time is the globally optimal record;

[0014] If there is a globally optimal record, select the memory size of the buffer corresponding to the globally optimal record as the final buffer size. If there is no globally optimal record, select the memory size of the buffer corresponding to the latest locally optimal record as the final buffer size;

[0015] Step 3: After adding the final buffer sizes corresponding to all the embedded databases to the current memory usage value, obtain the memory utilization rate, and then determine whether the memory utilization rate exceeds 60%. If it does not exceed 60%, apply the final buffer sizes of all the embedded databases. If it exceeds 60%, sort the final buffer sizes of the embedded databases obtained in Step 2 based on the memory size. Then, according to the sorting, restore the final buffer sizes of the embedded databases in descending order until the memory utilization rate does not exceed 60%. At this time, apply the final buffer sizes of the embedded databases that have not been restored;

[0016] Step 4: Set an interval time, and at each time the interval time is reached, input all the embedded databases into the shrinking sub-function and execute Step 5;

[0017] Step 5: In the shrinking sub-function, sort the input embedded databases in descending order according to the buffer size, and then, according to the sorting in descending order, restore the buffers of the embedded databases to the default size in sequence. After each restoration, judge the memory utilization rate again. When the memory utilization rate returns below 60%, stop the restoration of the buffer. When the buffers of all the embedded databases have been restored to the default size and the memory utilization rate of the system still fails to return below 60%, end the shrinking sub-function.

[0018] Further, the specific steps of Step 1 are as follows:

[0019] Judge whether an embedded database is currently connected. If an embedded database is not currently connected, send a message to all the embedded databases for connection. If the embedded database agrees to the connection, connect to it and maintain the connection of the embedded database through a heartbeat mechanism in the follow-up;

[0020] If an embedded database is currently connected, resend a message to the unconnected embedded database for connection. If the embedded database agrees to the connection, connect to it and maintain the connection to the embedded database through a heartbeat mechanism subsequently.

[0021] Further, when the buffer size is the globally optimal record, continue the exploration in the ε-greedy mode.

[0022] Further, within the difference range, when gradually increasing the memory size, the memory size increased each time is 20MB.

[0023] Further, the interval time is 10 minutes.

[0024] A multi-embedded database buffer size coordination and optimization system includes an embedded database connection module, a buffer adjustment module, an increasing sub-function module, a buffer recovery module, and a decreasing sub-function module;

[0025] The embedded database connection module is used to connect to the embedded database and listen for the transaction status signals sent by the embedded database in the connected embedded databases. The transaction status includes active embedded databases and inactive embedded databases. The active embedded database signal refers to an embedded database where a read or write transaction is in progress or a read or write transaction has not been completed. The inactive embedded database signal refers to an embedded database where no transaction is currently in progress;

[0026] The buffer adjustment module is used to, after collecting the transaction status signals sent by all the embedded databases, input the active embedded databases into the increasing sub-function module and the inactive embedded databases into the decreasing sub-function module according to the transaction status. The increasing sub-function module and the decreasing sub-function module are respectively used for increasing and decreasing the buffer of the embedded database;

[0027] The increasing sub-function module is used to determine whether the current memory usage rate is greater than or equal to 60%. If so, stop increasing the buffer of the active embedded database and input the active embedded database into the decreasing sub-function module. Otherwise, continue the operation of increasing the buffer of the active embedded database;

[0028] If the increasing operation continues, first calculate the difference between the current memory usage rate and 60%, and combine the difference with the actual memory size to obtain the maximum allowable memory size for increasing the buffer. Within the allowable range, for each embedded database input into the increasing sub-function module according to the order of the embedded database, perform the following steps:

[0029] Gradually increase the size of the memory in the embedded database buffer, and record the throughput rate during the operation of the embedded database. If the throughput rate monotonically increases with the size of the memory in the buffer, then the buffer size at this time is the locally optimal record. Then continue to increase the size of the memory in the buffer and update the locally optimal record. If the throughput rate does not monotonically increase with the size of the memory in the buffer, then the buffer size at this time is the globally optimal record;

[0030] If there is a globally optimal record, select the memory size of the buffer corresponding to the globally optimal record as the final buffer size. If there is no globally optimal record, select the memory size of the buffer corresponding to the latest locally optimal record as the final buffer size;

[0031] The buffer recovery module is used to add the current memory usage value to the final buffer size corresponding to all embedded databases to obtain the memory utilization rate, and then determine whether the memory utilization rate exceeds 60%. If it does not exceed 60%, apply the final buffer sizes of all embedded databases. If it exceeds 60%, sort the final buffer sizes of the obtained embedded databases based on the memory size. Then, according to the sorting, recover the final buffer sizes of the embedded databases from large to small in sequence until the memory utilization rate does not exceed 60%. At this time, apply the final buffer sizes of the embedded databases that have not been recovered;

[0032] The shrinking sub-function module is used to set an interval time, and input all embedded databases into the shrinking sub-function module every time the interval time is reached;

[0033] The shrinking sub-function module is used to sort the input embedded databases in descending order according to the buffer size, and sequentially restore the buffers of the embedded databases to the default size, that is, 2000KB, from large to small according to the sorting. After each restoration, judge the memory utilization rate again. When the memory utilization rate returns below 60%, stop the restoration of the buffer. When the buffers of all embedded databases have been restored to the default size and the memory utilization rate of the system still fails to return below 60%, end the shrinking sub-function module.

[0034] Furthermore, the specific steps of the embedded database connection module are as follows:

[0035] Judge whether an embedded database is currently connected. If an embedded database is not currently connected, send a message to all embedded databases for connection. If the embedded database agrees to the connection, connect to it and maintain the connection of the embedded database through a heartbeat mechanism in the follow-up;

[0036] If an embedded database is currently connected, a message is re-sent to the unconnected embedded database for connection. If the embedded database agrees to the connection, it is connected, and the connection to the embedded database is maintained through a heartbeat mechanism subsequently.

[0037] Further, when the buffer size is the globally optimal record, exploration continues in the ε-greedy mode.

[0038] Further, in the difference range, the memory size is gradually increased, and each increase in the memory size is 20MB.

[0039] Further, the interval time is 10 minutes.

[0040] The beneficial effects of the present invention are:

[0041] In view of the lightweight characteristics of the embedded database, this application uniformly optimizes all embedded databases instead of optimizing each database separately, improving the optimization efficiency and avoiding the time waste caused by individual optimization. And this application helps the system tide over the peak period of memory usage by shrinking the sub-function and restoring the buffer to the default size in a sorted manner, thereby solving the problem that the cache of the database grows significantly, resulting in insufficient memory when other programs are running and other programs cannot run. This application is uniformly managed by the optimizer for resources, locates different requirements, and maximizes the throughput when resources are in short supply. Description of the Drawings

[0042] Figure 1 It is the overall flowchart of this application;

[0043] Figure 2 It is the flowchart of the increasing sub-function;

[0044] Figure 3 It is the flowchart of the shrinking sub-function. Detailed Embodiments

[0045] It should be specifically noted that, without conflict, the various embodiments disclosed in this application can be combined with each other.

[0046] Detailed Embodiment 1: Refer to Figure 1 This embodiment is specifically described. The method for coordinating and optimizing the buffer size of multiple embedded databases described in this embodiment includes the following steps:

[0047] Step 1: Connect to the embedded database and listen for the transaction status signals sent by the connected embedded database. The transaction status includes active embedded databases and inactive embedded databases. An active embedded database signal refers to an embedded database where a read or write transaction is in progress or the read or write transaction has not been completed. An inactive embedded database signal refers to an embedded database where no transaction is currently in progress.

[0048] Step 2: After collecting the transaction status signals sent by all the embedded databases, according to the transaction status, input the active embedded databases into the enlargement sub-function and the inactive embedded databases into the reduction sub-function, and execute Step 5. The enlargement sub-function and the reduction sub-function are respectively used for enlarging and reducing the buffer of the embedded database.

[0049] In the enlargement sub-function, first determine whether the current memory usage rate is greater than or equal to 60%. If so, stop enlarging the buffer of the active embedded database, input the active embedded database into the reduction sub-function, and execute Step 5. Otherwise, continue with the buffer enlargement operation for the active embedded database.

[0050] If the enlargement operation continues, first calculate the difference between the current memory usage rate and 60%, and based on the actual memory size and the difference, obtain the maximum memory size allowed for enlarging the buffer. Within the allowed range, for each embedded database input into the enlargement sub-function in the order of the embedded database, perform the following steps:

[0051] Gradually increase the memory size in the buffer of the embedded database and record the throughput rate during the operation of the embedded database. If the throughput rate increases monotonically with the memory size in the buffer, then the buffer size at this time is the local best record. Then continue to increase the memory size in the buffer and update the local best record. If the throughput rate does not increase monotonically with the memory size in the buffer, then the buffer size at this time is the global best record.

[0052] If there is a global best record, select the memory size of the buffer corresponding to the global best record as the final buffer size. If there is no global best record, select the memory size of the buffer corresponding to the latest local best record as the final buffer size.

[0053] Step 3: Add the current memory usage value to the final buffer size corresponding to all embedded databases to obtain the memory utilization rate, and then determine whether the memory utilization rate exceeds 60%. If it does not exceed 60%, apply the final buffer size of all embedded databases. If it exceeds 60%, sort the final buffer size of the embedded databases obtained in Step 2 based on the memory size. Then, according to the sorting, restore the final buffer size of the embedded databases in descending order until the memory utilization rate does not exceed 60%. At this time, apply the final buffer size of the embedded databases that have not been restored;

[0054] Step 4: Set the interval time, and when each interval time arrives, input all embedded databases into the shrinking sub-function and execute Step 5;

[0055] Step 5: In the shrinking sub-function, sort the input embedded databases in descending order according to the buffer size, and according to the sorting, restore the buffers of the embedded databases to the default size, i.e., 2000KB, in descending order. After each restoration, judge the memory utilization rate again. When the memory utilization rate returns below 60%, stop the buffer restoration. When the buffers of all embedded databases have been restored to the default size and the memory utilization rate of the system still fails to return below 60%, end the shrinking sub-function.

[0056] Specific Embodiment 2: This embodiment is a further description of Specific Embodiment 1. The difference between this embodiment and Specific Embodiment 1 is that the specific steps of Step 1 are as follows:

[0057] Judge whether an embedded database is currently connected. If an embedded database is not currently connected, send a message to all embedded databases for connection. If the embedded database agrees to the connection, connect to it and maintain the connection of the embedded database through a heartbeat mechanism in the follow-up;

[0058] If an embedded database is currently connected, resend a message to the unconnected embedded database for connection. If the embedded database agrees to the connection, connect to it and maintain the connection of the embedded database through a heartbeat mechanism in the follow-up.

[0059] Specific Embodiment 3: This embodiment is a further description of Specific Embodiment 1. The difference between this embodiment and Specific Embodiment 1 is that when the buffer size is the global best record, the ε-greedy mode is adopted to continue the exploration.

[0060] Specific Embodiment 4: This embodiment is a further description of Specific Embodiment 1. The difference between this embodiment and Specific Embodiment 1 is that in the difference range, the memory size is gradually increased, and each increase in the memory size is 20MB.

[0061] Specific Embodiment 5: This embodiment is a further description of Specific Embodiment 1. The difference between this embodiment and Specific Embodiment 1 is that the interval time is 10 minutes.

[0062] Specific Embodiment 6: The multi-embedded database buffer size coordination and optimization system described in this embodiment includes an embedded database connection module, a buffer adjustment module, an increase sub-function module, a buffer recovery module, and a decrease sub-function module;

[0063] The embedded database connection module is used to connect to the embedded database and listen for the transaction status signals sent by the embedded database in the connected embedded database. The transaction status includes active embedded databases and inactive embedded databases. The active embedded database signal refers to an embedded database where a read or write transaction is in progress or the read or write transaction has not been completed. The inactive embedded database signal refers to an embedded database where no transaction is currently in progress;

[0064] The buffer adjustment module is used to, after collecting all the transaction status signals sent by the embedded databases, input the active embedded databases into the increase sub-function module and the inactive embedded databases into the decrease sub-function module according to the transaction status. The increase sub-function module and the decrease sub-function module are respectively used to increase and decrease the buffer of the embedded database;

[0065] The increase sub-function module is used to determine whether the current memory usage rate is greater than or equal to 60%. If so, stop increasing the buffer of the active embedded database and input the active embedded database into the decrease sub-function module. Otherwise, continue the operation of increasing the buffer of the active embedded database;

[0066] If the increase operation continues, first calculate the difference between the current memory usage rate and 60%, and based on the actual memory size and the difference, obtain the maximum memory size allowed for increasing the buffer. Within the allowable range, for each embedded database input into the increase sub-function module in the order of the embedded database, perform the following steps:

[0067] Gradually increase the memory size in the embedded database buffer and record the throughput rate during the operation of the embedded database. If the throughput rate monotonically increases with the memory size in the buffer, then the buffer size at this time is the local best record. Then continue to increase the memory size in the buffer and update the local best record. If the throughput rate does not monotonically increase with the memory size in the buffer, then the buffer size at this time is the global best record;

[0068] If there is a global best record, select the memory size of the buffer corresponding to the global best record as the final buffer size. If there is no global best record, select the memory size of the buffer corresponding to the latest local best record as the final buffer size;

[0069] The buffer recovery module is used to add the current memory usage value to the final buffer size corresponding to all embedded databases to obtain the memory utilization rate, and then determine whether the memory utilization rate exceeds 60%. If it does not exceed 60%, apply the final buffer sizes of all embedded databases. If it exceeds 60%, sort the final buffer sizes of the obtained embedded databases based on the memory size, and then, according to the sorting, recover the final buffer sizes of the embedded databases from large to small in sequence until the memory utilization rate does not exceed 60%. At this time, apply the final buffer sizes of the embedded databases that have not been recovered;

[0070] The shrinking sub-function module is used to set an interval time and input all embedded databases into the shrinking sub-function module every time the interval time is reached;

[0071] The shrinking sub-function module is used to sort the input embedded databases from large to small according to the buffer size, and then, according to the sorting from large to small, restore the buffers of the embedded databases to the default size, that is, 2000KB in sequence. After each restoration, judge the memory utilization rate again. When the memory utilization rate returns below 60%, stop the buffer restoration. When the buffers of all embedded databases have been restored to the default size and the memory utilization rate of the system still fails to return below 60%, end the shrinking sub-function module.

[0072] Specific Embodiment 7: This embodiment is a further description of Specific Embodiment 6. The difference between this embodiment and Specific Embodiment 6 is that the specific steps of the embedded database connection module are as follows:

[0073] Judge whether an embedded database is currently connected. If an embedded database is not currently connected, send a message to all embedded databases for connection. If the embedded database agrees to the connection, connect to it and maintain the connection of the embedded database through a heartbeat mechanism in the follow-up;

[0074] If an embedded database is currently connected, resend a message to the unconnected embedded database for connection. If the embedded database agrees to the connection, connect to it and maintain the connection of the embedded database through a heartbeat mechanism in the follow-up.

[0075] Embodiment VIII: This embodiment is a further elaboration of Embodiment VI. The difference between this embodiment and Embodiment VI is that when the buffer size is the globally optimal record, the ε-greedy mode is adopted to continue the exploration.

[0076] Embodiment IX: This embodiment is a further elaboration of Embodiment VI. The difference between this embodiment and Embodiment VI is that within the difference range, when gradually increasing the memory size, the memory size increased each time is 20MB.

[0077] Embodiment X: This embodiment is a further elaboration of Embodiment VI. The difference between this embodiment and Embodiment VI is that the interval time is 10 minutes.

[0078] Example:

[0079] 1. A method for adjusting the buffer size of an embedded database, characterized in that:

[0080] It includes the following steps:

[0081] S1. If the embedded database is not currently connected, send a message to all embedded databases for connection. If the embedded database agrees to the connection, connect to it and maintain the connection of the embedded database through the heartbeat mechanism subsequently; if the embedded database is currently connected, resend a message to the unconnected embedded database for connection. After connecting to these databases, listen for the signal of the transaction status sent by the database in all the connected embedded databases. The signals are divided into active and inactive. An active embedded database refers to an embedded database where a read or write transaction is in progress and not yet completed, while an inactive embedded database refers to an embedded database where no transaction is currently in progress. Active embedded databases represent that the current embedded database is conducting transactions and using the system's computing resources such as the most important memory resources. In order to speed up their transaction processing speed, for this part of the databases, the amount of resources they use can be increased. On the other hand, inactive databases are not currently processing transactions and do not require excessive memory resources. Therefore, the system's memory resources need to be reclaimed from their buffers;

[0082] S2. After collecting the signals sent by all the embedded databases, input all the active embedded databases into the increase sub-function, and input all the inactive embedded databases into the decrease sub-function. The increase sub-function and the decrease sub-function respectively manage the increase and decrease of the buffer of the embedded database;

[0083] S3. In the increasing sub-function in S2, first, check whether the memory already used in the system is greater than or equal to 60%. If it exceeds, it means that the total memory usage rate of the current system is already too large, imposing a relatively heavy burden on the system. At this time, the buffer of the embedded database should not be further increased. Therefore, stop increasing the buffer of the active embedded database at this time and re-enter these active embedded databases into the shrinking sub-function. If the current memory usage size is less than 60%, it means that the current system is relatively idle and has the capacity to bear the operation of increasing the buffer of the embedded database, thus accelerating the speed of the embedded database. Therefore, continue to perform the operation of increasing the buffer of these active embedded databases;

[0084] S4. If the increasing operation continues, the algorithm first calculates the difference between the current memory usage rate and 60%, and uses this difference as the maximum memory size allowed for increasing the buffer. Here, introduce a new structure: record. The role of the record is to record the throughput rate of a certain embedded database during its operation corresponding to each buffer size, and sort these records according to the buffer size. If the throughput rate monotonically increases with the buffer size, the current record is the local best record, and the local best record value takes the buffer size corresponding to the current highest throughput rate. If the throughput rate does not monotonically increase with the buffer size, the record with the highest throughput rate among all records is the global best record. After calculating the difference, query the previous records for each embedded database, which can be divided into two cases: The first case is when the record is the local best record rather than the global best record, select the buffer size of the local best record and continue to increase the buffer to explore the global best record. The second case is when the global best record has been reached. At this time, adopt the ε-greedy mode, making it have a 10% probability of exploration, that is, randomly generate a buffer size. Once a situation with a higher throughput rate is explored, modify the global best record and use the corresponding higher buffer size in this embedded database;

[0085] S5. After obtaining the records, determine whether the utilization rate of the sum of the buffer sizes in these records plus the memory usage value used by other programs in the current system exceeds 60%, so as to prevent the system state from deteriorating due to the increase in buffer size. If it does not exceed, apply the buffer sizes in the records to these embedded databases in sequence, continuously track the operation of these embedded databases, and record the exploration of subsequent local best and global best; if it exceeds 60%, it means that the increase in the buffer has affected the system environment, and a part of the buffer needs to be reset to its original state. The specific method is to sort these record values from largest to smallest, and starting from the embedded database with the largest buffer, restore the buffers of these databases to the original buffer size of the embedded database until it does not exceed 60%. After ensuring that the impact on the system is reduced, apply the buffer size values that have not been restored to their corresponding embedded databases in actuality, and continuously track the operation of these databases;

[0086] S6. Input the embedded databases into the shrinking sub-function in S2, S3, and S5. The purpose of these situations is to avoid a large impact of the buffer size adjustment of the embedded database on the overall memory environment of the system, thus slowing down the running speed of other processes; and every 10 times the algorithm runs, it will automatically enter the shrinking sub-function. This situation is to prevent the problem of high utilization rate of system memory resources caused by changes in the system's own memory environment. For example, when a series of new processes start running in the system and the buffer size of the embedded database remains unchanged, the system resource utilization rate is higher than 60%. At this time, the buffer of the embedded database needs to be shrunk to make these processes run normally. After entering the shrinking sub-function, first arrange all the currently input embedded databases in descending order according to the buffer sizes they use.

[0087] S7. Arrange the buffers of the embedded databases in S6 from largest to smallest, and restore the buffers of these embedded databases to their default size of 2000KB. After restoring the buffer of each embedded database to the default value, judge the system memory utilization rate again. When the system memory utilization rate returns to below 60% again, stop the buffer restoration. Or, when the system memory utilization rate still fails to return to below 60% after restoring the buffers of all embedded databases to the default value, select to end the shrinking sub-function at this time. At this time, the impact of the buffer of the embedded database has been reduced to the minimum, so as to help the system tide over the peak period of memory usage.

[0088] This application provides higher database performance in the face of a multi-embedded database environment.

[0089] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code. The solutions in the embodiments of the present application can be implemented in various computer languages. For example, object-oriented programming languages such as Java and interpreted scripting languages such as JavaScript, etc.

[0090] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0091] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0093] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments as well as all changes and modifications falling within the scope of the present application.

[0094] It should be noted that the specific implementation manners are only explanations and illustrations of the technical solutions of the present invention, and the scope of the rights protection cannot be limited thereby. Any partial changes made only in accordance with the claims and the specification of the present invention shall still fall within the protection scope of the present invention.

Claims

1. Method for Coordinating and Optimizing Buffer Sizes of Multiple Embedded Databases Characterized in that it includes the following steps: Step 1: Connect to the embedded database and listen for the transaction status signals sent by the embedded database in the connected embedded database. The transaction status includes active embedded databases and inactive embedded databases. An active embedded database signal refers to an embedded database where a read or write transaction is in progress or a read or write transaction has not been completed. An inactive embedded database signal refers to an embedded database where no transaction is currently in progress; Step 2: After collecting the transaction status signals sent by all embedded databases, according to the transaction status, input the active embedded databases into the increase sub-function and the inactive embedded databases into the decrease sub-function, and execute Step 5. The increase sub-function and the decrease sub-function are respectively used for increasing and decreasing the buffer of the embedded database; In the increase sub-function, first determine whether the current memory utilization rate is greater than or equal to 60%. If so, stop increasing the buffer of the active embedded database, input the active embedded database into the decrease sub-function, and execute Step 5. Otherwise, continue with the buffer increase operation for the active embedded database; If the increase operation continues, first calculate the difference between the current memory utilization rate and 60%, and based on the actual memory size and the difference, obtain the maximum memory size allowed for increasing the buffer. Within the allowed range, for each embedded database input into the increase sub-function according to the order, perform the following steps: Gradually increase the memory size in the buffer of the embedded database and record the throughput rate during the operation of the embedded database. If the throughput rate monotonically increases with the memory size in the buffer, then the buffer size at this time is the local best record. Then continue to increase the memory size in the buffer and update the local best record. If the throughput rate does not monotonically increase with the memory size in the buffer, then the buffer size at this time is the global best record; If there is a global best record, select the memory size of the buffer corresponding to the global best record as the final buffer size. If there is no global best record, select the memory size of the buffer corresponding to the latest local best record as the final buffer size; Step 3: Add the final buffer sizes corresponding to all embedded databases to the current memory usage value to obtain the memory utilization rate, and then determine whether the memory utilization rate exceeds 60%. If it does not exceed 60%, apply the final buffer sizes of all embedded databases. If it exceeds 60%, sort the final buffer sizes of the embedded databases obtained in Step 2 based on the memory size. Then, according to the sorting, restore the final buffer sizes of the embedded databases in descending order until the memory utilization rate does not exceed 60%. At this time, apply the final buffer sizes of the embedded databases that have not been restored; Step 4: Set an interval time, and when each interval time arrives, input all embedded databases into the decrease sub-function and execute Step 5; Step Five: In the shrinking sub-function, sort the input embedded databases in descending order according to the buffer size, and restore the buffers of the embedded databases to the default size in order from largest to smallest. After each restoration, judge the memory usage rate again. When the memory usage rate returns below 60%, stop restoring the buffers. When the buffers of all embedded databases have been restored to the default size and the memory usage rate of the system still fails to return below 60%, end the shrinking sub-function.

2. The method for coordinating and optimizing the buffer sizes of multiple embedded databases according to claim 1, characterized in that the specific steps of Step One are: Judge whether an embedded database is currently connected. If an embedded database is not currently connected, send a message to all embedded databases for connection. If the embedded database agrees to the connection, connect to it and maintain the connection of the embedded database through a heartbeat mechanism in the follow-up; If an embedded database is currently connected, resend a message to the unconnected embedded database for connection. If the embedded database agrees to the connection, connect to it and maintain the connection of the embedded database through a heartbeat mechanism in the follow-up.

3. The method for coordinating and optimizing the buffer sizes of multiple embedded databases according to claim 1, characterized in that When the buffer size is the globally optimal record, continue the exploration in the ε-greedy mode.

4. The method for coordinating and optimizing the buffer sizes of multiple embedded databases according to claim 1, characterized in that In the difference range, gradually increase the memory size, and the memory size increased each time is 20MB.

5. The method for coordinating and optimizing the buffer sizes of multiple embedded databases according to claim 1, characterized in that The interval time is 10 minutes.

6. A system for coordinating and optimizing the buffer sizes of multiple embedded databases, characterized in that it includes an embedded database connection module, a buffer adjustment module, an increasing sub-function module, a buffer restoration module, and a shrinking sub-function module; The embedded database connection module is used to connect to the embedded database and listen for the transaction status signals sent by the embedded database in the already connected embedded databases. The transaction status includes active embedded databases and inactive embedded databases. The active embedded database signal refers to an embedded database where a read or write transaction is in progress or a read or write transaction has not been completed. The inactive embedded database signal refers to an embedded database where no transaction is currently in progress; The buffer adjustment module is used to, after collecting the transaction status signals sent by all embedded databases, input the active embedded databases into the increasing sub-function module and the inactive embedded databases into the shrinking sub-function module according to the transaction status. The increasing sub-function module and the shrinking sub-function module are respectively used for increasing and shrinking the buffers of the embedded databases; The increasing sub-function module is used to determine whether the current memory usage rate is greater than or equal to 60%. If so, it stops increasing the buffer of the active embedded database and inputs the active embedded database into the shrinking sub-function module. Otherwise, it continues to perform the buffer increasing operation on the active embedded database; If the increasing operation continues, first calculate the difference between the current memory usage rate and 60%, and based on the actual memory size and the difference, obtain the maximum memory size of the buffer that can be increased. Within the allowable range, for each embedded database according to the order in which the embedded databases are input into the increasing sub-function module, perform the following steps: Gradually increase the memory size in the buffer of the embedded database and record the throughput rate during the operation of the embedded database. If the throughput rate monotonically increases with the memory size in the buffer, then the buffer size at this time is the local best record. Then continue to increase the memory size in the buffer and update the local best record. If the throughput rate does not monotonically increase with the memory size in the buffer, then the buffer size at this time is the global best record; If there is a global best record, select the memory size of the buffer corresponding to the global best record as the final buffer size. If there is no global best record, select the memory size of the buffer corresponding to the latest local best record as the final buffer size; The buffer recovery module is used to add the final buffer size corresponding to all embedded databases to the current memory usage value to obtain the memory usage rate, and then determine whether the memory usage rate exceeds 60%. If it does not exceed 60%, apply the final buffer sizes of all embedded databases. If it exceeds 60%, sort the obtained final buffer sizes of the embedded databases based on the memory size. Then, according to the sorting, restore the final buffer sizes of the embedded databases from large to small in sequence until the memory usage rate does not exceed 60%. At this time, apply the final buffer sizes of the embedded databases that have not been restored; The shrinking sub-function module is used to set an interval time and input all embedded databases into the shrinking sub-function module every time the interval time is reached; The shrinking sub-function module is used to sort the input embedded databases in descending order of buffer size, and restore the buffers of the embedded databases to the default size, that is, 2000KB, in descending order according to the sorting. After each restoration, judge the memory usage rate again. When the memory usage rate returns below 60%, stop the buffer restoration. When the buffers of all embedded databases have been restored to the default size and the memory usage rate of the system still fails to return below 60%, end the shrinking sub-function module.

7. The multi-embedded database buffer size coordination and optimization system according to claim 6, characterized in that The specific steps of the embedded database connection module are: Determine whether an embedded database is currently connected. If the embedded database is not currently connected, send a message to all embedded databases to establish a connection. If the embedded database agrees to the connection, establish a connection with it and maintain the connection with the embedded database through a heartbeat mechanism in the subsequent process; If an embedded database is currently connected, resend a message to the unconnected embedded database to establish a connection. If the embedded database agrees to the connection, establish a connection with it and maintain the connection with the embedded database through a heartbeat mechanism in the subsequent process.

8. The multi-embedded database buffer size coordination and optimization system according to claim 6, characterized in that When the buffer size is the globally optimal record, continue the exploration in the ε-greedy mode.

9. The multi-embedded database buffer size coordination and optimization system according to claim 6, characterized in that In the difference range, gradually increase the memory size, and the memory size increased each time is 20MB.

10. The multi-embedded database buffer size coordination and optimization system according to claim 6, characterized in that The interval time is 10 minutes.

Images