Database parameter configuration method and device and storage medium

Abstract

The embodiment of the invention discloses a database parameter configuration method and device and a storage medium. The method comprises the steps that a first computing cluster used for operating a target database and a second computing cluster used for conducting database parameter configuration are determined, and first computing nodes in the first computing cluster and second computing nodes in the second computing cluster are the same in specification; for database parameters of the target database, the database parameters are configured for all the second computing nodes respectively, a target parameter value set is obtained, and the target parameter value set comprises target parameter values corresponding to all the second computing nodes respectively; and synchronizing the target parameter value group to the first computing cluster so as to configure the target parameter value corresponding to the first computing node in the target parameter value group to the first computing node for each first computing node. According to the technical scheme of the embodiment of the invention, resource utilization balance can be realized.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of computers, and in particular to a database parameter configuration method and device and storage medium. BACKGROUND

[0002] With the development of technology, the market demand for cloud data analysis services continues to grow. Communication operators often need to process massive user data (such as call records, SMS records, and traffic usage, etc.) to charge on this basis.

[0003] The target database (such as a cloud analysis database) using a storage-computing separation architecture as a new solution, by decoupling data storage and computing processes, realizes independent storage of data and flexible analysis and processing through multiple computing nodes, and thus enhances the parallel processing capability of data query and analysis.

[0004] It should be noted that when the target database is applied, database parameters need to be configured for each computing node. However, the current database parameter configuration scheme is prone to uneven resource utilization, which needs to be solved urgently. SUMMARY

[0005] Embodiments of the present application provide a database parameter configuration method, device and storage medium, which solve the problem of uneven resource utilization.

[0006] According to an aspect of the present application, a database parameter configuration method can include:

[0007] determining a first computing cluster for operating a target database and a second computing cluster for performing database parameter configuration, wherein a first computing node in the first computing cluster and a second computing node in the second computing cluster have the same specifications;

[0008] configuring database parameters for each second computing node for the database parameters of the target database, to obtain a target parameter value group, wherein the target parameter value group includes target parameter values corresponding to each second computing node;

[0009] synchronizing the target parameter value group to the first computing cluster, to configure the target parameter values corresponding to each first computing node in the target parameter value group to the first computing node for each first computing node.

[0010] According to another aspect of the present application, a database parameter configuration device can include:

[0011] The computing cluster determining module is configured to determine a first computing cluster for operating the target database and a second computing cluster for performing the database parameter configuration, wherein a first computing node in the first computing cluster has the same specification as a second computing node in the second computing cluster;

[0012] The target parameter value set obtaining module is configured to configure, for each second computing node, a database parameter of the target database to obtain a target parameter value set, wherein the target parameter value set comprises a target parameter value corresponding to each second computing node.

[0013] The database parameter configuration module is configured to synchronize the target parameter value set to the first computing cluster, and for each first computing node, configure a target parameter value corresponding to the first computing node in the target parameter value set to the first computing node.

[0014] According to another aspect of the present application, an electronic device can comprise:

[0015] at least one processor; and

[0016] a memory connected to the at least one processor in communication; wherein

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to implement the database parameter configuration method provided by any of the embodiments of the present application.

[0018] According to another aspect of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions for enabling a processor to implement the database parameter configuration method provided by any of the embodiments of the present application.

[0019] According to another aspect of the present application, a computer program product is provided, and the computer program product stores a computer program, and the computer program is executed by a processor to implement the database parameter configuration method provided by any of the embodiments of the present application.

[0020] The technical scheme of the embodiment of the present application is to determine a first computing cluster for operating a target database and a second computing cluster for performing database parameter configuration, wherein the first computing nodes in the first computing cluster and the second computing nodes in the second computing cluster have the same specifications; for the database parameters of the target database, the database parameters are respectively configured for each second computing node to obtain a target parameter value group, wherein the target parameter value group includes target parameter values corresponding to each second computing node; the target parameter value group is synchronized to the first computing cluster, so as to configure the target parameter values corresponding to each first computing node in the target parameter value group to the first computing node. The above technical scheme makes full use of the characteristics of the compute-storage separation architecture, and under the premise of ensuring that the normal operation of the existing computing cluster (i.e., the first computing cluster applied as a production cluster) is not affected, a second computing cluster specially used for database parameter configuration is added, the database parameters are respectively configured for each second computing node in the second computing cluster, and then the database parameters are synchronized to the first computing cluster, so that the resources on each first computing node can be fully utilized, thereby solving the problem of uneven resource utilization, which helps to improve the overall performance of the target database and has great significance for large-scale data analysis and processing.

[0021] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0023] Figure 1 is a schematic diagram of an optional example of various database parameter configuration methods provided according to the embodiments of the present application;

[0024] Figure 2 is a flowchart of a database parameter configuration method provided according to the embodiments of the present application;

[0025] Figure 3 is a flowchart of another database parameter configuration method provided according to the embodiments of the present application;

[0026] Figure 4 is a structural block diagram of a database parameter configuration device provided according to the embodiments of the present application;

[0027] Figure 5Fig. 1 is a structural schematic diagram of an electronic device for realizing a database parameter configuration method according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the personnel in the technical field better understand the present application scheme, the technical scheme in the embodiment of the present application will be described clearly and completely below in combination with the drawings in the embodiment of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. The case of "target", "original" and the like is similar, which will not be repeated here. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0030] It should be noted that in the technical scheme of the present application, the collection, collection, updating, analysis, processing, use, transmission, storage and the like of user personal information are in line with the relevant legal regulations, are used for legal purposes, and do not violate public order and good customs. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain user personal information security and network security.

[0031] Before introducing the embodiments of the present application, the application scenarios of the embodiments of the present application and the reasons why the related database parameter configuration scheme is easy to cause uneven resource utilization are exemplarily described, so as to better understand how the database parameter configuration scheme proposed by the embodiments of the present application realizes the balanced resource utilization.

[0032] For example, according to the above description, for a communication operator, it is crucial to process massive user data. By configuring (or optimizing) the database parameters, on the one hand, the speed and efficiency of data processing can be significantly improved, supporting real-time data analysis, so as to complete the statistical and analysis work of a large amount of data in a short time, which is of great significance to timely generate bills; on the other hand, each transaction of the user can be more accurately tracked and recorded, the billing accuracy is improved, the billing disputes are reduced, and the billing abnormalities can be more quickly detected and corrected in time to ensure the correctness of the bill. At the same time, optimizing the database parameters can achieve more effective load balancing, prevent service interruption caused by overload of a single computing node, and respond more quickly to user query requests (such as querying the balance and traffic usage, etc.), thereby improving the user experience.

[0033] The current bill processing system usually applies an analytical database under a storage-computing separation architecture, which can be considered as a database architecture design decoupling data storage and computing process. The data is stored in an independent storage system, and the computing node accesses the data in the storage system for computing through the network. In this way, independent storage of data and flexible allocation of computing resources can be achieved.

[0034] Since the storage-computing separation architecture supports multiple operating systems and hardware environments, different computing nodes can run based on different hardware configurations, and the related database parameter configuration scheme allocates uniform database parameters to each computing node, which makes it difficult to adapt to such a heterogeneous environment, thereby limiting the full play of the performance of each host (i.e., each computing node), causing certain performance loss, i.e., causing uneven resource utilization, specifically, some hosts have abundant resources and some hosts have insufficient resources. Therefore, the embodiment of the present application proposes a new method as shown in Figure 1 to optimize the parameter configuration of the computing node in the analytical database under the storage-computing separation architecture, so as to improve the resource utilization and query speed. Next, this will be described in detail.

[0035] Figure 2 is a flowchart of a database parameter configuration method provided by the embodiment of the present application. The embodiment can be applied to the case of allocating database parameters to the computing node, especially to the case of allocating database parameters to the database process running on the computing node, which makes the database process apply and use resources (such as central processing units, memories, and disks, etc.) to the computing node according to the allocated database parameters to complete the data processing task. The method can be executed by the database parameter configuration device provided by the embodiment of the present application, which can be realized by software and / or hardware, and the device can be integrated on an electronic device, which can be a second computing cluster (i.e., a computing node cluster) or a third computing cluster (i.e., a database process cluster). Figure 1The certain computing node in the computing cluster B or D of the example can also be a certain computing node outside the second computing cluster, which is related to the actual situation and is not specifically limited here.

[0036] Referring to Figure 2 The method of the embodiment of the application specifically comprises the following steps:

[0037] S110. Determine a first computing cluster for operating a target database and a second computing cluster for performing database parameter configuration, wherein a first computing node in the first computing cluster and a second computing node in the second computing cluster are of the same specification.

[0038] The target database can be understood as a database for which database parameters are to be configured, and on this basis, in combination with the application scenarios that can be involved in the embodiment of the application, the database can be an analytical database applied to a bill processing system. The analytical database can be implemented using a storage-computing separation architecture.

[0039] The first computing cluster can be understood as a computing cluster through which a user operates a target database, i.e., a computing cluster for processing business requirements. In the embodiment of the application, the computing nodes in the computing cluster are referred to as first computing nodes.

[0040] The second computing cluster can be understood as a computing cluster for performing database parameter configuration, so as to perform database parameter configuration therein and avoid affecting the normal operation of the first computing cluster. In the embodiment of the application, the computing nodes in the computing cluster are referred to as second computing nodes, so as to be distinguished from the computing nodes in the first computing cluster.

[0041] It should be noted that the specification of the first computing node is the same as that of the second computing node, for example, the number of the first computing nodes is the same as that of the second computing nodes, and for another example, the resource configuration on the first computing nodes is the same as that on the second computing nodes, and the like, so that the database parameters obtained in the second computing cluster can be directly applied in the first computing cluster.

[0042] For example, step one: preparation

[0043] Referring to Figure 1 The computing cluster A (i.e., the first computing cluster) and the computing cluster B (i.e., the second computing cluster) in the example, the computing cluster C and the computing cluster D are the same, and the following operations are performed: a computing cluster A is created for processing business requirements, and the computing cluster is used for the main database operation of the user. A computing cluster B of the same specification as the computing cluster A is created, which is used for configuring (or said) optimizing database parameters. On the basis of the storage-computing separation architecture, the two computing clusters can use the same storage data. The data inserted in the computing cluster A can also be seen in the computing cluster B. Specifically,

[0044] A core computing cluster named Computing Cluster A is established, which should have the ability to meet daily business needs, including but not limited to data read / write, transaction management, etc.

[0045] A computing cluster B with the same number of hosts and hardware configuration as Computing Cluster A is constructed. The two clusters share the same object storage, ensuring that any changes occurring on Cluster A can be accessed by Cluster B in real time.

[0046] During the creation of Computing Cluster A and Computing Cluster B, an automated deployment tool is introduced to ensure consistency and repeatability of the creation process. Detailed performance testing and benchmarking are conducted on each host (i.e., computing node), including CPU performance, memory read / write speed, and disk I / O, to better understand hardware performance and provide more accurate references for subsequent database parameter optimization. Furthermore, a monitoring system is established to monitor the running state of Computing Cluster A and Computing Cluster B in real time, including CPU usage, memory usage, disk space, and network traffic. By setting thresholds and alarm mechanisms, potential performance problems or failures can be discovered in a timely manner, and appropriate measures can be taken to address them.

[0047] (1) Integration of automated deployment and records

[0048] During the creation of Computing Cluster A and Computing Cluster B, automated deployment tools can be fully utilized to ensure efficient, consistent, and repeatable creation processes. Detailed records of each step and configuration information can be kept during the deployment process, including network settings, storage allocation, and software installation. Such records provide detailed references for subsequent troubleshooting and optimization. For example, for the creation of a computing cluster, a standardized deployment process is developed: first, define the required hardware resources and software configuration requirements; then, use automated tools to deploy step by step according to the pre-set process, and record each key step during the deployment process, such as specific network configuration parameters, storage allocation methods, and software installation versions.

[0049] (2) Integration of comprehensive performance testing and archive establishment

[0050] For each host, comprehensive and in-depth performance testing and benchmarking are conducted, covering multiple key aspects such as CPU performance, memory read / write speed, and disk I / O. Professional performance testing tools are used to obtain accurate performance indicator data, and the performance of different hardware configurations is compared and analyzed. Based on the test results, a detailed performance profile is established for each host, recording its performance under different load conditions, strengths and weaknesses. The performance profile includes the host's hardware configuration information, performance indicator data under different test scenarios, and analysis and summary of performance. Such performance profiles can better adjust to hardware characteristics during subsequent database parameter optimization.

[0051] (3) Establishment and integration of powerful monitoring system

[0052] A powerful and comprehensive monitoring system is established to monitor the running status of computing cluster A and computing cluster B in real time. The monitoring indicators include not only basic indicators such as CPU usage, memory usage, disk space, and network traffic, but also indicators closely related to database performance such as target database connection number, query execution time, and transaction processing speed. Advanced monitoring tools are used to create a visual monitoring interface, allowing administrators to intuitively understand the system's running status. By setting reasonable thresholds and alarm mechanisms, when the monitoring indicators exceed the preset thresholds, an alarm is automatically sent to the administrator to take appropriate measures for processing in a timely manner. At the same time, historical records and analysis functions of monitoring data are established to allow administrators to review the system's running trends, detect potential performance problems in a timely manner, and perform preventive maintenance. Historical records of monitoring data are stored in the target database for long-term analysis and trend prediction. Through analysis of historical data, the changing trend of system performance is discovered, potential problems are predicted in advance, and appropriate measures are taken for optimization and adjustment.

[0053] S120. For the database parameters of the target database, configure the database parameters for each second computing node respectively to obtain a target parameter value group, wherein the target parameter value group includes target parameter values corresponding to each second computing node respectively.

[0054] Among them, the database parameter can be understood as the parameter to be configured (or said to be optimized) of the target database. Configure the parameter for each second computing node respectively to obtain a target parameter value group, which includes target parameter values corresponding to each second computing node respectively, and the target parameter value is the value of the parameter under the corresponding second computing node.

[0055] On this basis, in combination with the application scenarios that the embodiments of the present application can involve, optionally, the database parameter configuration process can be implemented based on a particle swarm optimization (PSO) algorithm. The PSO is an optimization algorithm based on swarm intelligence, which simulates the foraging behavior of a bird swarm. Particles in the parameter space continuously adjust their positions and velocities according to their own and group experiences to find the optimal solution. In the embodiments of the present application, the PSO is used to optimize the database parameters. It should be noted that the PSO can efficiently find the optimal solution in a complex parameter space, thereby improving the accuracy and applicability of the database parameter optimization. In particular, the PSO can be suitable for processing large-scale and variable cluster environments, and has good universality.

[0056] S130. Synchronize the target parameter value set to the first computing cluster to configure, for each first computing node, the target parameter value corresponding to the first computing node in the target parameter value set to the first computing node.

[0057] Since the first computing nodes and the second computing nodes have the same specifications, the database parameters obtained in the second computing cluster can be directly synchronized to the first computing cluster. Specifically, for each first computing node, the target parameter value corresponding to the first computing node in the target parameter value set can be configured to the first computing node, so that the database process running on the first computing node can apply for and use resources based on the target parameter value, thereby ensuring the full use of the resources.

[0058] The above technical solution can be used in a real-time charging system. Specifically, by optimizing the database parameters, real-time charging is realized, and users can immediately view the latest call records and expense details. The consistency of the data shared between different computing clusters is ensured, and charging errors caused by data synchronization problems are prevented. The optimized database can recover from failures more quickly, and the occurrence of charging delays or data loss caused by charging system failures is reduced.

[0059] The technical solution of the embodiment of the application determines a first computing cluster for operating a target database and a second computing cluster for performing database parameter configuration, wherein the first computing nodes in the first computing cluster and the second computing nodes in the second computing cluster have the same specifications; for the database parameters of the target database, the database parameters are respectively configured for each second computing node to obtain a target parameter value group, wherein the target parameter value group includes target parameter values corresponding to each second computing node; the target parameter value group is synchronized to the first computing cluster, so as to, for each first computing node, configure the target parameter values corresponding to the first computing node in the target parameter value group to the first computing node. The above technical solution fully utilizes the characteristics of the compute-storage separation architecture, under the premise of ensuring that the normal operation of the existing computing cluster (i.e., the first computing cluster applied as a production cluster) is not affected, a second computing cluster specially used for database parameter configuration is added, the database parameters are respectively configured for each second computing node in the second computing cluster, and then the database parameters are synchronized to the first computing cluster, so that the resources on each first computing node can be fully utilized, thereby solving the problem of uneven resource utilization, which helps to improve the overall performance of the target database and has great significance for large-scale data analysis and processing.

[0060] The above technical solution also has at least the following technical effects:

[0061] 1. Improve operational efficiency

[0062] 1.1 Automatic database parameter optimization: The automatic database parameter optimization mechanism can significantly reduce the workload of manual parameter tuning, allowing database administrators to focus more on higher-level strategic planning and business innovation.

[0063] 1.2 Reduce downtime: By continuously optimizing database parameters, configurations can be adjusted in a timely manner to cope with different load situations without affecting normal business, reducing service interruptions caused by performance bottlenecks.

[0064] 2. Enhance customer satisfaction

[0065] 2.1 Improve user experience: Optimized database parameters can improve query response time and system throughput, thereby improving user experience and increasing user satisfaction and loyalty.

[0066] 2.2 Improve service quality: Continuous performance optimization helps maintain high-quality service levels, ensuring service stability and reliability even during peak business periods.

[0067] 3. Reduce operational costs

[0068] 3.1 Resource saving: In the storage-compute separation architecture, optimized database parameters can better utilize computing resources, reduce unnecessary resource waste, and reduce the cost of cloud computing.

[0069] 3.2 Reduce failure rate: Through automated monitoring and optimization, potential problems can be found in time and adjustments can be made in advance to reduce the probability of failure.

[0070] 4. Promote business growth

[0071] 4.1 Support business expansion: As the business grows, it can support larger data volumes and higher concurrent requests, helping businesses smoothly transition to higher business levels.

[0072] 4.2 Promote new business models: By improving the efficiency and accuracy of data analysis, businesses can more easily try new business models and service innovations.

[0073] 5. Compliance and security

[0074] 5.1 Data consistency guarantee: The storage-compute separation architecture ensures data consistency, helps comply with industry standards and regulatory requirements, and enhances data security and privacy protection.

[0075] Figure 3 is another flowchart of a database parameter configuration method provided by an embodiment of the application. This embodiment is optimized based on the above technical solutions. In this embodiment, the database parameters of each second computing node are optionally configured to obtain a target parameter value group, including: determining the value range of the database parameters, and determining the parameter space to be searched according to the value range; create a particle swarm, wherein the particles in the particle swarm represent candidate parameter value groups of the database parameters, and the candidate parameter value groups include candidate parameter values corresponding to each second computing node; control each particle to move in the parameter space for searching; after each movement, test the candidate parameter value groups represented by each particle after movement based on the fitness function, to determine the target parameter value group from each candidate parameter value group obtained after movement. Wherein, the explanations of the same or corresponding terms as in the above embodiments are not repeated here.

[0076] Referring to Figure 3 , the method of this embodiment can specifically include the following steps:

[0077] S210. Determine the first computing cluster for operating the target database and the second computing cluster for configuring the database parameters, wherein the first computing nodes in the first computing cluster and the second computing nodes in the second computing cluster have the same specifications.

[0078] S220. For the database parameters of the target database, determine the value range of the database parameters, and determine the parameter space to be searched according to the value range.

[0079] The value range can represent the safe and reasonable upper and lower bounds of the database parameters to be configured. By determining the upper and lower bounds, the parameter space can be determined according to the upper and lower bounds, so that the database parameters can be searched in the parameter space.

[0080] Here, the above example is taken as an example. Step two: parameter space definition

[0081] It is crucial to determine which database parameters need to be optimized. The following are some key database parameters and setting strategies for their value ranges.

[0082] (1) In-depth analysis of traditional parameters

[0083] For the buffer pool size: it is the size of the memory area used to cache data pages in the database. The buffer pool size directly affects the read and write speed of data. A small buffer pool may cause frequent disk I / O operations, reducing system performance; while a large buffer pool may occupy too much memory resources, affecting the running of other processes. Reasonable adjustment of the buffer pool size can significantly improve the read and write speed of data. The value range is set according to the actual load of the database and hardware resources. For example, for a medium-sized database system, the buffer pool size is adjusted between 128MB and 2GB. When determining the specific value range, the historical access pattern of the database can be analyzed to understand the average data access volume and peak access volume, so as to determine the appropriate buffer pool size range.

[0084] By analyzing the historical access pattern of the database, assuming that the average data access volume is , the peak access volume is , and the total hardware memory is , the initial value range of the buffer pool size can be set as , where and are adjustment coefficients, for example, , .

[0085] Considering the impact of hardware resources, the range can be further adjusted to , where and are coefficients determined according to hardware performance and database type.

[0086] For the upper limit of the number of connections: it is the maximum value of the number of clients allowed to connect to the database simultaneously. The upper limit of the number of connections determines the number of clients that can connect to the database simultaneously. Setting the upper limit of the number of connections too high may cause the system resources to be exhausted, affecting performance; while setting it too low may limit the system's concurrent processing capability. The value range of the upper limit of the number of connections needs to consider the expected load of the database and the hardware resources of the server. For an enterprise-level application, the upper limit of the number of connections is adjusted between 100 and 1000. By monitoring the system in real time to observe the usage of the number of connections, the value range of the upper limit of the number of connections is dynamically adjusted according to the actual demand.

[0087] Assuming the expected number of concurrent users is , the server hardware resource evaluation coefficient is , the value range of the upper limit of the number of connections can be set to , where and are adjustment coefficients, for example, , .

[0088] For the on and off state of query cache: query cache improves the performance of repeated queries, but may also bring additional memory overhead and cache consistency problems. The on and off state of query cache needs to be decided according to the specific application scenario. For scenarios with fixed query patterns and frequent repeated queries, query cache can be considered; while for scenarios with frequent query pattern changes and frequent data updates, turning off query cache may be more appropriate.

[0089] For memory usage strategy: it is the way the database stores and manages data in memory. Memory usage strategy will directly affect the way the database stores and manages data in memory. Different memory usage strategies are suitable for different application scenarios. For example, choose to store frequently accessed data in memory to improve access speed; or use memory paging mechanism to swap infrequently used data to disk to save memory resources. The value range of memory usage strategy is adjusted according to the load characteristics of the database and hardware resources. For example, set the maximum storage ratio of data in memory, the size of memory paging, etc.

[0090] (2) Expansion of parameter types

[0091] In addition to traditional parameters such as buffer pool size, upper limit of the number of connections, on and off state of query cache, and memory usage strategy, consider the storage engine parameters, index strategy, and query optimizer parameters of the database in depth and comprehensively.

[0092] For the storage engine, fine-tune its cache size, log level, data file storage mode, and other parameters. For example, adjusting the cache size reasonably can significantly improve the read and write speed of data. If the cache size is set too large, it may occupy too much memory resources, affecting the overall performance of the system; while setting too small, it may lead to frequent disk I / O operations, reducing data access efficiency. Optimizing the log level effectively reduces disk I / O operations, different log levels correspond to different write frequencies and data recovery capabilities, and need to be weighed according to the actual business needs. Selecting the appropriate data file storage mode improves the efficiency and reliability of storage, such as using distributed storage or redundant storage to cope with different failure scenarios.

[0093] Storage engine parameters: various configuration parameters of the database storage engine, such as cache size, log level, and data file storage mode. Different storage engine parameter settings will affect the performance and reliability of the database.

[0094] For indexes, carefully select different index types, such as B-tree indexes and hash indexes, and fine-tune parameters such as index fill factor and index coverage. Different index types are suitable for different query scenarios, B-tree indexes perform well in range queries, while hash indexes have advantages in equality queries. By reasonably selecting and adjusting index parameters, the query efficiency is greatly improved. For example, adjusting the fill factor of the index affects the storage efficiency and query performance of the index, a too high fill factor may cause index page splitting, reducing query speed; while a too low fill factor may waste storage space.

[0095] For the query optimizer, accurately adjust its optimization level, statistical information update strategy, and query rewriting rules, and other parameters. Through the fine optimization of these parameters, the performance of the database is significantly improved. For example, adjusting the optimization level allows the query optimizer to balance between performance and accuracy. A higher optimization level may spend more time on query optimization, but get a better query plan; while a lower optimization level can generate a query plan faster, but may not be optimal. The update strategy of statistical information is also very important, timely updating statistical information allows the query optimizer to make more accurate decisions, but too frequent updates may affect the performance of the system. Query rewriting rules allow the database to automatically optimize complex queries, improving the execution efficiency of queries.

[0096] (3) Establishment of parameter sensitivity analysis model

[0097] A scientific and rigorous parameter sensitivity analysis model is established, and a variety of advanced analysis methods are used, such as single factor analysis method and multi-factor analysis method. Next, the two analysis methods are illustrated.

[0098] Single-factor analysis allows for the analysis of the impact of a single parameter on database performance while keeping other parameters constant. This method determines the degree of independent influence of each parameter. For example, for the parameter of buffer pool size, its value can be gradually increased or decreased while keeping other parameters constant, and changes in performance indicators such as database query response time, throughput, and resource utilization can be observed. Through multiple experiments, the relationship curve between buffer pool size and performance indicators can be obtained, thereby determining the degree of impact of buffer pool size on database performance.

[0099] Regarding the buffer pool size parameter, let the buffer pool size be... The query response time is throughput is Resource utilization rate Establish functional relationships , , Through multiple experiments, the size of the buffer pool was changed. The value of , and observe , , The changes in [the buffer pool size] are used to determine the impact of the buffer pool size on database performance. Optionally, the above functional relationship is [formula missing]. , , .

[0100] Multifactor analysis considers the impact of changes in multiple parameters on database performance simultaneously. This method more realistically reflects the actual database operating environment because, in practical applications, multiple parameters often interact with each other. For example, using orthogonal experimental design, several key parameters are selected, and a series of experiments are designed, with different combinations of parameter values ​​in each experiment. By analyzing the results of these experiments, the interaction between the parameters and their overall impact on database performance can be obtained.

[0101] Using orthogonal experimental design, the following assumptions were selected. Key parameters Each parameter has Each value level. Design. There are several experiments, each with different combinations of parameter values. Let the database performance metric be... Functional relationships can be established. Analysis of these experimental results reveals the interactions between various parameters and their overall impact on database performance.

[0102] Through a large number of tests and simulations of different parameter combinations, the influence of each parameter on database performance is analyzed in depth. According to the analysis results, it is accurately determined which parameters are key parameters that need to be optimized, and which parameters have less impact on performance and can be appropriately ignored during optimization. At the same time, the mutual relationship between parameters is analyzed in depth to find out possible synergies or conflicts, so as to make reasonable adjustments during optimization. For example, the combination of some parameters may have a synergistic effect to improve the performance of the database, while some parameters may have conflicts that need to be weighed and optimized.

[0103] Step three: parameter encoding

[0104] Mapping the value range of each parameter to a continuous parameter space is to facilitate the subsequent particle swarm optimization algorithm search. For example, if the buffer pool size can be 16MB to 512MB, it can be linearly mapped to the [0, 1] interval.

[0105] Different mapping methods are used for different types of parameters. For buffer pool size, linear mapping, logarithmic mapping, or exponential mapping methods can be selected according to the actual value range and data distribution characteristics. For example, if the buffer pool size value range is [16MB, 512MB] and the data distribution is relatively uniform, linear mapping can be used to map it to the [0, 1] interval, and the mapping formula is: where is the original buffer pool size value, is the mapped value.

[0106] For integer type parameters such as the maximum number of connections, a discretization mapping method can be used to map them to an integer interval. Let the original maximum number of connections be with a value range of . The discretization mapping formula can be set as where is the mapped integer value, is the size of the discretized integer interval. For example, , then the mapped maximum number of connections value range is [0, 99] integer interval.

[0107] For Boolean type parameters such as the on and off states of query caching, a binary encoding method is used to map them to the [0, 1] interval, with the on state set to 1 and the off state set to 0. The binary encoding formula is where is the original Boolean value, is the mapped [0, 1] interval value.

[0108] S230. Create a particle swarm, wherein the particle swarm represents the candidate parameter value group of the database parameters, and the candidate parameter value group includes the candidate parameter values ​​corresponding to each second computing node.

[0109] In this embodiment of the invention, optionally, there are multiple database parameters. After determining the parameter space to be searched based on the value range, the database parameter configuration method further includes: dividing the parameter space into multiple parameter levels, and for each parameter level, configuring the database parameters corresponding to that parameter level within the multiple database parameters. The multiple parameter levels include at least two of storage layer parameters, computation layer parameters, and network layer parameters. This division of the parameter space helps improve the accuracy of database parameter configuration.

[0110] For example, step four: particle swarm initialization

[0111] When initializing particles, a hierarchical initialization approach is adopted, dividing the parameter space into different levels, with each level corresponding to a set of related database parameters. For example, the parameter space is divided into storage layer parameters, computation layer parameters, and network layer parameters.

[0112] Storage layer parameters may include buffer pool size and storage engine cache size. When initializing these parameters, a reasonable initial range should be determined based on the database's historical data and expected load. For example, if the database typically handles a large number of small transactions, the initial buffer pool size should be set relatively small to avoid wasting memory resources; if the database mainly handles large and complex queries, a larger buffer pool may be needed to improve the cache hit rate. A reasonable initial range for storage layer parameters can be determined through analysis of historical query patterns and data access frequency.

[0113] Assuming the database typically handles a number of small transactions. The number of large transactions is The total number of transactions is Regarding the initial range setting of the buffer pool size, if the primary focus is on handling a large number of small transactions, the average data volume of each small transaction can be set to [value missing]. The initial range of the buffer pool size can then be set to... ,in and To adjust the coefficients, for example, , .

[0114] If the primary focus is on handling large and complex queries, let the average data volume of a large transaction be... The initial range of the buffer pool size can then be set to... ,in and To adjust the coefficients, for example, , .

[0115] The computation layer parameters include the connection limit and the query optimizer's optimization level. The initialization of the connection limit takes into account the server's hardware resources and the expected number of concurrent users. If the server has strong hardware performance and a large number of concurrent users are expected, the initial value of the connection limit should be set relatively high. The query optimizer's optimization level is determined based on the database query complexity and response time requirements. If the database query is complex and has high response time requirements, then the optimization level should be initialized to a higher value to obtain a better query plan.

[0116] For the initialization of the connection limit, let the server's hardware performance evaluation coefficient be... The expected number of concurrent users is The initial value for the maximum number of connections can be set to... ,in and To adjust the coefficients, for example, , .

[0117] Regarding the query optimizer's optimization level, assuming the database query complexity is and the response time requirement is , a functional relationship can be established. ,in This refers to the optimization level of the query optimizer. For example, the optimization level can be determined by a weighted sum of query complexity and response time requirements. ,in and These are weighting coefficients, which can be adjusted according to actual circumstances.

[0118] Network layer parameters include network bandwidth allocation and data transmission compression strategies. When initializing network layer parameters, the network environment where the database resides and the data transmission requirements should be considered. If network bandwidth is limited, the data transmission compression strategy should be adjusted appropriately to reduce network transmission overhead. Simultaneously, based on the database's distributed architecture and data synchronization needs, network bandwidth can be allocated rationally to ensure efficient data transmission.

[0119] Regarding network bandwidth allocation, let the total bandwidth of the network where the database resides be... Data transmission requirements are The initial value for network bandwidth allocation can be set to... ,in and The adjustment factor can be determined based on data transmission requirements and network load. For example, , If the data transmission demand is large, the values of and may be appropriately increased.

[0120] For the compression strategy of data transmission, let the compression ratio be , the data volume be , and the compressed data volume be . The initial value of the compression ratio can be determined according to the relationship between the data volume and the network bandwidth. For example, if the network bandwidth is limited and the data volume is large, a higher compression ratio, such as , can be selected to reduce the network transmission overhead.

[0121] Random initialization is performed at each level to ensure that the exploration range of particles at different levels is more reasonable. At the same time, in order to improve the efficiency and quality of initialization, some heuristic methods are flexibly used, such as experience-based initialization, history data-based initialization, etc. For example, according to the past optimization experience, some key parameters are initialized in a targeted manner to speed up the convergence speed of the algorithm. In combination with the characteristics of the database and the application scenario, targeted initialization is performed to make the particles closer to the direction of the optimal solution in the initialization stage.

[0122] A particle swarm is created, and each particle corresponds to a candidate parameter value group, that is, each particle represents a possible solution of the database parameters. After initialization, these particles are distributed at different positions in the parameter space, covering various possible parameter combinations. Through the evolution and search process of the particle swarm, the optimal database parameters are gradually found.

[0123] Diversity measurement indicators such as the distance between particles and the variance of fitness values are introduced to ensure that the initialized particle swarm has sufficient diversity.

[0124] The distance between particles is measured by calculating the difference between the parameters represented by each particle. For example, the Euclidean distance or Manhattan distance measurement method is used. If the distance between particles is small, it indicates that the diversity of the particle swarm is insufficient, which may cause the algorithm to fall into a local optimal solution. In this case, the diversity is increased by adding random disturbance or reinitializing part of the particles. Specifically, a certain degree of random noise is added to the position or speed of the particles, or a part of the particles are randomly selected for reinitialization.

[0125] The variance of the fitness value reflects the distribution of the particle swarm on the fitness function. If the variance is small, it indicates that the fitness values of the particle swarm are concentrated, and the diversity may be insufficient. The diversity of the particle swarm is increased by adjusting the initialization strategy, such as expanding the value range of the parameters or increasing the number of random initialization.

[0126] Meanwhile, the diversity of the particle swarm is closely monitored, and the initialization strategy is adjusted in time according to the need to maintain the diversity of the particle swarm. For example, a diversity threshold is set, and when the diversity of the particle swarm is lower than the threshold, an adjustment mechanism is automatically triggered to increase random disturbance or reinitialize part of the particles to improve the diversity of the particle swarm. The distance between particles and the variance of the fitness value are calculated regularly, and whether the diversity of the particle swarm is sufficient is determined according to these indicators. If the diversity is found to be insufficient, appropriate measures are taken to adjust it to ensure that the algorithm can fully explore the parameter space during the search process and improve the possibility of finding the global optimal solution.

[0127] S240. Controlling each particle to move in the parameter space for searching.

[0128] Among them, each particle is controlled to move in the parameter space, so as to search for a particle that can make the performance of the target database best.

[0129] For example, step five: velocity initialization

[0130] Each particle is assigned a velocity vector to determine how it moves in the parameter space, that is, the velocity vector determines the direction and distance of the particle moving in the search space.

[0131] The velocity vector is usually set to a small random value at initialization to avoid starting to deviate in a certain direction, that is, to avoid locking in some local optimal solution too early.

[0132] At initialization, the initial positions of the particles are randomly distributed in the parameter space to ensure that the particles can explore different regions of the entire parameter space. The parameter values of each particle are randomly selected from the value range of the respective parameter.

[0133] The velocity vector is usually set to a small random value at initialization to avoid starting to deviate in a certain direction. For parameters with a value range in the [0, 1] interval, the initial value range of the velocity vector can be set to [-0.1, 0.1]. The random number generator is used to generate the initial velocity vector to ensure that each particle's velocity vector has a certain randomness at the beginning, while not being too large or too small.

[0134] S250. After each movement, the candidate parameter value group represented by each particle after movement is tested based on the fitness function to determine the target parameter value group from the candidate parameter value groups obtained after movement, wherein the target parameter value group includes the target parameter values corresponding to each second computing node.

[0135] Among them, during the movement of the particle in the parameter space, that is, during the search of the particle, the influence of the corresponding candidate parameter value group on the performance of the target database can be determined based on the fitness function, and then the target parameter value group is obtained.

[0136] S260. Synchronizing the target parameter value group to the first computing cluster to configure, for each first computing node, the target parameter value corresponding to the first computing node in the target parameter value group to the first computing node.

[0137] Exemplarily, step fourteen: verification and deployment

[0138] Verify the optimal parameter found in computing cluster B and synchronize the optimal parameter to computing cluster A.

[0139] The technical scheme of the embodiment of the application can efficiently find an optimal solution in a complex parameter space by using a particle swarm optimization algorithm, thereby improving the accuracy and applicability of database parameter optimization, and being particularly suitable for processing large-scale and variable cluster environments and having good universality.

[0140] On this basis, an optional technical scheme is that the fitness function is constructed based on multiple performance indicators, and the performance indicators are indicators representing the performance of the target database, and the following steps can be performed to determine the target parameter value group:

[0141] Obtain the query statement synchronized from the first computing cluster, wherein the query statement is a statement executed by the first computing cluster to query the stored data in the target database, and the stored data is data shared by the first computing cluster and the second computing cluster;

[0142] After each movement, for each particle after movement, execute the query statement under the candidate parameter value group represented by the particle after movement to obtain a second indicator value of the performance indicator, and substitute the second indicator values of the multiple performance indicators into the fitness function to obtain a function value of the particle after movement;

[0143] According to all function values, determine the target parameter value group from the candidate parameter value groups obtained after movement.

[0144] The above technical scheme, in combination with the characteristics of the storage-computing separation architecture, the two computing clusters can share the same stored data, so that no additional data synchronization is required, and only the query statement on the first computing cluster needs to be synchronized and executed on the second computing cluster, that is, the performance of different candidate parameter value groups can be evaluated according to the performance indicators of the actual business scenario, and then the target parameter value group is determined.

[0145] Exemplarily, step six: fitness function creation

[0146] All query requests (i.e., query statements) in the computing cluster A are collected for creating a fitness function to evaluate the performance of the database under given parameters. Metrics can be selected as the evaluation results. Factors that can be considered by the fitness function include, but are not limited to, query response time, throughput, resource utilization, and concurrency, etc.

[0147] The following are the collection and calculation methods of specific metrics and the construction method of the fitness function:

[0148] Query response time: Insert some test data in the computing cluster A, and record the response time of each query request, then calculate the average response time as the query response time metric. Let the total number of query requests be , the response time of each query request be , and the average response time be .

[0149] Throughput: Calculate the throughput by counting the number of query requests processed per unit of time. Let the unit of time be , the number of query requests processed be , and the throughput be .

[0150] Resource utilization: Calculate the resource utilization by monitoring the usage of CPU, memory, disk, and other resources of the database server. Let the CPU usage rate of the database server be , the memory usage rate be , and the disk usage rate be , and the resource utilization be , where , , are weight coefficients that can be adjusted according to actual conditions. For example, , , .

[0151] Concurrency: Calculate the concurrency by counting the number of query requests processed simultaneously. Let the maximum value of the number of query requests processed simultaneously be , and the minimum value be , and the concurrency be .

[0152] The fitness function adopts a weighted summation method to consider these factors comprehensively. According to different application scenarios and requirements, a weight coefficient is assigned to each metric, and the fitness function can be represented as:

[0153] ;

[0154] where The value of the fitness function, The query response time, The throughput, The resource utilization, The concurrency, , , , The weight coefficients of each index, respectively, can be adjusted according to different application scenarios and requirements. For example, for scenarios with high real-time requirements, the weight of the query response time can be set higher; for large-scale data processing scenarios, the weights of the throughput and resource utilization may be larger. , , , , , , .

[0155] Step seven: Particle performance evaluation

[0156] Testing each particle represents a candidate parameter value group is a key step to evaluate the performance of the particle. By using the fitness function to calculate its performance score (i.e. the function value described above), while introducing multi-index evaluation method and establishing performance evaluation benchmark model, to evaluate the performance of the particle comprehensively and accurately.

[0157] (1) Fitness function combined with multi-index

[0158] When calculating the performance score using the fitness function, it is no longer dependent on a single index, but combines multiple indexes such as query response time, throughput, resource utilization, and concurrency. For example, design a fitness function to weight and sum these indexes, where the weight of each index is determined according to the actual application scenario and requirement. If it is a scenario with high real-time requirements, the weight of the query response time is set higher; while for large-scale data processing scenarios, the weights of the throughput and resource utilization may be larger.

[0159] When calculating the fitness function, first standardize each index to make it comparable. Then, according to the importance of the index, assign the corresponding weight, and finally multiply the value of each index by the weight and add it up to get the fitness score of the particle.

[0160] (2) Comprehensive use of multi-index evaluation method

[0161] In addition to using the fitness function to calculate the performance score, introduce multi-index evaluation methods such as Principal Component Analysis (PCA) or Analytic Hierarchy Process (AHP). ​

[0162] When using PCA, first standardize the multiple performance indicators to eliminate the dimensional differences between the indicators. For query response time, throughput, resource utilization, concurrency, etc. indicators, the Z-score standardization method can be used to convert the value of each indicator to a standard value with a mean of 0 and a variance of 1. Then calculate the covariance matrix between the indicators, and get the principal components by eigenvalue decomposition. Principal components are linear combinations of original indicators, which can reflect the main characteristics of the data. Reduce the query response time, throughput, resource utilization, concurrency, etc. indicators to a comprehensive indicator, reduce the correlation between the indicators, and improve the accuracy of the evaluation. For example, through PCA analysis, it may be found that a principal component mainly reflects the comprehensive performance of query response time and throughput, while another principal component mainly reflects the comprehensive performance of resource utilization and concurrency. In this way, a more comprehensive understanding of the performance of the parameters represented by the particles in different aspects is obtained.

[0163] Let the matrix composed of query response time, throughput, resource utilization, concurrency, etc. indicators be , the dimension of , where is the number of samples, is the number of indicators. First calculate the covariance matrix .

[0164] Eigenvalue decomposition is performed on the covariance matrix to obtain the eigenvalues and the corresponding eigenvectors .

[0165] The principal components are linear combinations of the original indicators, that is , where is a matrix composed of eigenvectors, is the principal component matrix after dimensionality reduction.

[0166] When using AHP, a hierarchical model is established, and the target layer, criterion layer, and scheme layer are clearly divided into levels. The target layer is to evaluate the performance of particles, the criterion layer includes query response time, throughput, resource utilization, concurrency, and other indicators, and the scheme layer is the database parameters represented by each particle. The relative importance of each indicator is determined by pairwise comparison, for example, using the 1-9 scale method, 1 indicates that the two indicators are equally important, and 9 indicates that one indicator is extremely important to another indicator. The weight vector is calculated, and finally the performance evaluation results of the particles are obtained by synthesizing the scores of each indicator. At the same time, combined with the actual application scenario, different weights are given to different indicators to better reflect the performance of the database in a specific scenario. For example, in an application scenario with high real-time requirements, a higher weight is given to query response time; while in a large-scale data processing scenario, throughput and resource utilization may be more important.

[0167] A hierarchical model is established, the target layer is to evaluate the performance of particles, the criterion layer can include query response time, throughput, resource utilization, and concurrency, and the scheme layer is the database parameters represented by each particle.

[0168] The relative importance of each indicator is determined by pairwise comparison, and the judgment matrix is set as , The element of the judgment matrix indicates the importance of the indicator relative to the indicator, and the 1-9 scale method is used for comparison.

[0169] The weight vector is calculated, first the product of each row element of the judgment matrix is calculated , then the power of is calculated , and finally the weight vector is obtained by normalizing .

[0170] (3) Establish a performance evaluation benchmark model

[0171] A benchmark model for performance evaluation is established, and some open-source database performance testing tools are selected to conduct comprehensive benchmark testing on different parameters.

[0172] When using database performance testing tools, simulate different types of database loads, such as online transaction processing (OLTP) and on-line analysis processing (OLAP), etc., to test the performance of the target database under different loads. By adjusting test parameters such as the number of concurrent users, transaction size and data volume, simulate different actual application scenarios. For testing, simulate a complex online transaction processing environment to test the performance of the target database under high-concurrency transaction scenarios.

[0173] At the same time, widely collect and analyze performance data of the same type of database system as a reference benchmark. Through channels such as database performance reports, academic research papers and industry forums, obtain performance data of the same type of database under different parameters. By comparing with known excellent database configurations, evaluate the relative performance of the current parameters. In the evaluation process, not only consider the performance of a single indicator, but also consider the balance of multiple indicators and the overall performance improvement. For example, use weighted average method or comprehensive evaluation method to comprehensively evaluate multiple indicators to get more comprehensive and accurate performance evaluation results.

[0174] If the performance of a parameter represented by a particle is better than that of known excellent configurations in multiple indicators, it is considered to have high performance potential. Conversely, if it performs poorly in multiple indicators, it may need to be adjusted or re-searched.

[0175] Step eight: individual extreme value update

[0176] For each particle, record the best parameter it finds (i.e. the highest fitness score). In this process, dynamic updating strategy can be introduced and the influence of historical optimal solution can be considered to better achieve the update of individual extreme value.

[0177] (1) Detailed implementation of dynamic updating strategy

[0178] The dynamic updating strategy is introduced to dynamically adjust the updating frequency of individual extreme value according to the number of iterations or performance improvement. In the early stage of iteration, since the position and speed of particles change greatly, frequent updating of individual extreme value encourages particles to quickly explore different parameters. For example, set the first 20% stage of iteration, update the individual extreme value every time the iteration is completed, to encourage particles to search widely in the parameter space. At this time, if the particle finds a new parameter with higher fitness score, it is recorded as the current best parameter.

[0179] As the number of iterations increases, the particles gradually converge, at which point the update frequency is reduced to ensure that the particles converge more stably to the global optimal solution. For example, in the middle stage of iteration, update the individual extreme value every 3-5 iterations to allow the particles to perform local search within a certain range and avoid excessive oscillation. In this stage, if the fitness score of the particle improves and exceeds the fitness score of the previously recorded best parameters, update the record.

[0180] In the later stage of iteration, when the particles approach convergence, the update frequency is further reduced, for example, updating the individual extreme value every 10 iterations to ensure that the particles can stably move towards the optimal solution. Similarly, only when the new parameters found by the particle have a significantly higher fitness score, the record of the individual extreme value is updated.

[0181] Set an update frequency adjustment function to automatically adjust the update frequency according to the number of iterations or the magnitude of performance improvement. For example, use an exponential decay function, and as the number of iterations increases, the update frequency gradually decreases. Assuming the number of iterations is t, the maximum number of iterations is T, and the initial update frequency is f0, then the update frequency f (t) can be expressed as f (t)=f0exp(-kt / T), where k is an adjustment parameter that is adjusted according to actual conditions.

[0182] (2) Influence and integration of historical optimal solution

[0183] Consider the influence of the historical optimal solution when updating the individual extreme value, not only consider the best parameters of the current iteration, but also consider the historical optimal solution. By setting a weight coefficient, the historical optimal solution and the current best parameters are weighted and summed to obtain the updated individual extreme value.

[0184] In the early stage of iteration, give a larger weight to the current best parameters, because at this time the particles are still exploring the parameter space extensively, and the reference value of the historical optimal solution is relatively small. For example, set the weight of the current best parameters to 0.7 and the weight of the historical optimal solution to 0.3. If the fitness score of the best parameters found by the current particle is high, the updated individual extreme value will be affected by a larger weight. As the number of iterations increases, gradually increase the weight of the historical optimal solution to fully utilize historical information and improve the convergence speed and accuracy of the algorithm. For example, in the middle stage of iteration, adjust the weights of the current best parameters and the historical optimal solution to 0.5 each. At this time, if the historical optimal solution performs well in some aspects, it will have a greater impact on the update of the individual extreme value.

[0185] In the later stage of iteration, when the particles approach convergence, further increase the weight of the historical optimal solution, such as setting it to 0.8 and the weight of the current best parameters to 0.2. In this stage, the stability and reliability of the historical optimal solution are higher, and it plays a leading role in the update of the individual extreme value.

[0186] The weight coefficient is dynamically adjusted according to the number of iterations or performance improvement. For example, at the beginning of iteration, when the fitness value of the particle changes greatly, it indicates that the particle is actively exploring, and at this time, a higher weight is maintained for the current best parameter. As the iteration proceeds, if the fitness value of the particle gradually decreases, it indicates that the particle begins to converge, and at this time, the weight of the historical optimal solution is gradually increased.

[0187] At the same time, a storage and updating mechanism of historical optimal solution is set to update the historical optimal solution regularly to ensure that it always reflects the best performance of the algorithm. For example, after a certain number of iterations, the current global optimal solution is updated as the historical optimal solution. Or when the fitness value of the particle is significantly improved, the historical optimal solution is also updated. If the fitness score of the best parameter found by the current particle is much higher than the historical optimal solution, the record of the historical optimal solution is updated in time.

[0188] Step nine: global extreme value update

[0189] The configuration with the highest fitness score among all particles is determined as the global best. In this process, a distributed updating strategy is adopted and a stability judgment mechanism is introduced to better achieve the update and determination of the global extreme value.

[0190] (1) Specific implementation of distributed updating strategy

[0191] The distributed updating strategy is adopted to divide the particle swarm into multiple subgroups, and each subgroup independently updates the global extreme value. Using the distributed computing framework, the updating task of the particle swarm is distributed to multiple computing nodes to improve the parallelism and efficiency of the algorithm.

[0192] First, the entire particle swarm is divided into several subgroups according to the characteristics of the particles or random division method. For example, according to the type of parameters represented by the particles, or according to certain random rules to allocate particles to different subgroups.

[0193] In each subgroup, the local optimal solution can be used as the global extreme value of the subgroup. The particles in the subgroup constantly update the best parameters they find, i.e. the local optimal solution, in the search process. Through communication and cooperation with other subgroups, the global extreme value is constantly updated. For example, a regular communication mechanism is set up to exchange information of local optimal solutions among subgroups. After receiving the local optimal solution of other subgroups, each subgroup compares it with its own local optimal solution. If the local optimal solution of other subgroups is better, the global extreme value of its own subgroup is updated to the better local optimal solution.

[0194] Then, the global extreme values of each subgroup are merged and updated through a coordination mechanism to obtain the final global extreme value. The coordination mechanism adopts a centralized or distributed manner, which is selected according to the actual situation. For example, a centralized coordination server is used to collect the global extreme values of each subgroup, and comprehensive analysis and updating are performed. The coordination server calculates the average value, median value or weighted sum according to certain weights of the global extreme values of each subgroup to obtain a new global extreme value. A distributed coordination mechanism is also used to update the global extreme value through communication and negotiation among subgroups. For example, each subgroup determines a new global extreme value through voting or iteratively updates the global extreme value according to certain rules.

[0195] In this process, the global extreme values of each subgroup and the fitness scores of all particles are compared to determine the configuration with the highest fitness score. Once a configuration with a higher fitness score is found, it is taken as a new global best candidate.

[0196] Let the number of subgroups be , the local optimal solution of the i-th subgroup be , and the global extreme value be . In each subgroup, the particles in the subgroup constantly update the best parameters they find, i.e., the local optimal solution, during the search process. Through communication and cooperation with other subgroups, the global extreme value is constantly updated.

[0197] For example, a regular communication mechanism is set up to exchange information about the local optimal solutions among subgroups. After receiving the local optimal solutions of other subgroups, each subgroup compares its own local optimal solution with the received local optimal solutions. If the local optimal solution of another subgroup is better, the global extreme value of the subgroup is updated to the better local optimal solution. That is, .

[0198] A centralized coordination server is used to collect the global extreme values of each subgroup, and comprehensive analysis and updating are performed. The coordination server can calculate the average value, median value or weighted sum according to certain weights of the global extreme values of each subgroup to obtain a new global extreme value.

[0199] Let the global extreme value of each subgroup be , and the weight vector be . Then the new global extreme value is .

[0200] A distributed coordination mechanism can also be used to update the global extreme value through communication and negotiation among subgroups. For example, each subgroup determines a new global extreme value through voting or iteratively updates the global extreme value according to certain rules.

[0201] (2) Introduction of stability judgment mechanism

[0202] A stability judgment mechanism is introduced to determine whether the global extremum has changed significantly in consecutive iterations. If the global extremum has not changed significantly in consecutive iterations, it is considered that the algorithm has converged to a better solution. Specifically, a stability threshold is set. If the change in the global extremum is less than the threshold, the stability judgment mechanism is triggered.

[0203] For example, the change in the global extremum in consecutive iterations, such as the difference in fitness scores, is calculated. If the difference is less than the preset stability threshold, it is considered that the algorithm may have converged. The stability threshold can be determined by statistical analysis, such as historical data or through multiple experiments.

[0204] If it is determined that the algorithm has converged, a local search is performed to search for a better solution in the vicinity of the global extremum. Some local optimization algorithms, such as gradient descent and Newton's method, are used to search in a small range around the global extremum. Alternatively, a certain degree of random disturbance is added to make the particle jump out of the local optimal solution and continue to explore a wider parameter space. For example, a certain random noise is added to the position or velocity of the particle based on the global extremum to allow the particle to continue searching in a new position.

[0205] At the same time, the stability of the global extremum is continuously monitored during the determination of the global optimum. If the global extremum is not stable during the search process, i.e., the change is large, the search and update need to be continued until the stability condition is met or other stopping criteria are met.

[0206] A maximum number of iterations is set. When the maximum number of iterations is reached, the iteration is stopped even if the algorithm has not converged, to avoid excessive calculation. The maximum number of iterations is set according to the complexity of the actual problem and the limitation of the computing resources. If the algorithm has not converged when the maximum number of iterations is reached, the parameters are adjusted or the algorithm is re-run.

[0207] Let the fitness scores of the global extremum in consecutive iterations be , respectively. The change is .

[0208] If the difference is less than the preset stability threshold , it is considered that the algorithm may have converged. The stability threshold can be determined by statistical analysis, such as historical data or through multiple experiments.

[0209] Step 10: Update the velocity and position using the PSO formula

[0210] The velocity and position of each particle are adjusted using the velocity update formula and position update formula of PSO. Since the parameters can be discontinuous or have specific ranges, appropriate adjustments need to be made:

[0211] ;

[0212] where, is the velocity of particle at the th iteration, and is the learning factor, usually set according to experience, such as = = 2. and are random numbers within the interval [0, 1], which can be generated by a random number generator. is the individual extreme value of particle , i.e., the best position found by particle during the search process. is the global extreme value, i.e., the best position found by the entire particle swarm during the search process. is the position of particle at the th iteration.

[0213] For position normalization, take the parameter with the value range [a, b] as an example. If the updated position exceeds this range, the following normalization method can be used:

[0214] If , then ; if , then . This ensures that the updated position is always within the valid range.

[0215] Step eleven: boundary handling

[0216] If the updated position exceeds the valid range, boundary handling techniques such as reflection, folding back, or contraction need to be used. The specific processing methods are as follows:

[0217] Reflection technique: if the updated position exceeds a small part of the valid range, use the reflection technique. Take the part that exceeds the range and add the boundary value of the valid range after taking the inverse, to get the reflected position. For example, for a parameter with a value range in the interval [0, 1], if the updated position is 1.2, the reflected position is .

[0218] Foldback technique: If the range exceeds significantly, foldback technique is adopted. The part exceeding the range is subtracted by the width of the valid range to get the folded-back position. For example, for a parameter with value range in [0, 1], if the updated position is 1.5, the folded-back position is .

[0219] Shrinking technique: If the updated position frequently exceeds the valid range, shrinking technique is adopted. The search space is gradually reduced to avoid particles exceeding the range again. The range of the search space is reduced by a certain proportion, for example, to 0.8 times of the original, and then the positions and velocities of particles are reinitialized.

[0220] Step twelve: Iterative optimization

[0221] Repeat steps seven to eleven until the stopping condition is met, such as reaching the maximum number of iterations or the fitness no longer significantly improves.

[0222] Step thirteen: Optimal configuration (i.e., optimal database parameter) selection

[0223] Determine the configuration with the highest fitness score among all particles as the global best. In this process, a distributed update strategy is adopted and a stability judgment mechanism is introduced to better achieve the update and determination of the global extreme value.

[0224] (1) Specific implementation of distributed update strategy

[0225] The distributed update strategy is adopted to divide the particle swarm into multiple subgroups, and each subgroup independently updates the global extreme value. Distributed computing framework can be used to distribute the update tasks of the particle swarm to multiple computing nodes, improving the parallelism and efficiency of the algorithm.

[0226] First, the entire particle swarm is divided into several subgroups according to the characteristics of the particles or random division methods. For example, it can be divided according to the type of parameters represented by the particles, or the particles can be allocated to different subgroups according to certain random rules.

[0227] In each subgroup, the local optimal solution is used as the global extreme value of the subgroup. The particles in the subgroup constantly update their best parameters found in the search process, i.e., the local optimal solution. Through communication and cooperation with other subgroups, the global extreme value is constantly updated. For example, a regular communication mechanism can be set up to exchange information about the local optimal solutions between subgroups. After receiving the local optimal solutions of other subgroups, each subgroup compares its own local optimal solution with the received local optimal solutions. If the local optimal solution of another subgroup is better, the global extreme value of the subgroup is updated to the better local optimal solution.

[0228] Then, through a coordination mechanism, the global extreme values of each subgroup are merged and updated to obtain the final global extreme value. The coordination mechanism can be centralized or distributed, and can be selected according to actual conditions. For example, a centralized coordination server can be used to collect the global extreme values of each subgroup, and perform comprehensive analysis and update. The coordination server can calculate the average value, median value or weighted sum according to certain weights of the global extreme values of each subgroup to obtain the new global extreme value. A distributed coordination mechanism can also be used to update the global extreme value through communication and negotiation between subgroups. For example, each subgroup can determine the new global extreme value through voting or iterative update according to certain rules.

[0229] In this process, the global extreme values of each subgroup and the fitness scores of all particles are continuously compared to determine the configuration with the highest fitness score. Once a configuration with a higher fitness score is found, it is taken as the new global best candidate.

[0230] (2) Introduction of stability judgment mechanism

[0231] A stability judgment mechanism for the global extreme value is introduced. When the global extreme value does not change significantly in continuous multiple iterations, it can be considered that the algorithm has converged to a better solution. Specifically, a stability threshold can be set, and when the change amplitude of the global extreme value is less than the threshold, the stability judgment mechanism is triggered.

[0232] For example, the change amount of the global extreme value in continuous iterations, such as the difference in fitness score, can be calculated. If this difference is less than the preset stability threshold, it is considered that the algorithm may have converged. The size of the stability threshold can be determined by statistical analysis methods, such as historical data experience values or through multiple experiments for adjustment.

[0233] If it is judged that the algorithm has converged, a local search can be performed to search for a better solution in the vicinity of the global extreme value. Some local optimization algorithms, such as gradient descent method, Newton method, etc., can be used to search in a small range around the global extreme value. Or a certain degree of random disturbance can be added to make the particle jump out of the local optimal solution and continue to explore a wider parameter space. For example, a certain random noise can be added to the position or velocity of the particle based on the global extreme value to make the particle continue to search in a new position.

[0234] At the same time, the stability of the global extreme value is continuously monitored during the determination of the global best. If the global extreme value is not stable during the search process, i.e. the change amplitude is large, the search and update need to be continued until the stability condition is met or other stopping criteria are met.

[0235] A maximum number of iterations can be set, and when the maximum number of iterations is reached, the iteration is stopped even if the algorithm has not converged, to avoid excessive calculation. The maximum number of iterations can be set according to the complexity of the actual problem and the limitation of the computing resources. If the algorithm has not converged when the maximum number of iterations is reached, the parameters can be adjusted or the algorithm can be re-run.

[0236] In another optional technical solution, the fitness function is constructed based on a plurality of performance indicators, and the performance indicators are indicators representing the performance of the target database. After the target parameter value group is synchronized to the first computing cluster, the above-mentioned database parameter configuration method can further include: monitoring the change of the first indicator value corresponding to each performance indicator of the first computing cluster; in the case that it is determined that the database parameters need to be reconfigured according to the monitored changes, reconfiguring the database parameters according to the changes and at least part of the plurality of performance indicators to reconfigure the database parameters according to the reconfigured fitness function. On this basis, optionally, the performance indicators include at least one of the following: query response time, throughput, resource utilization, and concurrency.

[0237] Here, the above example is continued as an example. Step fifteen: continuous iteration optimization

[0238] The computing cluster B can construct a new fitness function according to the change of the demand of the computing cluster A at any time, and continuously iteratively optimize, and the specific method is as follows:

[0239] Perceive demand changes: the demand changes are perceived by monitoring the change of the first indicator value of the performance indicators such as the number of query requests, response time and resource utilization of the computing cluster A. If these first indicator values change greatly, it is considered that the demand of the computing cluster A has changed.

[0240] Generate a new fitness function: when the demand change is perceived, the weight coefficients in the fitness function can be adjusted according to the change, or new performance indicators can be added. For example, if the number of query requests increases greatly, the weight coefficients of the throughput and concurrency can be appropriately increased; if the resource utilization is too high, a resource limit indicator such as maximum CPU usage can be added to the fitness function for comprehensive consideration.

[0241] The above example can realize real-time optimization and synchronization of database parameters by using the dynamic parameter adjustment mechanism, realize optimal utilization of computing resources, and further continuously improve the effect of system performance.

[0242] On this basis, in order to better understand the above-mentioned various technical solutions as a whole, the following two examples of optimizing an analytical database of a call charge processing system by using a particle swarm optimization algorithm are exemplarily described.

[0243] Example one, continue to see Figure 1 The implementation process is as follows:

[0244] This example focuses on a database system with specific configuration and use scenarios, that is, a computing cluster containing 3 hosts, using ARM architecture and CentOS operating system, configured as 4-core 16GB memory, mainly used for business scenarios of processing 10+ table association queries. The goal is to optimize database parameters to improve the speed of multi-table association query and the efficiency of memory use, so as to ensure the efficient operation of the system and the rational use of resources.

[0245] Step one: preparation

[0246] Create computing cluster A and computing cluster B in the same availability zone, both with the same specifications, containing 3 hosts using ARM architecture and CentOS operating system, configured as 4-core 16GB memory. Computing cluster A is used for the user's main database operation, and computing cluster B is used for optimizing database parameters. Through the automated deployment tool, ensure that the creation process is efficient, consistent and repeatable, and record the network settings, storage allocation and software installation version information during the deployment process in detail, which provides the basis for subsequent troubleshooting and optimization. Conduct comprehensive performance testing and benchmarking on each host, covering CPU performance, memory read / write speed, disk I / O, etc. Use professional tools to obtain accurate performance index data and establish a performance archive, record the performance, advantages and disadvantages under different loads. In addition, establish a powerful monitoring system to monitor the running state of the two computing clusters in real time, including CPU usage, memory usage, disk space, network traffic, and database connection number, query execution time and transaction processing speed, etc. Create a visual monitoring interface, set reasonable threshold and alarm mechanism to discover problems and handle them in time, and establish historical record and analysis function of monitoring data for trend prediction and preventive maintenance.

[0247] Step two: parameter space definition

[0248] Identify the list of database parameters that need to be optimized for this particular cluster, focusing on memory usage and I / O performance. For example, Buffer Pool Size ranges from 16MB to 128MB, determine the appropriate range by analyzing historical access patterns to avoid excessive disk I / O due to small size or excessive memory usage due to large size. Connection Pool Size ranges from 100 to 500, consider expected load and hardware resources. Also include Import Batch Size adjusted according to memory capacity and gp_vmem_protect_limit (16MB-2GB) and other parameters. Establish a parameter sensitivity analysis model, use single-factor analysis method to gradually adjust the value of a single parameter (such as Buffer Pool Size) and observe the impact on query response time, throughput and resource utilization and other performance indicators to determine its independent influence; use multi-factor analysis method (such as orthogonal test design), select key parameters for combined experiments, analyze the interaction between parameters and the comprehensive impact on database performance, and determine the key parameters and the relationship between parameters.

[0249] Step three: parameter encoding

[0250] Map the value range of each parameter to a continuous search space to facilitate subsequent particle swarm optimization search. For example, linearly map Buffer Pool Size from 16MB to 128MB to the interval [0, 1], for integer type parameters, you can use discrete mapping, and for boolean type parameters, you can use binary encoding.

[0251] Step four: particle swarm initialization

[0252] Use hierarchical initialization method to divide parameter space into storage layer parameters (such as Buffer Pool Size), computing layer parameters (such as Connection Pool Size) and network layer parameters. Determine the initial range of parameters at each level based on historical database data and expected load, for example, if the database handles a large number of small transactions, set the initial range of buffer pool size smaller; if it handles large and complex queries, it may need a larger buffer pool to improve cache hit rate. Randomly initialize particles at each level to ensure reasonable exploration range, and use heuristic methods (such as initialization based on experience or historical data) to improve initialization efficiency and quality, making particles closer to the optimal solution direction. Create a particle swarm, each particle represents a set of database parameters, introduce diversity metrics (such as particle distance and fitness value variance) to ensure sufficient diversity in the particle swarm to fully explore the parameter space during the search process and improve the likelihood of finding the global optimal solution.

[0253] Step five: fitness function creation

[0254] Collect and organize the query statements of computing cluster A, create a fitness function, and focus on evaluating the speed of multi-table join query and the efficiency of memory usage. By inserting test data in computing cluster A, record the response time of multi-table join query request and calculate the average response time as the query response time indicator; count the number of multi-table join query requests processed per unit of time to calculate the throughput; monitor the database server resource usage to calculate the resource utilization rate (focus on memory). The fitness function uses a weighted summation method to consider these factors, and assigns weight coefficients to each indicator according to the specific application scenario and requirements of the cluster.

[0255] Step six: particle performance evaluation

[0256] Test the parameters represented by each particle, combine multiple indicators such as query response time, throughput, and resource utilization, and use the fitness function to calculate the performance score. Introduce multi-index evaluation methods such as PCA for standardization and eigenvalue decomposition of performance indicators to obtain principal components reflecting the performance of parameters in different aspects; AHP establishes a hierarchical model to determine the relative importance of each indicator and assigns different weights to obtain the performance evaluation result by combining the scores of each indicator. At the same time, establish a performance evaluation benchmark model, select an open-source database performance test tool suitable for multi-table join query scenarios for benchmark testing, adjust the test parameters to simulate different application scenarios, collect performance data of the same type of database as a reference benchmark, and comprehensively evaluate the performance of the parameters.

[0257] Step seven: individual extreme value update

[0258] Record the best parameters (highest fitness score) found by each particle, introduce a dynamic update strategy, and adjust the update frequency according to the number of iterations or performance improvement. In the early stage of iteration, frequently update to encourage exploration, reduce the frequency in the middle stage for local search, and further reduce the frequency in the later stage to ensure stable convergence. At the same time, consider the influence of the historical optimal solution, weight the current best parameters by setting a weight coefficient, and obtain the updated individual extreme value by weighted summation, the weight coefficient is dynamically adjusted according to the number of iterations or performance improvement, and the storage and update mechanism of the historical optimal solution is set.

[0259] Step eight: global extreme value update

[0260] Determine the configuration with the highest fitness score among all particles as the global best. Adopt a distributed update strategy, divide the particle swarm into subgroups, and use the local optimal solution as the global extreme value of the subgroup. Through communication and cooperation, exchange local optimal solution information with other subgroups and update the global extreme value. Introduce a stability judgment mechanism, when the global extreme value changes less than a threshold value in continuous multiple iterations, it is considered that the algorithm may converge, local search or random disturbance can be added to find better solutions, continuously monitor the stability of the global extreme value, and set the maximum number of iterations to avoid excessive calculation.

[0261] Step nine: Update velocity and position using PSO formula

[0262] Adjust the velocity and position of each particle using the velocity update formula and position update formula of PSO, ensuring that the parameters are mapped back to the effective value range.

[0263] Step ten: Boundary processing

[0264] If the updated position exceeds the effective range, use boundary processing techniques such as reflection, folding back, or contraction.

[0265] Step eleven: Iterative optimization

[0266] Repeat steps six to ten until the stopping condition is met (such as reaching the maximum number of iterations or the fitness no longer significantly improves).

[0267] Step twelve: Optimal configuration selection

[0268] Determine the parameters represented by the global best particle and set them as the optimal parameters of the database.

[0269] Step thirteen: Verification and deployment

[0270] Verify the optimal parameters found in the computing cluster B and synchronize them to the computing cluster A.

[0271] Step fourteen: Continuous iterative optimization

[0272] Generate a new fitness function based on changes in the demand of computing cluster A (such as by monitoring changes in query request quantity, response time, resource utilization, etc.), and continuously iterate and optimize.

[0273] Example two: Database parameter optimization for high-concurrency query clusters

[0274] Step one: Preparation

[0275] In a suitable environment, create computing cluster A and computing cluster B, both with identical specifications, consisting of 3 hosts. These hosts use X86 architecture and Ubuntu operating system, and are configured with 24 cores and 128GB of memory. Use automated deployment tools to ensure efficient, consistent, and repeatable creation process, while recording key information such as network settings, storage allocation, and software installation versions during deployment, providing strong evidence for subsequent troubleshooting and optimization. Conduct comprehensive and in-depth performance testing and benchmarking on each host, covering CPU performance, memory read / write speed, disk I / O, and other important aspects. Use professional performance testing tools to obtain accurate performance indicator data, and establish detailed performance profiles for each host, recording performance under different load conditions, strengths and weaknesses. In addition, establish a powerful monitoring system to monitor the running state of computing clusters A and B in real time, including CPU usage, memory usage, disk space, network traffic, and other basic indicators, as well as database connection number, query execution time, and transaction processing speed, which are closely related to database performance. Use monitoring tools to create a visual monitoring interface, set reasonable thresholds and alarm mechanisms, so that when monitoring indicators exceed the preset threshold, an alarm will be automatically sent to the administrator, and timely measures will be taken to handle it. At the same time, establish historical records and analysis functions for monitoring data, so that administrators can review the running trends of the system, and timely detect potential performance problems and perform preventive maintenance.

[0276] Step two: parameter space definition

[0277] Identify the list of database parameters that need to be optimized for this high-concurrency query scenario, focusing on query performance and cache efficiency.

[0278] Buffer Pool Size: Set the value range to 512MB-64GB. Reasonably adjusting the buffer pool size can significantly improve data read / write speed, avoid too small causing frequent disk I / O reducing system performance, or too large occupying too much memory resources affecting other processes. By analyzing the historical access patterns of the database, combined with the average data access volume and peak access volume under this high-concurrency query scenario, determine the appropriate buffer pool size range.

[0279] Connection Pool Size: Value range is 50-200. The upper limit of the number of connections needs to consider the expected load of the database and the hardware resources of the server, to ensure that in the high-concurrency query scenario, it can meet the connection needs of a large number of clients, and will not cause system resources to be exhausted affecting performance.

[0280] Query Cache Size: The value range is 128MB-1GB. The size of the query cache affects the performance of repeated queries, but may also bring additional memory overhead and cache consistency problems. According to the characteristics of high-concurrency query scenarios, the opening and closing state of the query cache and the appropriate cache size can be determined.

[0281] Other related configuration items: According to the specific database system and the needs of high-concurrency query scenarios, other parameters such as memory usage strategy and query optimizer parameters may also need to be considered. Establish a parameter sensitivity analysis model, and use single-factor analysis and multi-factor analysis to determine the influence degree and mutual relationship of each parameter. For a single parameter such as BufferPool Size, its independent influence degree can be determined by gradually adjusting its value and observing the changes in query response time, throughput, resource utilization and other performance indicators. Use multi-factor analysis methods such as orthogonal experimental design to select key parameters for combined experiments, and analyze the interaction between parameters and the comprehensive influence on database performance.

[0282] Steps three to fourteen:

[0283] Repeat steps three to fourteen in example one to adjust parameters to optimize query response time and system throughput.

[0284] In summary, the above examples define a reasonable value range for the parameters and map each configuration to a point in the search space. Thanks to the characteristics of the store-compute separation architecture, without additional data synchronization, only the query on computing cluster A needs to be synchronized and executed on computing cluster B. According to the query response time and concurrency of the actual business scenario, the performance of different configurations can be evaluated. Through in-depth analysis of the dynamic evolution process of the particle swarm, the best parameters are selected and applied to computing cluster A, thereby significantly improving the overall system performance and ensuring efficient resource utilization and query efficiency. According to the changes in the usage scenario of computing cluster A, the database parameters can be dynamically optimized.

[0285] Applying the above examples to the billing system of a communication operator can improve data processing speed, improve billing accuracy, optimize resource utilization efficiency, and improve user experience. This combination not only helps to improve the service quality and user satisfaction of operators, but also reduces operating costs and enhances the competitiveness of enterprises. Through intelligent management and automated processes, communication operators can better serve the growing needs of users and respond to market and technological changes in a timely manner.

[0286] The above examples can be applied to at least the following technical scenarios:

[0287] 1. Financial industry

[0288] In financial institutions, there is a need to handle large amounts of transaction data, customer information, and risk assessment data.

[0289] 1.1 Transaction data analysis: By optimizing database parameters, transaction data can be quickly processed, and real-time analysis of transaction trends and risks can be performed, providing accurate data support for financial decision-making. For example, banks can quickly analyze credit card transaction data to detect abnormal transaction behavior and reduce fraud risk.

[0290] 1.2 Customer relationship management: Optimizing database parameters can improve customer information query speed and better understand customer needs, providing personalized financial services. For example, insurance companies can quickly query customer policy information and claim records, improving service efficiency and customer satisfaction.

[0291] 1.3 Risk assessment and management: Financial institutions need to assess and manage various risks, and optimizing database parameters can improve the speed and accuracy of risk model calculations. For example, investment companies can run risk assessment models more quickly to provide more reliable basis for investment decisions.

[0292] 2. E-commerce field

[0293] E-commerce platforms face massive amounts of product information, user behavior data, and order data.

[0294] 2.1 Product recommendation system: By optimizing database parameters, user behavior data can be processed faster, improving the accuracy and real-time nature of product recommendations. For example, e-commerce platforms can quickly recommend interested products to users based on their browsing history and purchase records.

[0295] 2.2 Order processing system: Optimizing database parameters can improve order processing speed, reduce order processing time, and improve user shopping experience. For example, during peak shopping periods, large numbers of orders can be quickly processed to ensure order accuracy and timeliness.

[0296] 2.3 Inventory management: E-commerce enterprises need to have real-time inventory information, and optimizing database parameters can improve inventory data query speed and update frequency, helping enterprises better manage inventory. For example, timely inventory quantity information can help avoid stockouts or overstocking.

[0297] 3. Logistics industry

[0298] Logistics companies need to manage large amounts of cargo transportation information, vehicle location information, and customer order information.

[0299] 3.1 Path Optimization: By optimizing database parameters, the vehicle location information and traffic data can be quickly processed, providing the most optimal transportation path for logistics vehicles. For example, express delivery companies can plan the fastest delivery route for delivery vehicles based on real-time traffic conditions and cargo destinations.

[0300] 3.2 Goods Tracking System: Optimizing database parameters can improve the query speed and accuracy of goods tracking information, allowing customers to know the location and status of goods at any time. For example, customers can use mobile phone software to query the transportation progress of packages in real time.

[0301] 3.3 Inventory Management and Distribution Scheduling: Logistics companies need to reasonably arrange inventory and distribution plans, and optimizing database parameters can improve the processing speed of inventory data and distribution demand, improving logistics efficiency. For example, based on inventory levels and customer order conditions, vehicles can be reasonably scheduled for distribution.

[0302] 4. Medical Industry

[0303] The medical field needs to handle a large amount of patient medical records, medical imaging data, and diagnostic reports.

[0304] 4.1 Electronic Medical Record System: Optimizing database parameters can improve the query speed and storage efficiency of electronic medical records, making it easy for doctors to quickly understand patient medical history and treatment. For example, doctors can quickly call up the electronic medical records of patients when they see them, providing a reference for diagnosis and treatment.

[0305] 4.2 Medical Imaging Storage and Analysis: Medical imaging data is large, and optimizing database parameters can improve the storage and reading speed of imaging data, supporting fast analysis and diagnosis of medical imaging. For example, radiologists can quickly view and analyze medical imaging, improving diagnostic efficiency.

[0306] 4.3 Medical Resource Management: Hospitals need to reasonably manage medical resources, and optimizing database parameters can improve the query and scheduling speed of medical equipment and bed resources. For example, hospital administrators can monitor the use of medical equipment in real time, and reasonably arrange equipment maintenance and allocation.

[0307] 5. Energy Industry

[0308] Energy companies need to manage a large amount of energy production data, equipment operation data, and user electricity data.

[0309] 5.1 Energy Production Monitoring and Optimization: By optimizing database parameters, the running status of energy production equipment can be monitored in real time, and the processing speed of production data can be improved to provide data support for energy production optimization. For example, power companies can quickly analyze the operation data of power generation equipment, adjust power generation plans, and improve energy production efficiency.

[0310] 5.2 Smart grid management: Optimizing database parameters can improve the transmission and processing speed of data in the smart grid, enabling real-time monitoring and optimal scheduling of the power grid. For example, based on user electricity demand and grid load conditions, power resources can be reasonably allocated to improve the stability and reliability of the power grid.

[0311] 5.3 Energy consumption analysis: Energy companies need to analyze users' energy consumption behavior, and optimizing database parameters can improve the processing speed and accuracy of energy consumption data analysis. For example, gas companies can analyze users' gas usage habits and provide personalized energy services to users.

[0312] Figure 4 The structure diagram of the database parameter configuration device provided by the embodiment of the application is provided. The device is used to execute the database parameter configuration method provided by any of the above embodiments. The device and the database parameter configuration method of each embodiment belong to the same inventive concept. Details not described in the embodiment of the database parameter configuration device can be referred to the embodiment of the database parameter configuration method. Referring to Figure 4 , the device can specifically include: a computing cluster determination module 310, a target parameter value group obtaining module 320, and a database parameter configuration module 330. Among them,

[0313] The computing cluster determination module 310 is configured to determine a first computing cluster for operating a target database and a second computing cluster for performing database parameter configuration, wherein the first computing nodes in the first computing cluster and the second computing nodes in the second computing cluster have the same specifications;

[0314] The target parameter value group obtaining module 320 is configured to configure database parameters for each second computing node for the database parameters of the target database, and obtain a target parameter value group, wherein the target parameter value group includes target parameter values corresponding to each second computing node;

[0315] The database parameter configuration module 330 is configured to synchronize the target parameter value group to the first computing cluster, so as to configure the target parameter values corresponding to each first computing node in the target parameter value group to the first computing node.

[0316] Optionally, the target parameter value group obtaining module 320 can include:

[0317] The parameter space determination sub-module is configured to determine the value range of the database parameters, and determine the parameter space to be searched according to the value range;

[0318] The particle swarm creation sub-module is configured to create a particle swarm, wherein the particles in the particle swarm represent candidate parameter value groups of the database parameters, and the candidate parameter value groups include candidate parameter values corresponding to each second computing node.

[0319] a particle control sub-module, configured to control each particle to move in the parameter space for searching;

[0320] a target parameter value set determination sub-module, configured to, after each movement, test candidate parameter value sets respectively represented by the moved particles based on the fitness function, to determine the target parameter value set from the candidate parameter value sets obtained after the movement.

[0321] On this basis, optionally, the fitness function is constructed based on a plurality of performance indicators, the performance indicators being indicators representing performance of the target database, and the target parameter value set determination sub-module can include:

[0322] a query statement acquisition unit, configured to acquire a query statement synchronized from the first computing cluster, wherein the query statement is a statement executed by the first computing cluster to query stored data in the target database, and the stored data is data shared by the first computing cluster and the second computing cluster;

[0323] a function value obtaining unit, configured to, after each movement, execute the query statement under a candidate parameter value set represented by each moved particle, to obtain a second indicator value of the performance indicators, and substitute the second indicator values of the plurality of performance indicators into the fitness function, to obtain a function value of the moved particle;

[0324] a target parameter value set determination unit, configured to determine the target parameter value set from the candidate parameter value sets obtained after the movement according to all the function values.

[0325] Optionally, the fitness function is constructed based on a plurality of performance indicators, the performance indicators being indicators representing performance of the target database, and the database parameter configuration apparatus can further include:

[0326] a change monitoring module, configured to, after the target parameter value set is synchronized to the first computing cluster, monitor changes of first indicator values respectively corresponding to the plurality of performance indicators of the first computing cluster;

[0327] a database parameter reconfiguration module, configured to, in a case where it is determined according to the monitored changes that the database parameters need to be reconfigured, reconfigure the database parameters according to the changes and at least some of the plurality of performance indicators, to reconfigure the database parameters according to the reconfigured fitness function.

[0328] Optionally, the performance indicators include at least one of the following: query response time, throughput, resource utilization, and concurrency.

[0329] Optionally, the number of database parameters is a plurality, and the database parameter configuration apparatus can further include:

[0330] The parameter space division module is configured to divide the parameter space into a plurality of parameter levels after determining the parameter space to be searched according to the value range, so as to perform, for each parameter level in the plurality of parameter levels, a configuration process of the database parameters corresponding to the parameter level in the plurality of database parameters in the parameter level.

[0331] Further, the particle swarm creation process and the particle control process are both implemented based on a particle swarm optimization algorithm.

[0332] Optionally, the target database is an analytical database applied by a charge bill processing system, and the analytical database is implemented by using a storage-computing separation architecture.

[0333] The database parameter configuration device provided by the embodiments of the present application makes full use of the characteristics of the storage-computing separation architecture, and adds a second computing cluster for database parameter configuration on the premise of ensuring that the normal operation of an existing computing cluster (i.e., a first computing cluster applied as a production cluster) is not affected, configures database parameters for each second computing node in the second computing cluster, and then synchronizes the database parameters to the first computing cluster, so that the resources on each first computing node can be fully utilized, thereby solving the problem of uneven resource utilization, which helps to improve the overall performance of the target database and has great significance for large-scale data analysis and processing.

[0334] The database parameter configuration device provided by the embodiments of the present application can execute the database parameter configuration method provided by any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0335] It should be noted that, in the embodiments of the above database parameter configuration device, each unit and module included is only divided according to the functional logic, but is not limited to the above division, as long as the corresponding functions can be implemented; in addition, the specific names of each functional unit are only for convenient mutual distinction, and do not limit the protection scope of the present application.

[0336] Figure 5A structural diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0337] As shown in Figure 5 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., connected to the at least one processor 11 in communication, where the memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0338] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0339] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, and microcontroller, etc. The processor 11 performs various methods and processes described above, such as the database parameter configuration method.

[0340] In some embodiments, the database parameter configuration method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the above-described database parameter configuration method can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the database parameter configuration method by way of other any suitable means, e.g., by way of firmware.

[0341] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0342] Computer programs used to implement the methods of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the machine, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0343] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0344] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0345] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0346] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0347] In particular, the processes described above with reference to the flow charts can be implemented as computer software programs in accordance with embodiments of the application. For example, embodiments of the application include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program comprising program code for performing the methods illustrated in the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication unit 19, or installed from the storage unit 18, or installed from the ROM 12. When the computer program is executed by the processor 11, the above-described functions defined in the methods of the embodiments of the application are performed.

[0348] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present application. For example, the steps recited in the present application can be performed in parallel, in series, or in a different order, without departing from the desired results of the technical solutions of the present application, and this is not limited herein.

[0349] The above detailed description does not constitute a limitation on the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A database parameter configuration method, characterized in that, include: A first computing cluster for operating the target database and a second computing cluster for configuring database parameters are determined, wherein the first computing node in the first computing cluster and the second computing node in the second computing cluster have the same specifications; For the database parameters of the target database, the database parameters are configured for each of the second computing nodes to obtain a target parameter value group, wherein the target parameter value group includes the target parameter values ​​corresponding to each of the second computing nodes; The target parameter value group is synchronized to the first computing cluster so that, for each first computing node, the target parameter value corresponding to the first computing node in the target parameter value group is configured on the first computing node.

2. The method according to claim 1, characterized in that, The step of configuring the database parameters for each of the second computing nodes to obtain a target parameter value group includes: Determine the value range of the database parameters, and determine the parameter space to be searched based on the value range; Create a particle swarm, wherein the particles in the particle swarm represent a group of candidate parameter values ​​for the database parameters, and the group of candidate parameter values ​​includes the candidate parameter values ​​corresponding to each of the second computing nodes. The particles are controlled to move within the parameter space to perform the search; After each move, the candidate parameter value set represented by each particle after the move is tested based on the fitness function, so as to determine the target parameter value set from the candidate parameter value sets obtained after the move.

3. The method according to claim 2, characterized in that, The fitness function is constructed based on multiple performance metrics, which characterize the performance of the target database. After each move, based on the fitness function, the candidate parameter value sets represented by each particle after the move are tested to determine the target parameter value set from the candidate parameter value sets obtained after the move, including: Obtain a query statement synchronized from the first computing cluster, wherein the query statement is a statement executed by the first computing cluster to query stored data in the target database, and the stored data is data shared by the first computing cluster and the second computing cluster; After each move, for each particle after the move, the query statement is executed under the candidate parameter value group represented by the particle after the move to obtain the second index value of the performance index, and the second index values ​​of multiple performance indices are substituted into the fitness function to obtain the function value of the particle after the move. Based on all the function values, the target parameter value group is determined from each of the candidate parameter value groups obtained after the shift.

4. The method according to claim 2, characterized in that, The fitness function is constructed based on multiple performance metrics, which are indicators characterizing the performance of the target database. After synchronizing the target parameter value group to the first computing cluster, the method further includes: Monitor the changes in the first indicator values ​​corresponding to the first computing cluster under various performance indicators; If it is determined that the database parameters need to be reconfigured based on the monitored changes, the fitness function is reconstructed based on the changes and at least some of the performance indicators, so as to reconfigure the database parameters based on the reconstructed fitness function.

5. The method according to claim 3 or 4, characterized in that, The performance metrics include at least one of the following: query response time, throughput, resource utilization, and concurrency.

6. The method according to claim 2, characterized in that, If there are multiple database parameters, then after determining the parameter space to be searched based on the value range, the method further includes: The parameter space is divided into multiple parameter levels, and for each of the multiple parameter levels, a configuration process is performed on the database parameters corresponding to the parameter level in the multiple database parameters, wherein the multiple parameter levels include at least two of storage layer parameters, computing layer parameters and network layer parameters.

7. The method according to claim 2, characterized in that, The creation process of the particle swarm and the control process of the particles are both implemented based on the particle swarm optimization algorithm.

8. The method according to claim 1, characterized in that, The target database is an analytical database used in the call billing system, and the analytical database is implemented using a storage-compute separation architecture.

9. A database parameter configuration device, characterized in that, include: The computing cluster determination module is used to determine a first computing cluster for operating the target database and a second computing cluster for configuring database parameters, wherein the first computing node in the first computing cluster and the second computing node in the second computing cluster have the same specifications. The target parameter value group obtaining module is used to configure the database parameters of the target database for each of the second computing nodes respectively, and obtain the target parameter value group, wherein the target parameter value group includes the target parameter values ​​corresponding to each of the second computing nodes respectively; The database parameter configuration module is used to synchronize the target parameter value group to the first computing cluster, so as to configure the target parameter value corresponding to the first computing node in the target parameter value group to the first computing node for each first computing node.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the database parameter configuration method as described in any one of claims 1-8.

Images