Database stand-alone distributed dynamic switching method, device and equipment and medium

Abstract

The invention relates to the technical field of database management, and discloses a database stand-alone distributed dynamic switching method which comprises the following steps: transmitting a service request to a stand-alone database and executing the service request, and obtaining a key index generated by executing the service request; comparing the key index with an index threshold value, and if the key index exceeds the index threshold value, switching a single-machine mode into a distributed mode; when the stand-alone mode is switched to the distributed mode, stopping the write-in operation of the stand-alone database according to preset time, and obtaining synchronous data of the stand-alone database; distributing the synchronous data of the stand-alone database in distributed nodes according to a preset fragmentation rule; and transmitting the service request to a distributed database from a stand-alone database according to a preset gradient, and executing the service request by the distributed database to complete dynamic database switching. According to the method, distributed node deployment is triggered through real-time monitoring, so that strong data consistency, non-inductive service switching and full-process automatic control are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field, and particularly relates to a database single-machine distributed dynamic switching method, device, equipment and storage medium. BACKGROUND

[0002] The current mainstream cloud database products (such as PolarDB, Aurora) have certain limitations in resource expansion. They only support the expansion operation of homogeneous resources, for example, increasing read-only nodes to improve the reading ability, but cannot dynamically switch the database engine architecture at runtime, that is, cannot flexibly convert between single-machine architecture and distributed architecture. This limitation makes the flexibility of database architecture adjustment insufficient when facing business demand changes.

[0003] Although the traditional database splitting and table splitting scheme can alleviate the database pressure to a certain extent, its disadvantages are also obvious. This scheme needs to rewrite the SQL logic at the business layer, which not only increases the complexity of development and maintenance, but also violates the application transparency requirement of cloud native infrastructure. Cloud native emphasizes the infrastructure's non-perception support for applications, and the traditional database splitting and table splitting scheme obviously cannot meet this demand.

[0004] In addition, the existing cross-architecture migration tools (such as DTS) also have some problems. They usually rely on a downtime window to complete data migration, which will cause business interruption and cannot guarantee business continuity. At the same time, these tools also have the risk of data loss, the recovery point target (RPO) is greater than zero, that is, part of the data may be lost during the migration process, which is unacceptable for enterprises with high requirements for data integrity and business continuity. SUMMARY

[0005] The main purpose of the present application is to provide a database single-machine distributed dynamic switching method, device, equipment and storage medium, which solves the problems of the existing cloud database that cannot dynamically switch the architecture at runtime, high business adaptation cost, and high data migration risk.

[0006] To achieve the above purpose, the present application provides a database single-machine distributed dynamic switching method, comprising: transmitting a business request to a single-machine database and executing, obtaining a key indicator generated by executing the business request; comparing the key indicator with an indicator threshold, if the key indicator exceeds the indicator threshold, switching the single-machine mode to a distributed mode; stopping the write operation of the single-machine database according to a preset time when the single-machine mode is switched to the distributed mode, and obtaining synchronization data of the single-machine database; distributing the synchronization data of the single-machine database in the distributed nodes according to a preset sharding rule; The business request is transmitted from the single-machine database to the distributed database according to a preset gradient, the business request is executed by the distributed database, and the dynamic switching of the database is completed.

[0007] Further, to achieve the above object, the present application provides a database single-machine distributed dynamic switching device, comprising: A key indicator module is configured to transmit a business request to a single-machine database and execute the business request, and obtain a key indicator generated by the execution of the business request. A mode switching module is configured to compare the key indicator with an indicator threshold value, and switch a single-machine mode to a distributed mode if the key indicator exceeds the indicator threshold value. A data acquisition module is configured to stop a write operation of the single-machine database according to a preset time when the single-machine mode is switched to the distributed mode, and acquire synchronization data of the single-machine database. A data distribution module is configured to distribute the synchronization data of the single-machine database in distributed nodes according to a preset sharding rule. A business request transmission module is configured to transmit the business request from the single-machine database to the distributed database according to a preset gradient, execute the business request by the distributed database, and complete the dynamic switching of the database.

[0008] Further, to achieve the above object, the present application further provides a computer device, which comprises a memory, a processor, and a database single-machine distributed dynamic switching program stored in the memory and executable on the processor, and the database single-machine distributed dynamic switching program realizes the steps of the database single-machine distributed dynamic switching method when executed by the processor.

[0009] Further, to achieve the above object, the present application further provides a computer readable storage medium, which stores a database single-machine distributed dynamic switching program, and the database single-machine distributed dynamic switching program realizes the steps of the database single-machine distributed dynamic switching method when executed by a processor.

[0010] Beneficial effects: The application relates to the technical field of database management, and discloses a database single-machine distributed dynamic switching method, which comprises the following steps: transmitting a business request to a single-machine database and executing the business request, acquiring a key indicator generated by executing the business request; comparing the key indicator with an indicator threshold value, and switching a single-machine mode into a distributed mode if the key indicator exceeds the indicator threshold value; stopping a write operation of the single-machine database according to a preset time and acquiring synchronization data of the single-machine database when the single-machine mode is switched into the distributed mode; distributing the synchronization data of the single-machine database in distributed nodes according to a preset sharding rule; transmitting the business request from the single-machine database to a distributed database according to a preset gradient, executing the business request by the distributed database, and completing database dynamic switching. The application realizes data strong consistency, business non-sensing switching and full-process automatic control by triggering distributed node deployment through real-time monitoring. BRIEF DESCRIPTION OF DRAWINGS

[0011] The application will be further described below in combination with the drawings and embodiments. In the drawings: Figure 1 An application environment schematic diagram of the database single-machine distributed dynamic switching method in an embodiment of the application; Figure 2 A flowchart of the database single-machine distributed dynamic switching method in an embodiment of the application; Figure 3 A functional module schematic diagram of the preferred embodiment of the database single-machine distributed dynamic switching device of the application; Figure 4 A structure schematic diagram of the computer equipment in an embodiment of the application; Figure 5 Another structure schematic diagram of the computer equipment in an embodiment of the application. DETAILED DESCRIPTION

[0012] It should be understood that the specific embodiments described herein are merely intended to explain the application, and are not intended to limit the application.

[0013] The database single-machine distributed dynamic switching method provided by the embodiments of the application can be applied to, for example, Figure 1The application environment is that a user end communicates with a service end through a network. The service end can transmit a business request to a single machine database through the user end and execute the business request to obtain a key indicator generated by executing the business request. The key indicator is compared with an indicator threshold value. If the key indicator exceeds the indicator threshold value, the single machine mode is switched to a distributed mode. When the single machine mode is switched to the distributed mode, a write operation of the single machine database is stopped according to a preset time, and synchronization data of the single machine database is obtained. The synchronization data of the single machine database is distributed in distributed nodes according to a preset sharding rule. The business request is transmitted from the single machine database to a distributed database according to a preset gradient, the business request is executed by the distributed database, and database dynamic switching is completed. The application realizes data strong consistency, business non-sensing switching and full-process automatic control by triggering distributed node deployment in real time. The user end can be but is not limited to various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The service end can be realized by an independent server or a server cluster composed of multiple servers. The application is described in detail through specific embodiments.

[0014] Please refer to Figure 2 , Figure 2 The flowchart of an embodiment of the database single machine distributed dynamic switching method provided by the application is shown. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown.

[0015] As shown in Figure 2 The database single machine distributed dynamic switching method provided by the application includes the following steps: S100, transmitting a business request to a single machine database and executing the business request to obtain a key indicator generated by executing the business request; S200, comparing the key indicator with an indicator threshold value, and if the key indicator exceeds the indicator threshold value, switching the single machine mode to a distributed mode; S300, when the single machine mode is switched to the distributed mode, stopping a write operation of the single machine database according to a preset time, and obtaining synchronization data of the single machine database; S400, distributing the synchronization data of the single machine database in distributed nodes according to a preset sharding rule; S500, transmitting the business request from the single machine database to a distributed database according to a preset gradient, executing the business request by the distributed database, and completing database dynamic switching.

[0016] In this embodiment, during the process of switching the database from single-machine mode to distributed mode, the system ensures the smooth progress of the switch through a series of ordered steps, while guaranteeing the stability of the system and the consistency of the data.

[0017] The application layer generates business requests (such as SQL queries, inserts, updates, etc.) and sends them to the single-machine database through the network. The single-machine database receives and executes these requests and returns the execution results to the application layer. The monitoring module collects key indicators generated by the single-machine database when executing business requests in real time, such as CPU usage, memory usage, disk I / O, response time, etc. These indicators can be collected through the tools provided by the operating system or the monitoring tools provided by the database itself. The monitoring module compares the collected key indicators with the preset threshold values, for example, the threshold value of CPU usage may be set to 80%. If the key indicators exceed the threshold value, the monitoring module will record this event and start timing. If the key indicators continue to exceed the threshold value within the preset time (for example, 5 minutes), the monitoring module will trigger the creation process of the distributed node. The monitoring module creates a distributed node by calling the Kubernetes API and switches the single-machine mode to the distributed mode. The newly created distributed node enters the standby state after starting, completes the network configuration, but does not participate in the processing of business requests temporarily.

[0018] When switching to the distributed mode, the system will stop the write operation of the single-machine database according to the preset time (such as 50 milliseconds), which can be achieved through the write lock mechanism of the database. The purpose of freezing the write operation is to ensure the consistency of the data when generating the snapshot. The system performs snapshot operation on the single-machine database to generate a full-database consistent snapshot. The snapshot operation captures the complete state of the database at a certain time, for example, the FLUSH TABLES WITH READ LOCK command of MySQL can be used to freeze the write operation and generate a snapshot. At the same time, the CDC (Change Data Capture) component is enabled to capture the incremental data changes generated by the single-machine database after the snapshot is generated. The CDC component captures data changes by reading the log files (such as Redo log) of the database and synchronizes these changes to the distributed node in real time. The system combines the snapshot data and the incremental data to generate complete synchronization data, which will be used for subsequent data migration.

[0019] The system traverses the data blocks in the single-machine database, and calculates the hash value of the partition key of each data block to obtain the shard ID of each data block. According to the shard ID and the preset shard rule, the target distributed node is calculated. For example, if there are 1024 shards, and each node carries 256 shards, then the data block with shard ID shard_id will be written to the target_node node. The system asynchronously writes the data block to the target distributed node, which can improve the efficiency of data migration and reduce the impact on business. For example, thread pool or asynchronous I / O library can be used to realize asynchronous writing.

[0020] After the data migration is completed, the agent layer starts to send business requests to the single-machine database and the distributed database simultaneously for double-writing operation. This ensures that the data states in the two architectures remain consistent. The agent layer monitors the execution results of the two nodes to ensure the consistency of transactions. If the execution in the single-machine database is successful but the execution in the distributed database fails, the agent layer will roll back the entire transaction. The agent layer gradually migrates the business requests from the single-machine database to the distributed database according to the preset gradient. In each gradient stage, the key indicators of the distributed database node are monitored in real time, such as CPU usage, memory usage, response time, etc. If the key indicators exceed the threshold, the agent layer will reduce the proportion of business request flow or roll back to the previous gradient.

[0021] If the distributed database runs stably within the preset observation period, the agent layer will completely switch all business requests to the distributed database and release the resources of the single-machine database. If the switching fails, the agent layer will fall back to the single-machine mode and clean up the resources of the distributed database.

[0022] Through the above mechanism, the system ensures the smooth switching of the database from the single-machine mode to the distributed mode. The key indicators of the single-machine database are monitored in real time, and the switching is triggered when the indicators exceed the threshold. The write operation is frozen during the switching, a snapshot is generated and the incremental data is captured. The data blocks are distributed in the distributed nodes according to the preset shard rule. The business requests are gradually migrated from the single-machine database to the distributed database according to the preset gradient. After the distributed database runs stably, the resources of the single-machine database are released; if the switching fails, the system falls back to the single-machine mode and cleans up the resources of the distributed database. This process not only ensures the stability of the system, but also reduces the impact on business.

[0023] In one embodiment, the step S100 comprises: S101, setting the initial state of the database cluster to the single-machine mode; S102, sending a business request to the single-machine database and executing the business request; S103, detecting the key indicators of the single-machine database node in executing the business request in real time through the monitoring module.

[0024] In this embodiment, when the database cluster starts, it can be set to single-machine mode through a configuration file or system parameters. For example, set mode=single in the configuration file. The database management system (DBMS) will load the configuration file when starting, and confirm that the cluster mode is single-machine mode. At this time, initialize the database connection pool, connect only to the single-machine database node, and ensure that all business request routes point to this single-machine node.

[0025] The application layer (such as Web application, mobile application, etc.) will generate business requests, such as SQL queries, insert, update, etc. These requests are sent to the database proxy layer (if exists) or directly to the single-machine database node through the network. The single-machine database node receives the request, parses the SQL statement and executes the corresponding query or update operation, and finally returns the execution result to the application layer. The application layer uses standard database connection library (such as JDBC, ODBC, etc.) to establish connection with the single-machine database node.

[0026] The monitoring module is a background service responsible for real-time monitoring of the performance indicators of the single-machine database node, such as CPU usage, memory usage, disk I / O, network bandwidth, response time, etc. The monitoring module can use the performance monitoring tools provided by the operating system (such as top, vmstat of Linux, etc.) or the monitoring tools provided by the database (such as the SHOW STATUS command of MySQL). The monitoring module can be a separate process or thread that periodically (such as every second or every minute) collects key indicators and records the collected indicator data to log files or sends them to centralized monitoring systems (such as Prometheus, Grafana, etc.) for visualization and analysis. The monitoring module will set thresholds (such as CPU usage exceeding 80%), and when the key indicators exceed the thresholds, it will trigger an alarm or automatically perform a preset operation, such as switching to distributed mode.

[0027] The database cluster is set to single-machine mode through a configuration file or system parameters, the application layer generates business requests and sends them to the single-machine database node, the node executes the request and returns the result. The monitoring module periodically collects key indicators and triggers an alarm or automatically switches to distributed mode when the indicators exceed the thresholds. This mechanism ensures that the system can run stably in the initial stage and trigger the distributed switching process in time when needed.

[0028] In one embodiment, the step S200 comprises: S201, comparing the key indicators with the indicator thresholds; S202, when the key indicators exceed the threshold for a preset time, invoke the API instruction to create a distributed node; S203, switch the single mode to the distributed mode, and set the distributed node to standby state.

[0029] In this embodiment, the monitoring module collects the key indicators of the single database node in real time, including CPU usage, memory usage, disk I / O, network bandwidth, etc. These indicators can be obtained through the system's own monitoring tools (such as Linux's top, vmstat, iostat, etc.) or the database's own monitoring tools (such as MySQL's SHOW STATUS command). The system presets the threshold of the key indicators, for example, the threshold of CPU usage may be set to 80%, and the threshold can be adjusted through the configuration file or the management interface to adapt to different system load requirements.

[0030] The monitoring module will periodically (such as every second) compare the collected key indicators with the preset threshold. If the key indicators exceed the threshold, the monitoring module will record this event and start timing. The monitoring module will detect whether the key indicators continuously exceed the threshold for a preset time, for example, the preset time is 5 minutes. If the key indicators continuously exceed 80% for 5 minutes, the monitoring module will trigger the creation process of the distributed node. The monitoring module creates a distributed node by invoking the Kubernetes API, and the API instruction includes the number of nodes, resource specifications (such as CPU, memory, storage capacity, etc.), and network configuration information. The Kubernetes API will create the specified number of distributed nodes according to the instruction, and the new nodes run in the form of containers, each container contains a database instance, and the new node will automatically join the network of the Kubernetes cluster after starting.

[0031] When the distributed node is created, the system will switch the single mode to the distributed mode, and this process is usually completed by updating the configuration file (such as switching the mode in the configuration file from single to distributed) or calling the system management interface. The newly created distributed node will enter standby state after starting, in which the distributed node completes network configuration, but does not participate in actual business request processing. The system will set the distributed node to only receive data synchronization requests, but not process business SQL requests.

[0032] In one embodiment, the step S300 comprises: S301, when switching the single mode to the distributed mode, stop the write operation of the single database according to the preset time; S302, perform snapshot operation on the single database to generate log records; S303, read the log records by data capture technology, capture the incremental data of the single-machine database in real time; S304, combine the log records and the incremental data to generate synchronization data.

[0033] In this embodiment, in order to ensure the consistency and integrity of data when switching from single-machine mode to distributed mode, the system will take a series of measures. First, the system will set a short time window (e.g. 50 milliseconds) during which the write operation of the single-machine database is stopped to reduce the impact on business. This process can be achieved through the write lock mechanism of the database management system (DBMS), such as using the FLUSH TABLES WITH READ LOCK command of MySQL. Write lock allows read operation to continue, but prevents write operation. At the same time, the system will record the current timestamp and related log information for subsequent audit and troubleshooting.

[0034] Next, the system will use the snapshot function of the database to generate a full-consistency snapshot of the single-machine database, capturing the complete state of the database at a certain time. During the snapshot operation, the system will generate detailed log records, including the timestamp of the snapshot, the status of the snapshot operation, and other information. These log records will be used for subsequent data synchronization and troubleshooting.

[0035] At the same time, the system will enable the CDC (Change Data Capture) component to capture the incremental data changes of the single-machine database after the snapshot is generated. The CDC component captures data changes by reading the database log files (such as Redo logs) and synchronizes these changes to the distributed nodes in real time. The captured incremental data includes insert, update, and delete operations, ensuring that the distributed nodes can receive the latest data changes in a timely manner.

[0036] The system will combine the log records generated by the snapshot operation and the incremental data captured by the CDC component to generate complete synchronization data. These synchronization data include snapshot data and incremental data, ensuring the integrity and consistency of data during the switching process. The system will send the combined synchronization data to the distributed nodes and perform data verification after the data synchronization is completed, such as using the CRC32 verification algorithm to compare the data of the source database and the target database to ensure the integrity and consistency of the synchronization data.

[0037] Through the above steps, the system ensures the consistency and integrity of data when switching from single-machine mode to distributed mode. Stopping the write operation of the single-machine database, generating snapshots, capturing incremental data, combining synchronization data and performing verification, and a series of operations lay a solid foundation for subsequent traffic switching and business seamless switching.

[0038] In one embodiment, the step S400 comprises: S401, traversing the data blocks in the single-machine database; S402, calculating the hash value of the partition key of each data block to obtain the shard ID of each data block; S403, calculating the target distributed node according to the shard ID; S404, asynchronously writing the data block to the target distributed node.

[0039] In this embodiment, in the process of migrating the data of the single-machine database to the distributed node, the system first traverses each table in the single-machine database and further traverses the data blocks in each table. The data block can be a partition, a range of records in the table, or a fixed-size data block. For example, the table can be divided into multiple partitions using the partition table function of the database, and each partition serves as a data block. The system traverses these data blocks through SQL query statements or APIs provided by the database.

[0040] For each data block, the system determines its partition key, which can be a field (such as the primary key or unique key) or a combination of multiple fields in the table. For example, the id field in the table can be selected as the partition key. The system calculates the hash value of the partition key of each data block, which can use common hash algorithms such as MD5, SHA-1, or more efficient hash algorithms. For example, the hashlib library in Python can be used to calculate the hash value.

[0041] According to the hash value and the preset shard rule, the system calculates the shard ID of each data block. The shard ID is used to determine the storage location of the data block in the distributed node. For example, the hash value can be taken modulo the number of distributed nodes to obtain the shard ID. The shard rule defines how to allocate data blocks to different distributed nodes. The shard rule can be a simple modulo operation or a more complex hash mapping. For example, the shard ID can be mapped to a specific distributed node.

[0042] The system calculates the target distributed node of each data block according to the shard ID and the shard rule. For example, if there are 1024 shards and each node carries 256 shards, the data block with shard ID shard_id will be written to target_node node. The system asynchronously writes the data block to the target distributed node, which can improve the efficiency of data migration and reduce the impact on business. For example, thread pools or asynchronous I / O libraries can be used to implement asynchronous writing. The system sends the data block to the target distributed node and executes the write operation on the distributed node. For example, the API or SQL statement of the database can be used to write the data block.

[0043] Through the above steps, the system ensures correct sharding and efficient migration of data in the process of migrating the data of the single-machine database to the distributed nodes. Each table and data block in the single-machine database is traversed, the hash value of the partition key of each data block is calculated, the shard ID is obtained, the target distributed node is calculated according to the shard ID and the shard rule, and the data block is asynchronously written to the target distributed node. This process not only improves the migration efficiency, but also ensures the correct distribution of data in the distributed nodes, providing a solid data foundation for subsequent traffic switching and business seamless switching.

[0044] In one embodiment, the step S500 comprises: S5011, sending the business request to the single-machine database and the distributed database for double-writing operation by the proxy layer; S5012, when the single-machine database and the distributed database jointly execute the business request to reach a stable stage, gradually migrating the business request from the single-machine database to the distributed database according to a preset gradient; S5013, in each gradient stage, monitoring the key indicators generated when the distributed database node executes the business request in real time; S5014, comparing the key indicators with the indicator threshold; S5015, when the key indicators exceed the indicator threshold, reducing the proportion of business request traffic or rolling back to the previous gradient.

[0045] In this embodiment, during the process of switching the database from single-machine mode to distributed mode, the proxy layer, as the core component of traffic lossless switching, is responsible for managing and coordinating the routing and execution of business requests. The proxy layer receives the business SQL requests of the application layer and parses these requests through the built-in SQL parser. According to the preset sharding rule, the proxy layer routes the requests to the correct nodes, ensuring that the requests can be correctly sent to the single-machine database or the distributed database.

[0046] After the data redistribution is completed, the proxy layer starts sending the business SQL requests to the single-machine database and the distributed database at the same time to ensure that the data states in the two architectures remain consistent. The proxy layer monitors the execution results of the two nodes to ensure the consistency of transactions. If the execution in the single-machine database is successful but the execution in the distributed database fails, the proxy layer will roll back the entire transaction to ensure the consistency of data.

[0047] The agent layer monitors the execution of the single-machine database and the distributed database to ensure that both databases can stably handle business requests. The judgment criteria in the stable stage can include execution success rate, response time, and other key indicators. The agent layer gradually migrates business requests from the single-machine database to the distributed database according to a preset gradient (e.g., 5%, 10%, 20%,..., 100%). For example, first switch 5% of the traffic to the distributed database, observe for a period of time, and confirm that the system is running stably before gradually increasing the traffic proportion.

[0048] In each gradient stage, the agent layer monitors key indicators generated when the distributed database nodes execute business requests, such as CPU usage, memory usage, response time, execution success rate, etc. The monitoring module records these indicators and performs real-time analysis. The system presets threshold values for key indicators, for example, the threshold value for CPU usage may be set to 80%, and the threshold value for response time may be set to 500 milliseconds. Threshold values can be adjusted through configuration files or management interfaces to adapt to different system load requirements.

[0049] The monitoring module compares the real-time collected key indicators with the preset threshold values. If the key indicators exceed the threshold values, the agent layer will take appropriate handling mechanisms. For example, if the key indicators exceed the threshold values, the agent layer will reduce the traffic proportion of business requests. If the current traffic proportion is 20%, it will be reduced to 15%. If the key indicators still exceed the threshold values after reducing the traffic proportion, the agent layer will roll back to the previous gradient, for example, from 20% to 10%. The rollback mechanism ensures that the system can quickly recover to a stable running state when the distributed database nodes have performance problems. The agent layer records the rollback operation and sends an alarm to the system administrator for further analysis and processing.

[0050] Through the above mechanism, the agent layer ensures that traffic lossless switching is achieved during the process of switching the database from single-machine mode to distributed mode. The agent layer not only ensures data consistency, but also ensures that the distributed database can stably handle business requests while minimizing the impact on business by gradually migrating traffic and real-time monitoring of key indicators.

[0051] In one embodiment, the step S500 further comprises: S5021, if the database dynamic switching is successful, converting the single-machine database into a read-only slave, and monitoring the performance of the distributed database in real time; S5022, if the distributed database runs stably within a preset observation period, releasing the resources of the single-machine database; S5023, if the database dynamic switching fails, returning to the single-machine mode, and recycling and cleaning the resources of the distributed database.

[0052] In this embodiment, during the database dynamic switching process, the system needs to take different measures according to the switching result to ensure the stability of the system and the effective management of resources.

[0053] When the distributed database successfully takes over all business requests, the system will convert the single-machine database into a read-only slave. This means that the single-machine database no longer accepts any write operations, but can still handle read requests. Technically, the single-machine database can be set to read-only mode by modifying the database configuration file or executing SQL commands. Subsequently, the system will start a real-time monitoring mechanism to continuously monitor the performance indicators of the distributed database, such as CPU usage, memory usage, response time, execution success rate, etc. The monitoring module will record these indicators and perform real-time analysis to ensure that the performance of the distributed database meets expectations. The system will set an observation period (e.g., 48 hours) to continuously monitor the performance of the distributed database during the observation period. The purpose of the observation period is to ensure that the distributed database can stably handle business requests and avoid subsequent failures due to potential problems during the switching process. If the distributed database runs stably during the observation period without any abnormal situations, the system will automatically release the resources of the single-machine database. Resource release includes closing the database instance of the single-machine database, releasing computing resources (such as CPU, memory), releasing storage resources, etc.

[0054] If performance problems or other abnormal situations are found in the distributed database during the switching process or the observation period, the system will automatically fall back to the single-machine mode. The rollback operation includes rerouting business requests to the single-machine database and restoring the write permissions of the single-machine database. For example, the single-machine database can be restored to read-write mode by modifying the database configuration file or executing SQL commands. After falling back to the single-machine mode, the system will recycle and clean up the resources of the distributed database. Resource recycling includes closing the database instance of the distributed database, releasing computing resources (such as CPU, memory), releasing storage resources, etc.

[0055] Through the above mechanism, the system can effectively handle both successful and failed situations during the database dynamic switching process. In the case of successful switching, the single-machine database is converted into a read-only slave, and the performance of the distributed database is monitored in real time. If the distributed database runs stably within the preset observation period, the resources of the single-machine database are released. In the case of failed switching, the system falls back to the single-machine mode and recycles and cleans up the resources of the distributed database. This process not only ensures the stability of the system but also provides support for subsequent troubleshooting and recovery.

[0056] In an embodiment, a database single-machine distributed dynamic switching device is provided, which corresponds one-to-one to the database single-machine distributed dynamic switching method in the above embodiment.Figure 3 , Figure 3 A function module schematic diagram of a preferred embodiment of the database single-machine distributed dynamic switching device. Key indicator module 10, mode switching module 20, data acquisition module 30, data distribution module 40, and service request transmission module 50. The detailed description of each function module is as follows: Key indicator module 10, for transmitting service requests to a single-machine database and executing, obtaining key indicators generated by executing service requests; Mode switching module 20, for comparing the key indicators with indicator thresholds, and if the key indicators exceed the indicator thresholds, switching the single-machine mode to a distributed mode; Data acquisition module 30, for stopping the write operation of the single-machine database according to a preset time when the single-machine mode is switched to the distributed mode, and acquiring synchronization data of the single-machine database; Data distribution module 40, for distributing the synchronization data of the single-machine database according to a preset sharding rule in distributed nodes; Service request transmission module 50, for transmitting the service requests from the single-machine database to the distributed database according to a preset gradient, executing the service requests by the distributed database, and completing the database dynamic switching.

[0057] In one embodiment, the key indicator module 10 includes: Initial state unit, for setting the initial state of a database cluster to a single-machine mode; Service request unit, for sending service requests to a single-machine database and executing the service requests; Key indicator unit, for detecting the key indicators of the single-machine database nodes in executing service requests in real time through a monitoring module.

[0058] In one embodiment, the mode switching module 20 includes: Indicator comparison unit, for comparing the key indicators with the indicator thresholds; Distributed node unit, for calling API instructions to create distributed nodes when the key indicators continuously exceed the indicator thresholds within a preset time; Mode switching unit, for switching the single-machine mode to a distributed mode and setting the distributed nodes to a standby state.

[0059] In one embodiment, the data acquisition module 30 includes: Task stop unit, for stopping the write operation of the single-machine database according to a preset time when the single-machine mode is switched to the distributed mode; A data block unit is configured to perform a snapshot operation on the single-machine database to generate a log record. A data capture unit is configured to read the log record by using a data capture technology to capture incremental data of the single-machine database in real time. A synchronization data unit is configured to combine the log record and the incremental data to generate synchronization data.

[0060] In an embodiment, the data distribution module 40 comprises: A data block unit is configured to traverse data blocks in the single-machine database. A shard ID unit is configured to calculate a hash value of a partition key of each data block to obtain a shard ID of each data block. A distributed node unit is configured to calculate a target distributed node according to the shard ID. An asynchronous write unit is configured to asynchronously write the data block to the target distributed node.

[0061] In an embodiment, the service request transmission module 50 comprises: A double-write operation unit is configured to simultaneously send the service request to the single-machine database and the distributed database by the proxy layer to perform double-write operation. A step-by-step transmission unit is configured to, when the single-machine database and the distributed database jointly execute the service request to reach a stable stage, gradually migrate the service request from the single-machine database to the distributed database according to a preset gradient. An index monitoring unit is configured to, at each gradient stage, monitor a key index generated when the distributed database node executes the service request in real time. An index comparison unit is configured to compare the key index with an index threshold. An abnormal operation unit is configured to, when the key index exceeds the index threshold, reduce a service request flow proportion or roll back to a previous gradient.

[0062] In an embodiment, the service request transmission module 50 further comprises: A mode switching unit is configured to, if the database dynamic switching succeeds, convert the single-machine database into a read-only slave database and monitor performance of the distributed database in real time. A switching success unit is configured to, if the distributed database stably operates within a preset observation period, release resources of the single-machine database. A switching failure unit is configured to, if the database dynamic switching fails, return to the single-machine mode and recycle and clean up resources of the distributed database.

[0063] In an embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown inFigure 4 As shown in the figure. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes non-volatile and / or volatile storage medium, internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external user terminal through the network connection. The computer program is executed by the processor to realize the functions or steps of the server side of the database single machine distributed dynamic switching method.

[0064] In one embodiment, a computer device is provided, which can be a user terminal, and its internal structure diagram can be as shown in the figure. Figure 5 As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes non-volatile storage medium, internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the functions or steps of the user terminal side of the database single machine distributed dynamic switching method In one embodiment, a computer device is provided, including a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to realize the following steps: Transmitting a business request to a single machine database and executing, obtaining a key indicator generated by executing the business request; Comparing the key indicator with an indicator threshold value, if the key indicator exceeds the indicator threshold value, switching the single machine mode to a distributed mode; When the single machine mode is switched to the distributed mode, stopping the write operation of the single machine database according to a preset time, and obtaining synchronization data of the single machine database; Distributing the synchronization data of the single machine database according to a preset sharding rule in a distributed node; Transmitting the business request from the single machine database to a distributed database according to a preset gradient, executing the business request by the distributed database, and completing the database dynamic switching.

[0065] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by the processor to realize the following steps: transmitting the service request to the single-machine database and executing the service request to obtain a key indicator generated by executing the service request; comparing the key indicator with an indicator threshold value, and if the key indicator exceeds the indicator threshold value, switching the single-machine mode to a distributed mode; when the single-machine mode is switched to the distributed mode, stopping a write operation of the single-machine database according to a preset time and obtaining synchronization data of the single-machine database; distributing the synchronization data of the single-machine database in the distributed nodes according to a preset sharding rule; transmitting the service request from the single-machine database to the distributed database according to a preset gradient, executing the service request by the distributed database, and completing the database dynamic switching.

[0066] It should be noted that the functions or steps that the computer readable storage medium or the computer device can achieve correspond to the descriptions of the server side and the user side in the foregoing method embodiments, and thus will not be described again.

[0067] Those skilled in the art can understand that all or part of the processes in the foregoing method embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the computer program can include the processes of the foregoing method embodiments. In the embodiments provided in the present application, any reference to a memory, storage, database or other medium can include a non-volatile and / or volatile memory. The non-volatile memory can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM) or a flash memory. The volatile memory can include a random access memory (RAM) or an external cache memory. As an illustration but not limitation, the RAM is available in various forms, such as a static RAM (SRAM), a dynamic RAM (DRAM), a synchronous DRAM (SDRAM), a double data rate SDRAM (DDR SDRAM), an enhanced SDRAM (ESDRAM), a synchronous link (Synchlink) DRAM (SLDRAM), a memory bus (Rambus) direct RAM (RDRAM), a direct memory bus dynamic RAM (DRDRAM) and a memory bus dynamic RAM (RDRAM).

[0068] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0069] It should be noted that if non-company software tools or components appear in the embodiments of the present application, they are only used for example introduction and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A database single-machine distributed dynamic switching method, characterized in that, The method comprises the following steps: transmitting a business request to a single-machine database and executing the same, obtaining a key indicator generated by executing the business request; comparing the key indicator with an indicator threshold value, and switching the single-machine mode to a distributed mode if the key indicator exceeds the indicator threshold value; stopping a write operation of the single-machine database according to a preset time and obtaining synchronization data of the single-machine database when the single-machine mode is switched to the distributed mode; distributing the synchronization data of the single-machine database in distributed nodes according to a preset sharding rule; transmitting the business request from the single-machine database to a distributed database according to a preset gradient, executing the business request by the distributed database, and completing dynamic switching of the database.

2. The database single-machine distributed dynamic switching method of claim 1, wherein, The method of transmitting a business request to a single-machine database and executing the same, and obtaining a key indicator generated by executing the business request comprises: setting an initial state of a database cluster as a single-machine mode; sending a business request to a single-machine database and executing the same; detecting a key indicator of the single-machine database node when executing the business request in real time by a monitoring module.

3. The database single-machine distributed dynamic switching method of claim 1, wherein, The method of comparing the key indicator with an indicator threshold value, and switching the single-machine mode to a distributed mode if the key indicator exceeds the indicator threshold value comprises: comparing the key indicator with the indicator threshold value; calling an API instruction to create a distributed node when the key indicator continuously exceeds the indicator threshold value within a preset time; switching the single-machine mode to the distributed mode and setting the distributed node as a standby state.

4. The database single-machine distributed dynamic switching method of claim 1, wherein, The method of stopping a write operation of the single-machine database according to a preset time and obtaining synchronization data of the single-machine database when the single-machine mode is switched to the distributed mode comprises: stopping the write operation of the single-machine database according to the preset time when the single-machine mode is switched to the distributed mode; performing a snapshot operation on the single-machine database to generate a log record; reading the log record by a data capture technology to capture incremental data of the single-machine database in real time; combining the log record and the incremental data to generate the synchronization data.

5. The database single-machine distributed dynamic switching method of claim 1, wherein, The method of distributing the synchronization data of the single-machine database in distributed nodes according to a preset sharding rule comprises: traversing data blocks in the single-machine database; calculating a hash value of a partition key of each data block to obtain a shard ID of each data block; calculating a target distributed node according to the shard ID; asynchronously writing the data block to the target distributed node.

6. The database single-machine distributed dynamic switching method of claim 1, wherein, The method of transmitting the business request from the single-machine database to the distributed database according to a preset gradient, executing the business request by the distributed database, and completing dynamic switching of the database comprises: sending the business request to the single-machine database and the distributed database simultaneously by a proxy layer to perform double-write operation; gradually migrating the business request from the single-machine database to the distributed database according to a preset gradient when the single-machine database and the distributed database jointly execute the business request to reach a stable stage; monitoring a key indicator generated by the distributed database node when executing the business request in real time at each gradient stage; comparing the key indicator with an indicator threshold value. When the key indicator exceeds the indicator threshold, reduce the proportion of business request traffic or roll back to the previous gradient.

7. The database single-machine distributed dynamic switching method of claim 1, wherein, The business request is transmitted from the single-machine database to the distributed database according to a preset gradient, and the business request is executed by the distributed database to complete the database dynamic switching. If the database dynamic switching is successful, the single-machine database is converted into a read-only slave database, and the performance of the distributed database is monitored in real time. If the distributed database is stably operated within a preset observation period, the resources of the single-machine database are released. If the database dynamic switching fails, the single-machine mode is returned to, and the resources of the distributed database are recycled and cleaned.

8. A database single-machine distributed dynamic switching device, characterized in that, The database single-machine distributed dynamic switching device comprises: a key indicator module configured to transmit a business request to a single-machine database and execute the business request to obtain a key indicator generated by executing the business request; a mode switching module configured to compare the key indicator with an indicator threshold, and switch a single-machine mode to a distributed mode if the key indicator exceeds the indicator threshold; a data acquisition module configured to stop a write operation of the single-machine database according to a preset time when the single-machine mode is switched to the distributed mode, and acquire synchronization data of the single-machine database; a data distribution module configured to distribute the synchronization data of the single-machine database in distributed nodes according to a preset sharding rule; a business request transmission module configured to transmit the business request from the single-machine database to a distributed database according to a preset gradient, and execute the business request by the distributed database to complete the database dynamic switching.

9. A computer device, comprising: The computer device comprises a memory, a processor, and a database single-machine distributed dynamic switching program stored on the memory and capable of running on the processor, and the database single-machine distributed dynamic switching program, when executed by the processor, implements the steps of the database single-machine distributed dynamic switching method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a database single-machine distributed dynamic switching program, and the database single-machine distributed dynamic switching program, when executed by the processor, implements the steps of the database single-machine distributed dynamic switching method in any one of claims 1-7.

Images