An intelligent switching method and device for primary and backup shard database nodes in a distributed memory database

Abstract

The application provides a kind of intelligent switching method and device for master and backup shard library nodes in distributed memory database, which can be applied to the field of database technology.The method comprises: using the connection pool state manager of distributed memory database to monitor the online connection number, effective connection number and connection state of shard library node in real time, to obtain the real-time load state of shard library node;based on the real-time load state of shard library node and / or user input service generation service mode, and store the configuration file corresponding to the service mode in the distributed memory database;using distributed memory database to select the service mode matched with application scenario, and based on the selected service mode, intelligently switch the master-slave relationship of shard library node in distributed memory database.

Description

Technical Field

[0001] This invention relates to the field of database technology, and more specifically to a method and apparatus for intelligent switching between primary and backup sharded database nodes in a distributed in-memory database. Background Technology

[0002] With the continuous development of big data technology, distributed database systems (such as GBaseXDM, which adopts a distributed memory architecture and supports cluster-level horizontal scaling) have gained market favor in handling massive amounts of data and high-concurrency requests. Distributed database systems improve data access speed and system scalability by distributing data across multiple nodes (i.e., data sharding). However, in traditional distributed database designs, each data shard has a primary node and one or more standby nodes (replicas). The main role of the standby nodes is to take over tasks when the primary node fails, ensuring system continuity.

[0003] However, in existing systems, standby nodes can only be activated when the primary node fails, goes offline, or becomes unavailable. While this design ensures high availability, under high concurrency and high load scenarios, the primary node's load may still reach a bottleneck, failing to effectively utilize the computing and storage capabilities of the standby node. Therefore, how to flexibly switch between primary and standby nodes based on real-time load conditions to achieve system load balancing and improve system performance has become a pressing issue for current distributed database systems. Summary of the Invention

[0004] In view of the above problems, the present invention provides a method and apparatus for intelligent switching of master and standby sharded database nodes in a distributed memory database, which is used to solve at least one of the above technical problems.

[0005] According to a first aspect of the present invention, a method for intelligent failover of master-slave sharded database nodes in a distributed in-memory database is provided, comprising:

[0006] The connection pool state manager of the distributed memory database is used to monitor the number of online connections, the number of active connections, and the connection status of the sharded database nodes in real time, so as to obtain the real-time load status of the sharded database nodes.

[0007] Based on the real-time load status of the sharded library nodes and / or user input, a service mode is generated, and the configuration file corresponding to the service mode is stored in a distributed memory database.

[0008] By utilizing a distributed in-memory database, a service mode that matches the application scenario is selected, and based on the selected service mode, the master-slave relationship of the sharded nodes in the distributed in-memory database is intelligently switched.

[0009] According to an embodiment of the present invention, the above-mentioned real-time monitoring of the online connection count, active connection count, and connection status of the sharded database node using the connection pool state manager of the distributed memory database, to obtain the real-time load status of the sharded database node, includes:

[0010] The connection pool load rate, connection failure rate per unit time, and average connection latency of the sharding library node are calculated based on the number of online connections, the number of effective connections, and the connection status of the sharding library node.

[0011] The real-time load status of sharded library nodes is determined based on the connection pool full load rate, connection failure rate per unit time, and average connection latency of the sharded library nodes.

[0012] According to an embodiment of the present invention, the connection pool load rate is calculated by combining the number of valid connections of the shard library node per unit time with the connection pool upper limit of the connection pool status manager.

[0013] The average connection latency is calculated by summing the query time corresponding to the number of valid connections within a preset time period and the number of valid connections.

[0014] Specifically, if the ratio of the average connection delay time of the primary sharding node to the average connection delay time of the backup sharding node in the sharding library is within a preset percentage error, it is determined that the primary sharding node and the backup sharding node in the sharding library have the same delay.

[0015] According to an embodiment of the present invention, the above-described method of selecting a service mode that matches the application scenario using a distributed in-memory database and intelligently switching the master-slave relationship of sharded nodes in the distributed in-memory database based on the selected service mode includes:

[0016] In applications where the configuration and number of primary shard nodes in a distributed in-memory database are both higher than those of backup shard nodes, the service mode should be set to guaranteed mode.

[0017] In situations where network latency and stability issues exist in the application scenario, the service mode will be set to intelligent mode;

[0018] In the application scenario where the primary sharding node and the backup sharding node in a distributed in-memory database perform data integrity and consistency checks, the service mode is set to data check mode.

[0019] When the application scenario involves users querying the validity and existence of data, the service mode should be set to sequential mode.

[0020] Based on the application scenario and the attribute information of the current service process in the distributed memory database, the system automatically adapts to the service mode through the automatic mode adaptation function, and intelligently switches the master-slave relationship of the sharded database nodes in the distributed memory database based on the selected service mode.

[0021] According to an embodiment of the present invention, in the above-described safeguard mode, when the primary sharding node fails or the connection fails, the backup sharding node is converted into the primary sharding node through the mode automatic adaptation function, and the backup sharding node provides services to the outside world.

[0022] According to an embodiment of the present invention, in the intelligent mode described above, the priority connection order of the primary sharding node and the backup sharding node in the distributed memory database is adjusted based on the information obtained by the connection pool state manager.

[0023] According to an embodiment of the present invention, the adjustment of the priority connection order of the primary sharding node and the backup sharding node in the distributed memory database includes:

[0024] Calculate the connection failure rate of the primary sharding node and the backup sharding node in the distributed memory database based on the upper limit of connection latency set by the user. Compare the connection failure rates of the primary sharding node and the backup sharding node in the distributed memory database, and select the sharding node with the lower connection failure rate as the first priority sharding node to connect to.

[0025] When the connection failure rates of the primary sharding node and the backup sharding node in the distributed memory database are the same, the sharding node with the lower average latency is selected as the second priority sharding node for connection, based on the average latency of the primary sharding node and the backup sharding node in the distributed memory database.

[0026] When the connection failure rate and average latency of the primary sharding node and the backup sharding node in the distributed memory database are the same, the sharding node with the lower load rate is selected as the third priority sharding node for connection, based on the full load rate of the primary sharding node and the backup sharding node in the distributed memory database.

[0027] In a distributed memory database, if the connection failure rate, average latency, and load factor of the primary shard node and the backup shard node are the same, the primary shard node is preferred as the fourth priority shard node for connection.

[0028] According to an embodiment of the present invention, in the above-described data inspection mode, when both the primary sharding node and the backup sharding node in the distributed memory database independently write data or when the backup sharding node is converted into the primary sharding node, data synchronization is performed on the primary sharding node and the backup sharding node.

[0029] In sequential mode, based on user query requests, a primary-backup round-robin approach is used to respond to user query requests. If the primary sharding node cannot complete the user query request, the backup sharding node is queried in turn.

[0030] According to an embodiment of the present invention, the above-mentioned automatic adaptation of the service mode through the pattern automatic adaptation function and the intelligent switching of the master-slave relationship of the sharded library nodes in the distributed memory database based on the selected service mode includes:

[0031] The service resource status of the primary sharding node and the backup sharding node in the distributed memory database is determined by the attribute information of the current service. The attribute information of the current service includes CPU utilization, memory utilization, communication rate, number of requested connections, and connection pool threshold.

[0032] When service resources are sufficient, the service mode is selected as the backup mode through the automatic mode adaptation function, and based on the backup mode, the master-slave relationship of the sharded library nodes in the distributed memory database is intelligently switched.

[0033] When the service resources are busy, the service mode is automatically adapted to the intelligent mode, and based on the intelligent mode, the master-slave relationship of the sharded library nodes in the distributed memory database is intelligently switched.

[0034] When service resources are missing, the automatic mode adaptation function selects the service mode as the backup mode, and based on the backup mode, intelligently switches the master-slave relationship of the sharded library nodes in the distributed memory database.

[0035] According to a second aspect of the present invention, an intelligent switching device for master-slave sharded database nodes in a distributed in-memory database is provided, characterized in that the device comprises:

[0036] The real-time load status acquisition module is used to monitor the number of online connections, the number of valid connections, and the connection status of sharded database nodes in real time using the connection pool status manager of the distributed memory database, so as to obtain the real-time load status of the sharded database nodes.

[0037] The service mode generation module is used to generate service modes based on the real-time load status of the sharded library nodes and / or user input, and store the configuration files corresponding to the service modes in a distributed memory database.

[0038] The master-slave relationship switching module is used to select a service mode that matches the application scenario using the distributed in-memory database, and intelligently switch the master-slave relationship of the sharded database nodes in the distributed in-memory database based on the selected service mode.

[0039] The intelligent switching method and apparatus for primary and standby sharded database nodes in a distributed in-memory database provided by this invention fully utilizes the computing resources of standby nodes by introducing an intelligent switching mechanism. It dynamically adjusts the primary and standby roles when the load is too high, achieving efficient utilization of standby nodes. Simultaneously, through dynamic load monitoring and intelligent decision-making, the method and apparatus automatically distribute query requests to nodes with lower loads when the load is too high, avoiding overload of a single node and ensuring overall system performance. Furthermore, by adjusting node roles and allocating query requests online, the method and apparatus improve the flexibility and scalability of the distributed database, enabling dynamic expansion of system resources according to business needs and supporting larger-scale data processing requirements. Additionally, in the event of node failure, the standby node can quickly take over tasks, reducing fault recovery time and improving system availability and fault tolerance. Attached Figure Description

[0040] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0041] Figure 1 This is an application scenario diagram of an intelligent switching method for master-slave sharded database nodes in a distributed in-memory database, according to an embodiment of the present invention.

[0042] Figure 2 This is a flowchart of an intelligent switching method for master-slave sharded database nodes in a distributed in-memory database, according to an embodiment of the present invention.

[0043] Figure 3 This is a schematic diagram of the primary and backup sharded library nodes of GBaseXDM according to an embodiment of the present invention.

[0044] Figure 4 This is a schematic diagram of the structure of the GBaseXDM distributed database system according to an embodiment of the present invention.

[0045] Figure 5 This is a flowchart illustrating the fragmentation library status check logic according to an embodiment of the present invention.

[0046] Figure 6 This is a flowchart of the automatic adaptation mode according to an embodiment of the present invention.

[0047] Figure 7 This is a schematic diagram of the intelligent switching device for master-slave sharded database nodes in a distributed memory database according to an embodiment of the present invention.

[0048] Figure 8 This is a block diagram of an electronic device suitable for implementing a method for intelligent switching of master and backup sharded library nodes in a distributed memory database, according to an embodiment of the present invention. Detailed Implementation

[0049] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0050] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0051] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0052] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0053] Currently, many distributed databases (including GBaseXDM) employ a master-slave architecture, with each data shard having one master node and multiple standby nodes. The master node handles data read and write operations, while the standby nodes are used for failover. However, existing systems are typically designed so that standby nodes only activate when the master node fails. The limitations of this traditional design approach are mainly reflected in the following aspects:

[0054] Low utilization of standby nodes: Standby nodes are only activated when the primary node fails, and remain idle at other times. In high-concurrency request scenarios, the computing and storage resources of standby nodes cannot be effectively utilized, causing the system to easily become a bottleneck when the request volume increases.

[0055] Load imbalance: In high-concurrency or big data query scenarios, the primary node may face excessive request pressure, while the standby node cannot actively participate in load sharing. The system cannot automatically adjust the roles of the primary and standby nodes, resulting in excessive consumption of primary node resources and the standby node failing to play its due role.

[0056] Poor scalability: The existing system cannot dynamically adjust the switching or activation strategy of primary and backup nodes according to the load. Whenever the system load increases, adding resources to the primary node or increasing the number of nodes often requires manual intervention, and it cannot intelligently scale according to load changes.

[0057] Delayed fault tolerance mechanism: The current primary-backup switchover mechanism is mainly based on fault triggering. When the primary node fails, the backup node is activated, which may lead to excessively long fault recovery time, affecting system availability and user experience.

[0058] Therefore, existing distributed database systems have shortcomings in load balancing, resource utilization, and high availability, and there is an urgent need to improve the existing master-slave node management methods to enhance the flexibility and performance of distributed database systems.

[0059] To address at least one of the problems in the prior art, this invention provides a method and apparatus for intelligent switching of primary and backup sharded database nodes in a distributed in-memory database. By dynamically adjusting the roles of primary and backup nodes and the load-sharing strategy, the method achieves load balancing and efficient resource utilization of the system, and overcomes the limitation in existing distributed database systems where primary and backup nodes can only switch in case of failure.

[0060] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0061] Figure 1 This is an application scenario diagram of an intelligent switching method for master-slave sharded database nodes in a distributed in-memory database, according to an embodiment of the present invention.

[0062] like Figure 1 As shown, application scenario 100 according to this embodiment may include the field of database technology. Network 104 is used as a medium to provide a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0063] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0064] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0065] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0066] It should be noted that the intelligent switching method for primary and secondary sharded database nodes in a distributed memory database provided in this embodiment of the invention can generally be executed by server 105. Correspondingly, the intelligent switching device for primary and secondary sharded database nodes in a distributed memory database provided in this embodiment of the invention can generally be located in server 105. The intelligent switching method for primary and secondary sharded database nodes in a distributed memory database provided in this embodiment of the invention can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the intelligent switching device for primary and secondary sharded database nodes in a distributed memory database provided in this embodiment of the invention can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0067] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0068] The following will be based on Figure 1 The described scene, through Figures 2-6 The intelligent switching method for master-slave sharded database nodes in a distributed in-memory database, according to the disclosed embodiments, is described in detail.

[0069] Figure 2 This is a flowchart of an intelligent switching method for master-slave sharded database nodes in a distributed in-memory database, according to an embodiment of the present invention.

[0070] like Figure 2 As shown, the intelligent switching method for primary and backup sharded database nodes in this embodiment includes operations S210 to S230.

[0071] When operating S210, the connection pool state manager of the distributed memory database is used to monitor the number of online connections, the number of active connections, and the connection status of the sharded database nodes in real time, so as to obtain the real-time load status of the sharded database nodes.

[0072] Those skilled in the art can select other attributes that can characterize the load status for real-time monitoring based on actual needs.

[0073] When operating S220, a service mode is generated based on the real-time load status of the sharded library nodes and / or user input, and the configuration file corresponding to the service mode is stored in a distributed memory database.

[0074] When operating the S230, a service mode that matches the application scenario is selected using a distributed in-memory database, and based on the selected service mode, the master-slave relationship of the sharded database nodes in the distributed in-memory database is intelligently switched.

[0075] The intelligent switching method and apparatus for primary and standby sharded database nodes in a distributed in-memory database provided by this invention fully utilizes the computing resources of standby nodes by introducing an intelligent switching mechanism. It dynamically adjusts the primary and standby roles when the load is too high, achieving efficient utilization of standby nodes. Simultaneously, through dynamic load monitoring and intelligent decision-making, the method and apparatus automatically distribute query requests to nodes with lower loads when the load is too high, avoiding overload of a single node and ensuring overall system performance. Furthermore, by adjusting node roles and allocating query requests online, the method and apparatus improve the flexibility and scalability of the distributed database, enabling dynamic expansion of system resources according to business needs and supporting larger-scale data processing requirements. Additionally, in the event of node failure, the standby node can quickly take over tasks, reducing fault recovery time and improving system availability and fault tolerance.

[0076] The following section uses GBaseXDM as a specific implementation method and combines it with the appendix. Figure 3 and 4 The present invention provides a more detailed description of the intelligent switching method for master-slave sharded database nodes in a distributed in-memory database.

[0077] Figure 3 This is a schematic diagram of the primary and backup sharded library nodes of GBaseXDM according to an embodiment of the present invention.

[0078] Figure 4 This is a schematic diagram of the structure of the GBaseXDM distributed database system according to an embodiment of the present invention.

[0079] like Figure 3 As shown, the purpose of this invention is to achieve intelligent switching between primary and backup sharded database nodes in a distributed memory database. Figure 3 This demonstrates how the GBaseXDM distributed database achieves high availability, high reliability, and efficient data processing through a master-slave backup architecture and data sharding technology. Figure 3 The main database is indicated in the middle (e.g.) Figure 3 The GBaseXDM data sharding database A (primary) and GBaseXDM data sharding database N (primary) shown, along with their backup databases (such as...), are shown. Figure 3 The diagram shows GBaseXDM data sharding databases A (backup 1), A (backup 2), N (backup 1), and N (backup 2). The primary database is used for normal operation and data writing, while the backup databases are used for data redundancy and disaster recovery. Data sharding databases A and N represent different data shards, each with its corresponding primary and backup databases. High availability and reliability: By setting up backup databases, a rapid switch to the backup database is possible in the event of a primary database failure, ensuring business continuity and improving system reliability and availability.

[0080] like Figure 4 As shown, the GBaseXDM distributed database system, during the intelligent failover of primary and secondary sharded database nodes, needs to utilize the connection pool state manager, which belongs to the load management system, to allocate and manage connections to the primary and secondary sharded database nodes. The connection pool state manager is responsible for managing and monitoring the load of the entire system, including the status of the connection pool and the allocation and management of connections. Figure 4 There are multiple connection pools, corresponding to the connection management of the primary and secondary databases respectively. The connection pools are responsible for managing and maintaining the pooling of database connections to improve the efficiency and performance of database operations. The data sharding database includes the primary and secondary databases, labeled A and N respectively (e.g., ...). Figure 4 The diagram shows GBaseXDM data sharding database A (primary), GBaseXDM data sharding database N (primary), GBaseXDM data sharding database A (backup), and GBaseXDM data sharding database N (backup). Figure 4 As shown, these libraries interact with the load management system through a connection pool. Figure 4Below is a list of servers connected to the various data shard databases; these servers are responsible for specific business applications and data operations. From... Figure 4 As can be seen, the GBaseXDM distributed database system uses a load management system and a connection pool to efficiently manage and allocate database connections, thereby achieving efficient intelligent switching between primary and backup sharded database nodes.

[0081] According to an embodiment of the present invention, the above-mentioned real-time monitoring of the number of online connections, the number of effective connections, and the connection status of the sharded database node using the connection pool state manager of the distributed memory database to obtain the real-time load status of the sharded database node includes: calculating the connection pool full load rate, the connection failure rate per unit time, and the average connection latency time of the sharded database node based on the number of online connections, the number of effective connections, and the connection status of the sharded database node; and determining the real-time load status of the sharded database node based on the connection pool full load rate, the connection failure rate per unit time, and the average connection latency time of the sharded database node.

[0082] The above embodiments dynamically monitor key metrics such as connection pool load rate, failure rate, and latency in real time, providing feedback on node load status to ensure the system can quickly respond to load fluctuations and avoid performance bottlenecks caused by latency or misjudgment. Furthermore, these embodiments integrate multi-dimensional data such as connection count, failure rate, and latency, avoiding the limitations of single metrics (such as CPU / memory alone) and providing a more comprehensive reflection of the true node load, thus improving decision-making accuracy. Simultaneously, through quantitative monitoring and dynamic adjustment, efficient load balancing, fault prevention, and resource optimization are achieved in the distributed system, significantly improving stability and performance.

[0083] According to an embodiment of the present invention, the connection pool load rate is calculated by combining the number of valid connections of the sharded database nodes per unit time with the connection pool upper limit of the connection pool status manager; wherein, the average connection latency time is calculated by combining the total query time corresponding to the number of valid connections within a preset time period with the number of valid connections; wherein, if the ratio of the average connection latency time of the primary sharded database node to the average connection latency time of the backup sharded database node is within a preset percentage error, it is determined that the primary sharded database node and the backup sharded database node have the same latency.

[0084] The following describes specific implementation methods in conjunction with appendices. Figure 5 Further improvements are made to the fragmented library status check logic provided by this invention.

[0085] Figure 5 This is a flowchart illustrating the fragmentation library status check logic according to an embodiment of the present invention.

[0086] like Figure 5As shown, the connection count check is initiated by the connection pool state manager. Taking GBaseXDM as an example, the primary and backup sharded database nodes are checked simultaneously (e.g., ...). Figure 5 The connection pools of GBaseXDM data sharding database A (primary) and GBaseXDM data sharding database A (backup) are shown to obtain the total number of connections, valid connections, and failed connections of the primary and backup sharding database nodes, thereby determining the lifecycle of the primary and backup sharding database nodes. Meanwhile, as shown... Figure 5 As shown, the connection pool state manager calculates the full load rate, connection failure rate per unit time, and average connection latency of the connection pool, thereby determining the priority connection order of each primary and backup sharding library node in GBaseXDM, and saving the connection status to the connection pool state manager.

[0087] The connection pool load factor is calculated as follows: (Number of valid connections / Connection pool maximum limit) * 100%.

[0088] Connection failure rate within a specified statistical period:

[0089] t: The statistical time unit is configured to be 60 seconds past the current time;

[0090] c: Total number of connections per unit time;

[0091] f: Total number of connection failures per unit time;

[0092] Failure rate = (f / c) * 100%.

[0093] Average latency = Total query time for each valid connection within the specified statistical period / Number of valid connections within the specified statistical period. A difference of less than 10% in the average latency between primary and backup services is considered consistent latency.

[0094] The collected information will be stored in the storage structure of dedicated storage data shard A in the connection pool state manager. Data shard N is logically identical to data shard A.

[0095] According to embodiments of the present invention, the above-described method of selecting a service mode that matches the application scenario using a distributed in-memory database and intelligently switching the master-slave relationship of shard nodes in the distributed in-memory database based on the selected service mode includes: setting the service mode to a guaranteed mode when the configuration and number of master shard nodes in the distributed in-memory database are both higher than those of slave shard nodes; setting the service mode to an intelligent mode when network latency and stability issues exist in the application scenario; setting the service mode to a data check mode when the master and slave shard nodes in the distributed in-memory database are performing data integrity and consistency checks; setting the service mode to a sequential mode when users are querying data validity and existence; and automatically adapting the service mode based on the application scenario and the attribute information of the current service process in the distributed in-memory database using an automatic mode adaptation function, and intelligently switching the master-slave relationship of shard nodes in the distributed in-memory database based on the selected service mode.

[0096] Due to the complexity and diversity of field applications, some scenarios require efficient data querying, others demand strict data consistency, and still others prioritize high availability and stability. Based on actual field needs, real-time monitoring data is used to assess the number and load of the "data sharding database-master" and flexibly configure master-slave service usage strategies. The strategy switching module is user-defined and written into a configuration file, according to field requirements. Additionally, users can configure an automatic mode adaptation mechanism.

[0097] The above embodiments offer strong scenario-adaptive flexibility: They automatically adapt service modes based on different application scenarios (such as high-configuration master nodes, network latency, data verification, and user query needs), ensuring a high degree of matching between system configuration and business requirements, avoiding resource waste from a "one-size-fits-all" approach. Furthermore, the above implementations enable intelligent master-slave failover to improve reliability: Based on mode selection (such as prioritizing master nodes in the protection mode and optimizing network fault tolerance in the intelligent mode), the master-slave relationship is dynamically adjusted, enhancing system disaster recovery capabilities, reducing the risk of single points of failure, and ensuring business continuity and high data availability. Simultaneously, the above embodiments optimize resource utilization: Targeted modes (such as master node expansion in the protection mode and query path optimization in the sequential mode) allocate master-slave resources on demand, avoiding redundant overhead and improving the throughput and response speed of the distributed in-memory database.

[0098] According to an embodiment of the present invention, in the above-described safeguard mode, when the primary sharding node fails or the connection fails, the backup sharding node is converted into the primary sharding node through the mode automatic adaptation function, and the backup sharding node provides services to the outside world.

[0099] Assurance Mode: When the number of "Data Shard Database - Primary" servers at the user's site is large and the configuration performance is high, and the "Data Shard Database - Backup" service is small and only used for supplementary purposes, the configuration is set to "Assurance Mode". In this mode, the "Data Shard Database - Primary" service will provide services to the outside world. When the connection status manager of the distributed database system detects that the primary service has failed or the connection has failed, it will switch to the backup data shard database and provide services to the outside world.

[0100] According to an embodiment of the present invention, in the intelligent mode described above, the priority connection order of the primary sharding node and the backup sharding node in the distributed memory database is adjusted based on the information obtained by the connection pool state manager.

[0101] According to an embodiment of the present invention, the adjustment of the priority connection order of the primary shard node and the backup shard node in the distributed memory database includes: calculating the connection failure rate of the primary shard node and the backup shard node in the distributed memory database based on the user-set connection latency time limit; comparing the connection failure rates of the primary shard node and the backup shard node in the distributed memory database; and selecting the shard node with the lower connection failure rate as the first priority shard node to connect to; if the connection failure rates of the primary shard node and the backup shard node in the distributed memory database are the same, comparing the connection failure rates of the primary shard node and the backup shard node in the distributed memory database... Based on the average latency of the shard nodes, the shard node with the lowest average latency is selected as the second-highest priority for connection. If the connection failure rate and average latency of the primary and backup shard nodes in the distributed memory database are the same, the shard node with the lowest load rate is selected as the third-highest priority for connection. If the connection failure rate, average latency, and load rate of the primary and backup shard nodes are the same, the primary shard node is selected as the fourth-highest priority for connection.

[0102] Smart Mode: When the user's application scenario has high network latency and poor stability, it is recommended to set it to smart switching mode. When the user configures "Smart Mode":

[0103] Priority connection order (based on information collected by the connection pool state manager):

[0104] (1) First, compare the failure rates. Services with lower failure rates have the highest priority. In addition, connections with a delay time exceeding the user-set limit are classified as connection failures.

[0105] (2) Compare the average latency time. When the failure rate is the same, compare the average latency time and give priority to the connection with the lower latency time.

[0106] (3) Compare the full load rate. When the failure rate and average latency are the same, prioritize the service with the lower full load rate for connection.

[0107] (4) When the load rate, average latency, and failure rate are the same, the primary service should be prioritized for connection.

[0108] According to an embodiment of the present invention, in the data inspection mode described above, when both the primary sharding node and the backup sharding node in the distributed memory database independently write data or when the backup sharding node is converted into the primary sharding node, data synchronization is performed on the primary sharding node and the backup sharding node; wherein, in the sequential mode, based on the user query request, a primary-backup round-robin method is used to respond to the user query request, and if the primary sharding node cannot complete the user query request, the backup sharding node is queried sequentially.

[0109] Data Inspection Mode: When users have strict requirements for the integrity and consistency of data between the "Data Shard Database - Primary" and "Data Shard Database - Backup," such as in primary / backup data integrity inspection scenarios or database migration / backup scenarios, configuring "Data Inspection Mode" will follow rigorous data consistency logic. It will query data from both the primary and backup services separately and compare the data. If inconsistencies or missing data are found, a query error message will be returned. This mode collects difference information between the two databases to determine their data synchronization status and, in database migration / backup scenarios, to determine the data difference between the backup and primary databases.

[0110] A typical application scenario of the above embodiments is that data writing to the database and data querying are two isolated scenarios, and data is written to the primary and backup databases respectively through two independent write services. Therefore, whether the data in the primary and backup databases is consistent or whether there are updates between the old and new data can be verified through consistency queries. Another typical scenario is when the backup database is moved or an old backup database is moved online to form a new primary and backup service, the difference between the old and new data can be determined through consistency checks.

[0111] Sequential Mode: When the user application scenario is relatively healthy (high server configuration, high network speed) and there are certain requirements for the validity and existence of the queried data, configuring "Sequential Mode" will follow a round-robin pattern between the "Data Shard Database - Primary" and "Data Shard Database - Backup" servers. If the query request cannot find the requested data in the "Data Shard Database - Primary", it will sequentially poll Backup Service 1, Backup Service 2, etc. Once the data is found, it will be returned immediately; otherwise, a query failure will be returned.

[0112] According to embodiments of the present invention, the above-mentioned automatic adaptation of the service mode through the mode automatic adaptation function and the intelligent switching of the master-slave relationship of sharded nodes in the distributed memory database based on the selected service mode includes: determining the service resource status of the master sharded node and the standby sharded node in the distributed memory database through the attribute information of the current service, wherein the attribute information of the current service includes CPU utilization, memory utilization, communication rate, number of requested connections, and connection pool threshold; when the service resource status is sufficient, selecting the guaranteed mode as the service mode through the mode automatic adaptation function, and intelligently switching the master-slave relationship of sharded nodes in the distributed memory database based on the guaranteed mode; when the service resource status is busy, selecting the intelligent mode as the service mode through the mode automatic adaptation function, and intelligently switching the master-slave relationship of sharded nodes in the distributed memory database based on the intelligent mode; when the service resource status is lacking, selecting the guaranteed mode as the service mode through the mode automatic adaptation function, and intelligently switching the master-slave relationship of sharded nodes in the distributed memory database based on the guaranteed mode.

[0113] The following describes specific implementation methods in conjunction with appendices. Figure 6 The automatic adaptation mode provided by this invention will be further explained.

[0114] Figure 6 This is a flowchart of the automatic adaptation mode according to an embodiment of the present invention.

[0115] like Figure 6 As shown, when the GBaseXDM distributed database is in auto-adaptation mode, the GBaseXDM distributed database service will automatically check the server environment and communication status with the database data nodes immediately after startup (i.e., Figure 6 The server resource check shown is performed by collecting data on the current service process's CPU (Central Processing Unit) utilization, memory utilization, CPU and memory resource availability, number of requested connections, and connection pool threshold. When the server configuration is high and the network communication rate meets requirements, the default guaranteed mode (i.e., ...) will be used. Figure 6 If the CPU utilization of the current process (or the currently running process) is <5%, the system memory free value is >40%, the preset data buffer limit is > the memory usage of the current process is >100MB, and meeting these conditions indicates sufficient server resources, then the guaranteed mode will be enabled when QPS (Queries per second) <1000 and the number of worker threads is <60% of the connection pool threshold. During operation, if the distributed database service connection load is too high (QPS > 100, connection request volume exceeds 60% of the connection pool) but server resources are sufficient, it will switch to intelligent mode (i.e., ...). Figure 6The process's CPU usage exceeds 20% and increases by more than 50%, but the preset data buffer value is less than the process's memory usage, indicating stable memory usage. This suggests the server's CPU resources are busy but memory is sufficient. Therefore, in cases where QPS > 1000 and the number of worker threads > 60% of the connection pool threshold, a smart mode switching mechanism is activated. This mechanism promotes a standby node to the new master node or distributes query requests to the standby node to balance the load across nodes. However, when the distributed database server experiences memory and CPU resource constraints, it will switch to a guaranteed mode (i.e., ...). Figure 6 The data shows that the CPU usage of this process exceeds 30% and the increase exceeds 50%, and the remaining server memory resources are <10%. The preset data buffer value is ≤ the memory of this process. This indicates that the current database system is busy and memory resources are insufficient. It is necessary to sacrifice response efficiency to maintain system stability.

[0116] The intelligent failover method for primary and secondary sharded database nodes in a distributed in-memory database provided by this invention can dynamically allocate query requests: In intelligent mode, the system intelligently distributes query requests to nodes with lower loads based on the current load of each node, avoiding overload of a single node and ensuring query efficiency. It also provides high availability: By dynamically adjusting the roles of primary and secondary nodes and implementing load balancing, even if a node fails or becomes unavailable, the backup node can quickly take over the tasks and maintain stable system operation. Furthermore, it offers flexible configuration: This invention also provides flexible configuration and control interfaces, allowing system administrators to adjust node failover strategies, load balancing algorithms, and other configurations as needed, with real-time effects to ensure the system makes optimal adjustments based on real-time load.

[0117] Figure 7 This is a schematic diagram of the intelligent switching device for master-slave sharded database nodes in a distributed memory database according to an embodiment of the present invention.

[0118] like Figure 7 As shown, the intelligent switching device 700 for primary and secondary sharded database nodes in a distributed memory database includes a real-time load status acquisition module 710, a service mode generation module 720, and a primary / secondary relationship switching module 730.

[0119] The real-time load status acquisition module 710 is used to monitor the number of online connections, the number of valid connections, and the connection status of the sharded database nodes in real time using the connection pool status manager of the distributed memory database, so as to obtain the real-time load status of the sharded database nodes. In one embodiment, the real-time load status acquisition module 710 can be used to perform the operation S210 described above, which will not be repeated here.

[0120] The service mode generation module 720 is used to generate service modes based on the real-time load status of the sharded library nodes and / or user input, and store the configuration file corresponding to the service mode in a distributed memory database. In one embodiment, the service mode generation module 720 can be used to perform the operation S220 described above, which will not be repeated here.

[0121] The master-slave relationship switching module 730 is used to select a service mode that matches the application scenario using the distributed memory database, and intelligently switch the master-slave relationship of the sharded database nodes in the distributed memory database based on the selected service mode. In one embodiment, the master-slave relationship switching module 730 can be used to execute the operation S230 described above, which will not be repeated here.

[0122] According to embodiments of the present invention, any plurality of modules among the real-time load status acquisition module 710, service mode generation module 720, and primary / standby relationship switching module 730 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of the present invention, at least one of the real-time load status acquisition module 710, service mode generation module 720, and primary / standby relationship switching module 730 can be at least partially implemented as hardware circuitry, such as field-programmable gate array (FPGA), programmable logic array (PLA), system-on-a-chip, system-on-a-substrate, system-on-package, application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in hardware or firmware, or in any one of software, hardware, and firmware implementations or a suitable combination of any of these. Alternatively, at least one of the real-time load status acquisition module 710, service mode generation module 720, and primary / standby relationship switching module 730 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0123] Figure 8 This is a block diagram of an electronic device suitable for implementing a method for intelligent switching of master and backup sharded library nodes in a distributed memory database, according to an embodiment of the present invention.

[0124] like Figure 8As shown, an electronic device 800 according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0125] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0126] According to an embodiment of the present invention, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the input / output (I / O) interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0127] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0128] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, a computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0130] Those skilled in the art will understand that the features described in the various embodiments of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments of the present invention can be combined and / or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0131] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A method for intelligent failover of master-slave sharded database nodes in a distributed in-memory database, characterized in that, The method includes: The connection pool state manager of the distributed memory database is used to monitor the number of online connections, the number of valid connections, and the connection status of the sharded database nodes in real time, so as to obtain the real-time load status of the sharded database nodes. Based on the real-time load status of the sharded library nodes and / or the service mode input by the user, a service mode is generated, and the configuration file corresponding to the service mode is stored in the distributed memory database; The distributed in-memory database is used to select a service mode that matches the application scenario. Based on the selected service mode, the primary-backup relationship of the sharding nodes in the distributed in-memory database is intelligently switched. This includes: setting the service mode to a guaranteed mode when the configuration and number of primary sharding nodes in the distributed in-memory database are both higher than those of backup sharding nodes; setting the service mode to an intelligent mode when network latency and stability issues exist in the application scenario; setting the service mode to a data checking mode when the primary and backup sharding nodes in the distributed in-memory database are performing data integrity and consistency checks; and setting the service mode to a sequential mode when users are querying data validity and existence. Based on the application scenario and the attribute information of the current service process in the distributed in-memory database, the service mode is automatically adapted using an automatic mode adaptation function, and the primary-backup relationship of the sharding nodes in the distributed in-memory database is intelligently switched based on the selected service mode.

2. The method according to claim 1, characterized in that, The connection pool state manager of the distributed in-memory database is used to monitor the number of online connections, the number of active connections, and the connection status of the sharded database nodes in real time, and the real-time load status of the sharded database nodes is obtained, including: Based on the number of online connections, the number of valid connections, and the connection status of the sharding library node, calculate the connection pool full load rate, the connection failure rate per unit time, and the average connection latency time of the sharding library node. The real-time load status of the sharding library node is determined based on the connection pool full load rate, connection failure rate per unit time, and average connection latency of the sharding library node.

3. The method according to claim 2, characterized in that, The connection pool load factor is calculated by combining the number of valid connections to the sharded library node per unit time with the connection pool upper limit of the connection pool status manager. The average connection latency is calculated by summing the query time corresponding to the number of valid connections within a preset time period and the number of valid connections. Specifically, if the ratio of the average connection delay time of the primary sharding node to the average connection delay time of the backup sharding node in the sharding library is within a preset percentage error, it is determined that the primary sharding node and the backup sharding node in the sharding library have the same delay.

4. The method according to claim 1, characterized in that, In the protection mode, when the primary sharding node fails or the connection fails, the backup sharding node is converted into the primary sharding node through the mode automatic adaptation function, and the backup sharding node provides services to the outside world.

5. The method according to claim 1, characterized in that, In the intelligent mode, the priority connection order of the primary sharding node and the backup sharding node in the distributed memory database is adjusted based on the information obtained by the connection pool state manager.

6. The method according to claim 5, characterized in that, Adjusting the priority connection order of the primary shard node and the backup shard node in the distributed memory database includes: The connection failure rate of the primary sharding node and the backup sharding node in the distributed memory database is calculated based on the connection latency time limit set by the user. The connection failure rates of the primary sharding node and the backup sharding node in the distributed memory database are compared, and the sharding node with the lower connection failure rate is selected as the first priority sharding node to connect to. If the connection failure rates of the primary sharding node and the backup sharding node in the distributed memory database are the same, the average latency of the primary sharding node and the backup sharding node in the distributed memory database is compared, and the sharding node with the lower average latency is selected as the second priority sharding node for connection. When the connection failure rate and average latency of the primary sharding node and the backup sharding node in the distributed memory database are the same, the full load rate of the primary sharding node and the backup sharding node in the distributed memory database is compared, and the sharding node with the lower full load rate is selected as the third priority sharding node for connection. If the connection failure rate, average latency, and load factor of the primary sharding node and the backup sharding node in the distributed memory database are the same, the primary sharding node is preferentially selected as the fourth preferred sharding node for connection.

7. The method according to claim 1, characterized in that, In the data inspection mode, when both the primary sharding node and the backup sharding node in the distributed memory database write data independently or when the backup sharding node is converted into the primary sharding node, data synchronization is performed on the primary sharding node and the backup sharding node. In the sequential mode, based on the user query request, a primary-backup round-robin method is used to respond to the user query request. If the primary sharding node cannot complete the user query request, the backup sharding node is queried in turn.

8. The method according to claim 1, characterized in that, The automatic mode adaptation function automatically adapts to the service mode, and intelligently switches the master-slave relationship of the sharded library nodes in the distributed memory database based on the selected service mode, including: The service resource status of the primary sharding node and the backup sharding node in the distributed memory database is determined by the attribute information of the current service, wherein the attribute information of the current service includes CPU utilization, memory utilization, communication rate, number of requested connections and connection pool threshold. When the service resources are sufficient, the service mode is selected as the protection mode through the automatic mode adaptation function, and the master-slave relationship of the sharded library nodes in the distributed memory database is intelligently switched based on the protection mode. When the service resource status is busy, the service mode is selected as the intelligent mode through the automatic mode adaptation function, and based on the intelligent mode, the master-slave relationship of the sharded library nodes in the distributed memory database is intelligently switched. In the event that the service resource status is missing, the service mode is selected as the backup mode through the automatic mode adaptation function, and based on the backup mode, the master-slave relationship of the sharded library nodes in the distributed memory database is intelligently switched.

9. An intelligent switching device for master-slave sharded database nodes in a distributed in-memory database, characterized in that, The device includes: The real-time load status acquisition module is used to monitor the number of online connections, the number of valid connections, and the connection status of the sharded database nodes in real time using the connection pool status manager of the distributed memory database, so as to obtain the real-time load status of the sharded database nodes. The service mode generation module is used to generate service modes based on the real-time load status of the sharding library nodes and / or the business input by the user, and store the configuration file corresponding to the service mode in the distributed memory database; The master-slave relationship switching module is used to select a service mode that matches the application scenario using the distributed memory database, and intelligently switch the master-slave relationship of the sharded database nodes in the distributed memory database based on the selected service mode. Specifically, the distributed in-memory database is used to select a service mode that matches the application scenario. Based on the selected service mode, the primary / backup relationship of the sharding nodes in the distributed in-memory database is intelligently switched. This includes: setting the service mode to a guaranteed mode when the configuration and number of primary sharding nodes in the distributed in-memory database are both higher than those of backup sharding nodes; setting the service mode to an intelligent mode when network latency and stability issues exist in the application scenario; setting the service mode to a data checking mode when the primary and backup sharding nodes in the distributed in-memory database are performing data integrity and consistency checks; setting the service mode to a sequential mode when users are querying data validity and existence; and automatically adapting the service mode based on the application scenario and the attribute information of the current service process in the distributed in-memory database using an automatic mode adaptation function, and intelligently switching the primary / backup relationship of the sharding nodes in the distributed in-memory database based on the selected service mode.

Images