A method and system for dynamic multi-node switching in MongoDB

Abstract

This application provides a method and system for dynamic multi-node switching of MongoDB. The method includes: acquiring configuration information of multiple MongoDB nodes; establishing a data access layer based on the configuration information; acquiring metric data of multiple operational metrics for each MongoDB node through the data access layer, according to the hierarchical frequency configured for each MongoDB node; converting node performance metrics into weight values ​​for the MongoDB nodes by combining preset node weight values; determining a request distribution strategy based on the connection pool management mechanism and node availability status data, whereby the request distribution strategy indicates the proportion of MongoDB requests allocated to the MongoDB nodes; and performing status detection on the MongoDB nodes through the data access layer, marking the detected MongoDB node as unavailable and updating the node availability status data when an abnormal state condition is met. This application solves the problems of existing technologies requiring manual intervention, inability to perceive real-time node load, and high code coupling.

Description

Technical Field

[0001] This invention relates to the field of database technology, and specifically to a method and system for dynamic switching of multiple nodes in MongoDB. Background Technology

[0002] MongoDB, a popular NoSQL database, is widely used in distributed systems and big data applications. As business scales, a single MongoDB node struggles to meet high-concurrency access demands, making multi-node deployment a common solution. In read-only scenarios, deploying multiple MongoDB nodes for load balancing can effectively improve overall system performance and availability.

[0003] Traditional MongoDB multi-node deployment solutions typically employ simple round-robin or fixed-weight load balancing strategies. For example, some systems explicitly create multiple database connections in the application code and then distribute requests among these connections using a round-robin approach; other solutions implement simple request forwarding through an intermediate proxy layer, but lack the ability to perceive the real-time status of nodes.

[0004] Currently, a more advanced approach is to implement multi-node connection management for MongoDB at the application level. This involves explicitly creating and maintaining multiple database connections and using round-robin or simple weighted algorithms for request distribution. However, this approach has significant limitations: First, it requires embedding connection management logic directly into the business logic, leading to high code coupling and difficult maintenance later on. Second, the simple round-robin method cannot perceive the actual load status of each node, potentially causing high-load nodes to continuously receive requests and resulting in performance degradation. Finally, node failures require manual intervention, involving taking the failed node offline and manually bringing it back online after recovery, increasing operational complexity and fault response time.

[0005] Consequently, these issues make it difficult for the system to achieve true elastic scaling and high availability, especially under high load or uneven node performance, resulting in limited overall system throughput and a degraded user experience. At the same time, frequent manual intervention increases operational costs and reduces the system's automation level. Summary of the Invention

[0006] In view of this, the embodiments of this application provide a method and system for dynamic switching of multiple nodes in MongoDB, which solves the problems of manual intervention, inability to perceive the real-time load status of nodes, and high code coupling in the prior art.

[0007] In a first aspect, embodiments of this application provide a method for dynamic multi-node switching in MongoDB, including:

[0008] Obtain configuration information for multiple MongoDB nodes, establish a data access layer based on the configuration information, and configure a connection pool management mechanism within the data access layer to indicate the management connection pool;

[0009] Through the data access layer, the MongoDB node acquires metric data about multiple operating metrics by collecting tasks at the hierarchical frequency configured for each MongoDB node. The collection tasks are used to indicate the multiple collection frequencies executed by the MongoDB node and the operating metrics to be collected at each collection frequency. The operating metrics include CPU utilization, memory utilization, IOPS, disk wait time, and network latency.

[0010] The metric data of the MongoDB nodes are aggregated to obtain the node performance metrics for each MongoDB node.

[0011] Based on the obtained preset values ​​of the node weights of the MongoDB nodes, the node performance metrics of the MongoDB nodes are converted into the weight values ​​of the MongoDB nodes.

[0012] Based on the connection pool management mechanism and the obtained node availability status data of the MongoDB node, a request distribution strategy is determined according to the weight value of the MongoDB node. The request distribution strategy is used to indicate the distribution ratio of MongoDB requests allocated to the MongoDB node.

[0013] The data access layer performs status detection on the MongoDB node. When the MongoDB node is detected to meet the preset abnormal status conditions, the detected MongoDB node is marked as unavailable and the node availability status data is updated.

[0014] As an improvement to the above solution, the step of obtaining configuration information of multiple MongoDB nodes and establishing a data access layer based on the configuration information includes:

[0015] Based on the interface specifications of the native driver of the MongoDB node and the business requirements of multiple MongoDB nodes, the overall architecture of the data access layer is determined.

[0016] Based on the overall architecture of the data access layer and the connection string, maximum number of connections, connection timeout time and idle connection hold time in the configuration information of the MongoDB node, a corresponding connection pool is created for each MongoDB node, and a connection pool management mechanism is configured.

[0017] A data access layer is established based on the connection pool management mechanism and the overall architecture of the data access layer.

[0018] As an improvement to the above scheme, the hierarchical frequency acquisition task includes a high-frequency indicator acquisition task and a low-frequency indicator acquisition task; the step of obtaining indicator data of the MongoDB node regarding multiple operational indicators through the data access layer according to the hierarchical frequency acquisition task configured for each MongoDB node includes:

[0019] Based on the aforementioned operational metrics, high-frequency metric collection tasks and low-frequency metric collection tasks are determined to obtain tiered frequency collection tasks. A corresponding tiered frequency collection task is configured for each MongoDB node. The high-frequency metric collection task is used to indicate a first collection frequency for high-frequency metrics, and the low-frequency metric collection task is used to indicate a second collection frequency for low-frequency metrics. The first collection frequency is greater than the second collection frequency. The high-frequency metrics include the CPU utilization and the memory utilization, and the low-frequency metrics include the IOPS, the disk wait time, and the network latency.

[0020] Based on the high-frequency indicator collection task and the connection pool management mechanism, a high-frequency collection thread is established for each MongoDB node, and the high-frequency indicator collection task is executed according to the high-frequency collection thread to obtain the high-frequency indicator data.

[0021] Based on the low-frequency indicator collection task and the connection pool management mechanism, a low-frequency collection thread is established for each MongoDB node, and the low-frequency indicator collection task is executed according to the low-frequency collection thread to obtain the low-frequency indicator data.

[0022] Configure a timeout mechanism and a retry strategy for the high-frequency index data and the low-frequency index data. The timeout mechanism is used to indicate that the current hierarchical frequency acquisition task is marked as acquisition failure when the data acquisition time exceeds a preset acquisition time threshold. The retry strategy is used to indicate that the corresponding hierarchical frequency acquisition task is re-executed when acquisition failure is detected.

[0023] As an improvement to the above solution, the aggregation of the MongoDB node's metrics data to obtain the node performance metrics for each MongoDB node includes:

[0024] Based on the high-frequency indicator collection task, a high-frequency indicator window is configured for the high-frequency indicator, and the average value of the high-frequency indicator data collected within the high-frequency indicator window is calculated to obtain the average value of the high-frequency indicator.

[0025] Based on the low-frequency index collection task, a low-frequency index window is configured for the low-frequency index, and the average value of the low-frequency index data collected within the low-frequency index window is calculated to obtain the average value of the low-frequency index.

[0026] When missing data is detected in the high-frequency indicator window and / or the low-frequency indicator window, the missing data is filled with adjacent data or the missing data is filled by interpolation.

[0027] Window data aggregation is performed on the average of the high-frequency indicators, the average of the low-frequency indicators, and the filled data to obtain the node performance indicators.

[0028] As an improvement to the above solution, the step of converting the node performance metrics of the MongoDB node into the weight value of the MongoDB node by combining the obtained preset node weight value of the MongoDB node includes:

[0029] When the CPU utilization index is lower than a preset first CPU utilization threshold, the CPU penalty factor is determined as the first CPU penalty factor.

[0030] Alternatively, when the CPU utilization index data exceeds the first CPU utilization threshold but is lower than the preset second CPU utilization threshold, the CPU penalty factor is determined as the second CPU penalty factor, wherein the first CPU utilization threshold is less than the second CPU utilization threshold, and the second CPU penalty factor is greater than the first CPU penalty factor.

[0031] Alternatively, when the CPU utilization index data exceeds the second CPU utilization threshold, the CPU penalty factor is determined as the third CPU penalty factor, wherein the third CPU penalty factor is greater than the second CPU penalty factor.

[0032] Based on the memory usage rate index data and the preset memory usage rate threshold, a memory usage rate penalty factor is determined;

[0033] Based on the IOPS metric data and the preset IOPS threshold, an IOPS penalty factor is determined;

[0034] Based on the disk wait time metric data and the preset disk wait time threshold, a disk wait time penalty factor is determined;

[0035] Based on the network latency index data and the preset network latency threshold, a network latency penalty factor is determined;

[0036] The weight value of the MongoDB node is calculated based on the preset node weight value of the MongoDB node, the determined CPU penalty factor, the memory utilization penalty factor, the IOPS penalty factor, the disk wait time penalty factor, and the network latency penalty factor.

[0037] As an improvement to the above solution, the step of determining the request distribution strategy based on the connection pool management mechanism and the obtained node availability status data of the MongoDB nodes, according to the weight value of the MongoDB nodes, includes:

[0038] Based on the weight value of the MongoDB node and the node availability status data of the MongoDB node, the selection probability of the MongoDB node is calculated, and a target MongoDB node is determined according to the selection probability using a preset random number generator, so as to generate a request distribution strategy based on the target MongoDB node.

[0039] If the identified target MongoDB node fails to respond to the MongoDB request, then based on the node availability status data, a MongoDB node marked as available is selected to respond to the MongoDB request.

[0040] As an improvement to the above solution, the step of determining the request distribution strategy based on the connection pool management mechanism and the obtained node availability status data of the MongoDB nodes, according to the weight value of the MongoDB nodes, further includes:

[0041] Based on the weight values, performance metrics, availability status data, and connection relationships of the MongoDB nodes, a MongoDB connection network graph is constructed. The connection relationships indicate the data dependencies between the MongoDB nodes. The vertices in the MongoDB connection network graph represent the MongoDB nodes, and the vertex weights of the vertices are determined by the performance metrics. The edges in the MongoDB connection network graph represent the connection relationships between the MongoDB nodes.

[0042] Based on the MongoDB connection network graph, an NMC sparser model is constructed, and the minimum cut set in the MongoDB connection network graph is identified according to the NMC sparser model. The minimum cut set is used to indicate MongoDB nodes whose node load exceeds a preset load threshold and / or MongoDB nodes whose network connection stability is lower than a preset stability threshold.

[0043] Based on the MongoDB connection network graph and the minimum cut set, determine the target node path from the specified source node to the MongoDB node;

[0044] Based on the minimum cut set and the weight values ​​of the MongoDB nodes, the performance of the request distribution strategy is predicted to obtain the performance prediction results, which include system bottleneck prediction values ​​and performance loss prediction values.

[0045] Based on the weight values ​​of the MongoDB nodes and the path of the target node, the top n MongoDB nodes are selected as candidate nodes.

[0046] The bottleneck risk of the candidate node is evaluated using the NMC sparser model, and the performance evaluation value of the candidate node is determined based on the bottleneck risk of the candidate node, the weight value, and the performance prediction result.

[0047] Candidate nodes whose performance evaluation values ​​are greater than a preset processing capacity threshold are selected as target MongoDB nodes.

[0048] As an improvement to the above solution, the aggregation of the MongoDB node's metric data to obtain the node performance metrics for each MongoDB node further includes:

[0049] Based on the performance characteristics of the MongoDB nodes, the node connection relationships, the configuration information, the node performance indicators, and the node availability status data, the MongoDB nodes are clustered.

[0050] MongoDB nodes belonging to the same group are hierarchically structured according to the node performance metrics, a tree structure is created based on the hierarchical relationship, and multiple tree structures are organized into a dynamic forest model. The dynamic forest model is used to indicate the node status that manages the MongoDB nodes.

[0051] When a change in the metric data of any MongoDB node's running metrics is detected, the updated metric data is obtained, the dynamic forest model is updated based on the updated metric data, and the corresponding tree structure change event is determined.

[0052] Based on the dynamic forest model, the tree structure change events, and the operational metrics, a multidimensional index is constructed, which is used to indicate the indexing and retrieval of the operational metrics.

[0053] As an improvement to the above scheme, the step of converting the node performance indicators of the MongoDB node into the weight values ​​of the MongoDB node by combining the obtained preset node weight values ​​of the MongoDB node further includes:

[0054] Each tree structure in the dynamic forest model is considered as a node cluster, and a cluster graph is constructed to indicate multiple node clusters and the communication links between the node clusters.

[0055] The cluster graph is divided into multiple subsets, each subset indicating the bottleneck link between the node clusters. The bottleneck link is used to characterize the communication link when the link performance data reaches the preset abnormal link performance condition.

[0056] The subset is mapped to a cactus structure, and based on the cactus structure and the subset, the target bottleneck point is identified. The cactus structure is used to describe the bottleneck association between the node clusters.

[0057] Based on the multidimensional index, the target bottleneck point, and the weight value of the MongoDB node, load prediction is performed on the MongoDB node to obtain the node load prediction value.

[0058] Based on the predicted node load of the MongoDB node and the target bottleneck, the system state is simulated, and the weight value of the MongoDB node is adjusted according to the predicted node load and the system state.

[0059] Secondly, embodiments of this application also provide a MongoDB multi-node dynamic switching system, including:

[0060] The data access layer creation module is used to obtain the configuration information of multiple MongoDB nodes and establish a data access layer based on the configuration information. The data access layer is configured with a connection pool management mechanism for instructing the management of the connection pool.

[0061] The metrics acquisition module is used to acquire metrics data of the MongoDB node about multiple operational metrics through the data access layer according to the hierarchical frequency acquisition task configured for each MongoDB node. The acquisition task is used to indicate the multiple acquisition frequencies executed by the MongoDB node and the operational metrics to be acquired at each acquisition frequency. The operational metrics include CPU utilization, memory utilization, IOPS, disk wait time and network latency.

[0062] The data aggregation module is used to aggregate the indicator data of the MongoDB nodes to obtain the node performance indicators of each MongoDB node.

[0063] The weight calculation module is used to combine the obtained preset values ​​of the node weights of the MongoDB nodes to convert the node performance indicators of the MongoDB nodes into the weight values ​​of the MongoDB nodes.

[0064] The routing and distribution module is used to determine a request distribution strategy based on the connection pool management mechanism and the obtained node availability status data of the MongoDB node, according to the weight value of the MongoDB node. The request distribution strategy is used to indicate the distribution ratio of MongoDB requests allocated to the MongoDB node.

[0065] The health management module is used to perform status detection on the MongoDB node through the data access layer. When the MongoDB node is detected to meet the preset abnormal status conditions, the detected MongoDB node is marked as unavailable and the node availability status data is updated.

[0066] The MongoDB multi-node dynamic switching method provided in this application achieves zero intrusion into business code through a non-intrusive data access layer design, reducing system coupling and improving maintainability. Furthermore, by collecting and aggregating multi-dimensional real-time metrics, it accurately perceives the real-time performance status of each node, providing precise basis for load balancing. Employing a non-linear weight model, it can dynamically adjust weights based on node performance status, achieving more intelligent request allocation. Then, through a dynamic routing and load balancing engine, it distributes requests according to node weight ratios, avoiding the instantaneous high load problem caused by concentrated request allocation. Finally, through automatic circuit breaking and recovery detection, it can automatically take faulty nodes offline and bring them back online after recovery, reducing manual intervention and improving system availability. Attached Figure Description

[0067] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 A flowchart illustrating a method for dynamic multi-node switching in MongoDB provided in this application embodiment;

[0069] Figure 2 This is a schematic diagram of the structure of a MongoDB multi-node dynamic switching system provided in an embodiment of this application. Detailed Implementation

[0070] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0071] like Figure 1 As shown in the figure, this application embodiment provides a method for dynamic switching of multiple nodes in MongoDB, which includes the following steps S1 to S6.

[0072] S1: Obtain configuration information for multiple MongoDB nodes, establish a data access layer based on the configuration information, and configure a connection pool management mechanism within the data access layer to indicate the management of the connection pool.

[0073] In a multi-node MongoDB deployment environment, configuration information refers to a set of data describing the characteristics and connection parameters of each MongoDB node. This includes the connection string (a URI format string specifying the MongoDB server's network address, port number, and authentication information), maximum connections (the maximum number of concurrent connections allowed per node, typically set based on the node's hardware performance), connection timeout (the maximum waiting time to establish a connection; exceeding this time is considered a connection failure), and idle connection hold time (the maximum time unused connections are retained in the connection pool; connections exceeding this time are released). These parameters directly impact the performance and stability of database connections; proper configuration can reduce connection establishment overhead and improve resource utilization.

[0074] In this application, the data access layer provides an abstraction layer between the application and data storage layers to isolate business logic from data access logic. Specifically, the data access layer refers to a code structure that encapsulates the native MongoDB driver, providing a unified interface to upper-layer business logic while internally handling complex logic such as connection management, load balancing, and failover. Therefore, by configuring the data access layer, zero intrusion into business code is achieved, freeing business developers from worrying about the connection and switching details of the underlying MongoDB nodes, significantly reducing system coupling and maintenance costs.

[0075] It should be noted that connection pooling mechanisms are used to effectively manage and reuse database connection resources. In high-concurrency systems, establishing and closing database connections are expensive operations that consume significant system resources and increase response latency. Connection pools significantly improve performance by pre-creating and maintaining a certain number of database connections, allowing applications to reuse these connections instead of creating new connections for each operation. The connection pooling mechanism includes core components such as connection creation strategies (how to initialize the connection pool), connection allocation strategies (how to select available connections), connection verification strategies (how to detect connection validity), and connection release strategies (when to reclaim idle connections).

[0076] For example, establishing a data access layer first requires analyzing the interface specifications of the native MongoDB driver and the business requirements of the application system to determine suitable design patterns and architectural styles. Typically, the Proxy pattern and Decorator pattern are used to ensure that the data access layer's API is highly consistent with the native MongoDB API, while internally adding enhancements such as load balancing and failover. Subsequently, a dedicated connection pool is created for each MongoDB node based on the configuration information, and corresponding management mechanisms are configured. Finally, the connection pool is integrated into the overall architecture to form a complete data access layer, enabling unified management and transparent access to MongoDB nodes.

[0077] In some embodiments, step S1 specifically includes:

[0078] Based on the interface specifications of the native driver of the MongoDB node and the business requirements of multiple MongoDB nodes, the overall architecture of the data access layer is determined.

[0079] Based on the overall architecture of the data access layer and the connection string, maximum number of connections, connection timeout time and idle connection hold time in the configuration information of the MongoDB node, a corresponding connection pool is created for each MongoDB node, and a connection pool management mechanism is configured.

[0080] A data access layer is established based on the connection pool management mechanism and the overall architecture of the data access layer.

[0081] In this embodiment, the overall architecture of the data access layer is first determined based on the MongoDB node's native driver interface specification and business requirements. The MongoDB native driver interface specification refers to the official database driver API definition, which specifies the standard interfaces for basic functions such as connection establishment, data manipulation, query execution, and transaction processing. In a Java environment, this typically refers to the core classes and interfaces provided by the MongoDB Java Driver, such as MongoClient, MongoDatabase, and MongoCollection; in a Node.js environment, it refers to the relevant APIs provided by the MongoDB module. Adherence to the native driver interface specification ensures consistency between the data access layer and standard MongoDB operations, enabling seamless migration for developers. Business requirements refer to the specific requirements of the application system for database access, including read / write separation strategies, batch operation support, query caching requirements, and monitoring and tracing requirements. Determining the overall architecture of the data access layer requires comprehensive consideration of these factors, selecting appropriate design patterns and structural organization.

[0082] When determining the overall architecture, a multi-layered design pattern is typically adopted. The surface layer is the business-oriented API layer, providing an interface completely consistent with the native MongoDB driver, achieving zero intrusion into business code; the middle layer is the core processing layer, containing enhanced functions such as load balancing, failover, and metric collection; the bottom layer is the resource management layer, responsible for connection pool management and node state maintenance. The architecture design also needs to consider cross-cutting concerns, such as logging, performance monitoring, and security control. Commonly used design patterns include the Proxy Pattern, used to add extra functionality without modifying the original interface; the Decorator Pattern, used to dynamically add new behaviors to objects; and the Factory Pattern, used to create and manage database connection instances. The overall architecture design principles are: maintaining API compatibility to ensure that business code does not need modification; cohesive functional modules to make each component have a single, clear responsibility; and loose coupling design to reduce dependencies between modules and facilitate expansion and maintenance.

[0083] Secondly, based on the overall architecture of the data access layer and the configuration information of the MongoDB nodes, a corresponding connection pool is created for each node, and a connection pool management mechanism is configured. It should be noted that a connection pool is a resource reuse technology that reduces the overhead of frequently creating and closing connections by pre-creating and maintaining a certain number of database connections, significantly improving system performance. Connection pools are particularly important in high-concurrency environments, as they effectively control the number of concurrent connections and prevent database server performance degradation due to excessive connections. Key parameters in the configuration information include: Connection String, which defines the network address, port number, authentication information, and database name of the MongoDB server, typically using a standard URI format; Max Pool Size, which limits the maximum number of concurrent connections per node, usually determined based on the node's hardware performance, application concurrency, and MongoDB configuration (such as maxConnections). Too large a size will increase server load, while too small a size may lead to connection waiting; Connection Timeout, which refers to the maximum waiting time to establish a connection. Exceeding this time is considered a connection failure. A shorter timeout helps to quickly identify network problems, but may increase the connection failure rate during network fluctuations; and Idle Connection Keep Alive, which refers to the longest time unused connections are kept in the connection pool. Too long a timeout may consume excessive resources, while too short a timeout may lead to frequent connection creation and destruction.

[0084] For example, the connection pool creation process includes: initializing the connection pool configuration, converting the above parameters into configuration objects supported by the MongoDB driver; warming up the connection pool, pre-creating a certain number of connections at startup to avoid delays in the first request; configuring a connection verification mechanism, periodically checking the validity of connections and removing invalid connections in a timely manner; setting connection borrowing and returning strategies, specifying the connection acquisition order (such as FIFO or LIFO) and timeout mechanism. In this regard, the connection pool management mechanism is a framework for unified management of multiple connection pools, including: connection allocation strategy, determining which node's connection pool a request should use; connection monitoring, collecting usage statistics for each connection pool, such as the number of active connections and the number of waiting connections; dynamic adjustment, dynamically adjusting the connection pool size and configuration according to the load; and fault handling, detecting connection pool anomalies and performing recovery operations. This embodiment, by configuring the connection pool management mechanism, can ensure efficient and reliable allocation of database connection resources in various scenarios.

[0085] Finally, based on the connection pool management mechanism and the overall architecture of the data access layer, a complete data access layer is established, forming a unified and complete data access layer that provides transparent database access services for upper-layer applications. In this embodiment, this specifically includes: interface implementation, implementing various APIs conforming to the MongoDB native driver interface specification according to the previously determined architecture; routing component integration, integrating the load balancing engine with the connection pool management mechanism to achieve intelligent request distribution; monitoring system embedding, adding performance monitoring and data collection points at key operation points to provide data sources for metric collection; exception handling mechanism, defining unified exception types and handling processes to ensure consistent behavior under various exception conditions; and configuration management, providing a dynamic configuration adjustment interface to support runtime modification of connection pool parameters, load balancing strategies, etc.

[0086] Optionally, the establishment process also needs to consider thread safety and concurrency control to ensure the stability and correctness of the data access layer in a high-concurrency environment. Commonly used techniques include: Read-Write Locks, which allow multiple read operations to run concurrently, but write operations require exclusive access; Atomic Operations, which ensure the atomicity of state updates and avoid race conditions; Thread Pools, which control the number of concurrent threads and prevent resource exhaustion; and ThreadLocal, which isolates the context information of different threads and reduces lock contention. The final data access layer should be a highly cohesive, loosely coupled, and easily extensible component, providing an interface completely consistent with the native MongoDB driver, while internally implementing advanced features such as connection pool management, load balancing, and failover, achieving zero intrusion into business logic code.

[0087] In practical implementation, the data access layer can be built using a progressive approach: first, implement basic functions such as connection pool management and simple routing; then add monitoring and metric collection; and finally, integrate load balancing and failover mechanisms. This approach allows for the provision of usable components early in development while maintaining architectural flexibility, facilitating adjustments and optimizations based on actual usage. After the data access layer is built, comprehensive functional and performance testing is required to verify its correctness and performance under various scenarios, especially its behavior under extreme conditions such as high concurrency, network fluctuations, and node failures.

[0088] Therefore, this embodiment systematically determines the architecture, creates a connection pool, and establishes a data access layer, which facilitates subsequent dynamic switching of MongoDB multi-nodes. It not only achieves zero intrusion into the business code but also provides efficient connection resource management and a flexible expansion mechanism.

[0089] Step S2: Through the data access layer, acquire the metric data of the MongoDB node for multiple operating metrics by acquiring tasks at the hierarchical frequency configured for each MongoDB node. The acquisition task is used to indicate the multiple acquisition frequencies executed by the MongoDB node and the operating metrics to be acquired at each acquisition frequency. The operating metrics include CPU utilization, memory utilization, IOPS, disk wait time and network latency.

[0090] It should be noted that tiered frequency acquisition is a data acquisition strategy that uses different sampling frequencies for different characteristic indicators. Different types of system indicators change at different rates and have varying degrees of impact on system performance. For example, CPU and memory usage change rapidly, directly affecting the system's real-time processing capabilities and requiring high-frequency sampling; while disk and network performance are relatively stable and change slowly, allowing for low-frequency sampling to reduce acquisition overhead. Therefore, this application, through tiered acquisition, can obtain real-time data for key indicators while avoiding excessive acquisition operations that would lead to additional burden.

[0091] Among these, operational metrics are key data points for measuring the performance and health of MongoDB nodes. Specifically, CPU utilization refers to the busyness of the node's CPU, usually expressed as a percentage, and is a primary indicator for assessing the saturation of the node's computing power; memory utilization represents the proportion of used memory to total available memory, reflecting the node's memory pressure; IOPS (Input / Output Operations Per Second) is a metric for measuring storage system performance, representing the number of I / O requests that the storage device can handle per second; disk latency is the time required for an I / O operation to complete from the issuance of a request, reflecting the storage system's response speed; and network latency represents the round-trip time of network communication, directly affecting the interaction efficiency of the distributed system. Therefore, this embodiment, by configuring the above operational metrics, collectively constitutes a multi-dimensional view for evaluating the performance of MongoDB nodes.

[0092] In some embodiments, the hierarchical frequency acquisition task includes a high-frequency indicator acquisition task and a low-frequency indicator acquisition task. Step S2 specifically includes:

[0093] Based on the aforementioned operational metrics, high-frequency metric collection tasks and low-frequency metric collection tasks are determined to obtain tiered frequency collection tasks. A corresponding tiered frequency collection task is configured for each MongoDB node. The high-frequency metric collection task is used to indicate a first collection frequency for high-frequency metrics, and the low-frequency metric collection task is used to indicate a second collection frequency for low-frequency metrics. The first collection frequency is greater than the second collection frequency. The high-frequency metrics include the CPU utilization and the memory utilization, and the low-frequency metrics include the IOPS, the disk wait time, and the network latency.

[0094] Based on the high-frequency indicator collection task and the connection pool management mechanism, a high-frequency collection thread is established for each MongoDB node, and the high-frequency indicator collection task is executed according to the high-frequency collection thread to obtain the high-frequency indicator data.

[0095] Based on the low-frequency indicator collection task and the connection pool management mechanism, a low-frequency collection thread is established for each MongoDB node, and the low-frequency indicator collection task is executed according to the low-frequency collection thread to obtain the low-frequency indicator data.

[0096] Configure a timeout mechanism and a retry strategy for the high-frequency index data and the low-frequency index data. The timeout mechanism is used to indicate that the current hierarchical frequency acquisition task is marked as acquisition failure when the data acquisition time exceeds a preset acquisition time threshold. The retry strategy is used to indicate that the corresponding hierarchical frequency acquisition task is re-executed when acquisition failure is detected.

[0097] In this embodiment, a tiered frequency data collection task is adopted to consider the characteristics and importance of different operational metrics. High-frequency metrics (CPU utilization and memory utilization) change rapidly and require a higher collection frequency (e.g., once per second), while low-frequency metrics (IOPS, disk wait time, and network latency) change relatively slowly and can be collected at a lower frequency (e.g., once every 10 seconds). Independent high-frequency and low-frequency collection threads are established for each MongoDB node to ensure that collection tasks do not interfere with each other. Simultaneously, a timeout mechanism and retry strategy are implemented to ensure the reliability of metric collection. When collection times out or fails, a retry mechanism is triggered to ensure data integrity.

[0098] Specifically, based on the characteristics of the operational metrics, high-frequency and low-frequency metric collection tasks are first determined, forming a complete hierarchical frequency collection task. Hierarchical frequency collection is a data collection strategy that uses different sampling frequencies for different types of metrics; that is, different collection resources are allocated according to the rate of change of the metric and its impact on system decisions. High-frequency metrics are those that change rapidly and have a direct impact on system performance evaluation, requiring frequent sampling to capture their instantaneous changes. Low-frequency metrics, on the other hand, are those that change relatively slowly or have a smaller impact on short-term decisions, allowing for a lower sampling frequency to reduce system overhead. In a MongoDB multi-node environment, CPU utilization and memory utilization are classified as high-frequency metrics because these two metrics typically fluctuate significantly and directly reflect the node's processing capacity and load status, having a decisive impact on load balancing decisions. IOPS, disk wait time, and network latency are classified as low-frequency metrics because these metrics typically change more gradually and are unlikely to experience drastic fluctuations in a short period.

[0099] It should be noted that the first sampling frequency is the time interval for high-frequency metrics, typically set to once per second (1Hz). This high-frequency sampling can capture instantaneous fluctuations in CPU and memory usage, providing a near real-time performance view. The second sampling frequency is the time interval for low-frequency metrics, typically set to once every 10 seconds (0.1Hz). This lower sampling frequency is sufficient to reflect the changing trends of IOPS, disk wait time, and network latency, while significantly reducing the impact of the sampling operation. Determining the sampling frequency requires balancing data accuracy and system overhead: too high a frequency will increase the system load and may affect business processing capabilities; too low a frequency may miss critical performance fluctuations, leading to decision delays or errors. In actual deployments, these frequencies can usually be adjusted through configuration files to adapt to different hardware environments and business needs.

[0100] Subsequently, tiered frequency data collection tasks are configured independently for each MongoDB node to ensure the isolation and targeting of collection operations. Different nodes may require different collection strategies due to differences in hardware configuration, load characteristics, or network environment. For example, nodes with weaker performance may need to reduce the collection frequency to alleviate additional burden; nodes located in remote data centers may need to adjust timeout parameters to accommodate higher network latency. Independent node configuration also ensures single-point-of-failure isolation, meaning that a collection problem on one node will not affect the monitoring status of other nodes. Specifically, the configuration process typically includes: selecting an appropriate collection frequency based on node characteristics; setting specific metric collection commands or queries (such as serverStatus, dbStats, etc.); configuring node-specific timeout parameters and retry policies; and setting local caching strategies for collected data.

[0101] Secondly, based on the high-frequency metric collection task and connection pool management mechanism, a high-frequency collection thread is established for each MongoDB node. It should be noted that the high-frequency collection thread is a dedicated worker thread responsible for executing high-frequency metric collection tasks. It is created at startup and periodically executes collection operations according to a predetermined initial collection frequency. Thread management is the core technology of high-frequency collection, ensuring that the collection operation can proceed stably and continuously without significantly impacting normal functionality. High-frequency collection threads typically employ a time-driven model, triggering collection operations at precise time points to ensure consistent sampling intervals. This model is usually implemented using a timer or scheduler, such as Java's ScheduledExecutorService or Node.js's setInterval function.

[0102] For example, a high-frequency data collection thread may include: thread priority settings, typically setting the collection thread to a lower priority than the business processing thread to ensure that core business operations are not affected when resources are scarce; execution time control, limiting the maximum execution time of a single collection operation to prevent the collection thread from being blocked due to slow node response; an asynchronous execution model, employing non-blocking I / O and callback mechanisms to reduce thread waiting time; resource isolation, allocating independent connection pools and resource quotas to the collection thread to avoid competing for resources with business operations; and error isolation, ensuring that exceptions in the collection operation should not affect the business processing flow, requiring a robust exception handling mechanism. The high-frequency collection thread obtains database connections through a connection pool management mechanism, executes predefined collection commands (such as MongoDB's `serverStatus` command), obtains high-frequency metrics such as CPU utilization and memory usage, and then stores this data in a local cache or directly passes it to the data processing component.

[0103] Similarly, based on the low-frequency metric collection task and connection pool management mechanism, a low-frequency collection thread is established for each MongoDB node. The low-frequency collection thread works similarly to the high-frequency collection thread, but executes at a lower frequency, typically triggering once every 10 seconds. It mainly collects metrics that change slowly, such as IOPS, disk wait time, and network latency. Low-frequency collection usually requires executing more complex commands or aggregation operations, such as details of collStats or serverStatus, or custom diagnostic commands. These operations may consume more resources, so reducing the execution frequency is reasonable.

[0104] Optionally, low-frequency data collection threads may include: longer execution timeouts, as collecting certain low-frequency metrics may involve heavy computation or I / O operations; batch query optimization, merging the collection of multiple low-frequency metrics into a single command execution to reduce the number of interactions; data caching strategies, as the longer sampling intervals may require more cautious data caching mechanisms to ensure that the most recent valid data is still available when collection fails; and staggered collection windows, where the low-frequency collection times of different nodes should be appropriately staggered to avoid a sudden increase in cluster pressure caused by a large number of simultaneous collection requests. Low-frequency data collection threads also obtain database connections through a connection pool, but typically choose lower-priority connections or perform collection operations during periods of low business load to minimize the impact on business operations.

[0105] Furthermore, timeout mechanisms and retry strategies are configured for both high-frequency and low-frequency indicator data to ensure that key indicator data can still be acquired despite network fluctuations or temporary node unavailability. The timeout mechanism is a control mechanism that limits the maximum waiting time for a single indicator collection operation, preventing indefinite waiting due to slow node response and thus blocking the collection thread. By configuring the timeout mechanism, a collection duration threshold is set, which is the maximum allowed execution time for a collection operation; exceeding this time is considered a collection failure. The collection duration threshold is typically set based on the indicator type and network environment: high-frequency indicator collection operations are usually lightweight, so the threshold can be set shorter, such as 200-500 milliseconds; low-frequency indicator collection operations may be more complex, so the threshold can be appropriately extended, such as 1-3 seconds. The timeout mechanism is usually implemented using built-in timeout control in the database driver or thread-level timeout monitoring to ensure that if the collection operation cannot be completed within the preset time, it can be actively interrupted and resources released.

[0106] A retry strategy is a set of rules that automatically attempt to re-execute a data collection task when a collection failure is detected. This embodiment, by configuring a retry strategy, can maintain data continuity and improve the reliability of the monitoring system in the event of brief network fluctuations or temporary node unavailability. A retry strategy typically includes the following key parameters: number of retries, i.e., the maximum number of additional attempts after a single collection failure, usually set to 2-3 times; retry interval, i.e., the waiting time between two retries, which can use a fixed interval (e.g., 1 second each time) or an exponential backoff algorithm (e.g., 100ms for the first time, 200ms for the second, and 400ms for the third); and maximum retry time, i.e., the longest allowed retry time window for a single collection point, ensuring that subsequent collection points are not delayed due to excessive retries. The retry strategy is typically implemented using a state machine or recursive call pattern, checking whether the conditions are met before each retry to avoid meaningless retries, and recording retry logs for subsequent analysis and optimization.

[0107] In this embodiment, handling data acquisition failures is a crucial part of the retry strategy, and different types of failures may require different handling methods. Connection failures typically indicate network problems or node unavailability, which may require triggering node health checks; command execution failures may be due to permission issues or command syntax errors, requiring configuration checks; timeout failures may be due to excessive node load, and temporarily reducing the acquisition frequency or increasing the timeout threshold can be considered. The complete data acquisition failure handling process includes: recording detailed failure information, including failure type, time, and error message; implementing targeted retry strategies; if retry still fails, marking the acquisition point as missing and potentially triggering an alarm; and employing data compensation mechanisms, such as filling with the most recent valid value or using statistical interpolation to estimate missing data, to ensure the continuity of subsequent analysis.

[0108] For example, the overall execution flow of the hierarchical frequency data acquisition task includes: during system initialization, creating corresponding high-frequency and low-frequency acquisition tasks for each MongoDB node according to the configuration; starting independent high-frequency and low-frequency acquisition threads for each node; the high-frequency acquisition thread performs an acquisition operation once per second to obtain high-frequency indicators such as CPU utilization and memory utilization; the low-frequency acquisition thread performs an acquisition operation once every 10 seconds to obtain low-frequency indicators such as IOPS, disk wait time, and network latency; each acquisition operation is controlled by a timeout mechanism to ensure completion or interruption within a preset time; if acquisition fails, a retry is performed according to the retry policy; finally, the acquired high-frequency and low-frequency indicator data are passed to the data processing component for subsequent aggregation and analysis.

[0109] Therefore, this embodiment distinguishes between high-frequency and low-frequency metrics, configuring the most suitable collection strategy for different types of performance data, and ensuring the reliability and efficiency of collection through dedicated threads, connection pool management, timeout control, and retry mechanisms. By collecting data in layers, real-time data of key metrics can be obtained while avoiding system burden caused by excessive collection, providing a comprehensive and accurate performance data foundation for dynamic switching of MongoDB multiple nodes.

[0110] Step S3: Aggregate the indicator data of the MongoDB nodes to obtain the node performance indicators of each MongoDB node.

[0111] It should be noted that aggregation processing is used to combine multiple data points and extract trend features and statistical information. In MongoDB node monitoring, the aggregation processing of metric data aims to eliminate the interference of instantaneous fluctuations and obtain smoother data that more accurately represents the true performance status of the nodes. Raw metric data often contains noise and outliers, and using it directly for decision-making may lead to frequent system fluctuations. Through aggregation processing, stable performance trends can be identified, enabling more reliable load balancing decisions.

[0112] Optionally, window data aggregation is the process of integrating data from multiple indicator windows into node performance metrics. This includes not only summarizing the averages of various metrics but may also involve correlation analysis and comprehensive scoring between metrics. For example, calculating the ratio of CPU utilization to IOPS assesses the computational efficiency of a node; or analyzing the relationship between memory usage and query response time to determine the impact of memory pressure on performance. The final generated node performance metric is a multi-dimensional data structure that comprehensively reflects the performance status of the node in various aspects, providing a foundation for subsequent weight calculations.

[0113] In some embodiments, step S3 specifically includes:

[0114] Based on the high-frequency indicator collection task, a high-frequency indicator window is configured for the high-frequency indicator, and the average value of the high-frequency indicator data collected within the high-frequency indicator window is calculated to obtain the average value of the high-frequency indicator.

[0115] Based on the low-frequency index collection task, a low-frequency index window is configured for the low-frequency index, and the average value of the low-frequency index data collected within the low-frequency index window is calculated to obtain the average value of the low-frequency index.

[0116] When missing data is detected in the high-frequency indicator window and / or the low-frequency indicator window, the missing data is filled with adjacent data or the missing data is filled by interpolation.

[0117] Window data aggregation is performed on the average of the high-frequency indicators, the average of the low-frequency indicators, and the filled data to obtain the node performance indicators.

[0118] In this embodiment, the indicator data aggregation process employs a sliding window algorithm to eliminate interference from instantaneous fluctuations and extract the trend characteristics of the indicators. The default window size for high-frequency indicators is 5 seconds (containing 5 sampling points), and the default window size for low-frequency indicators is 50 seconds (also containing 5 sampling points). The average value is calculated for the data within the window to obtain smoothed indicator values. For missing data points within the window, adjacent data or interpolation is used to fill in the gaps, ensuring data continuity. Finally, the average values ​​of the high-frequency and low-frequency indicators are combined to form a complete node performance indicator, providing a foundation for subsequent weight calculations.

[0119] First, based on the high-frequency indicator collection task, corresponding high-frequency indicator windows are configured for the high-frequency indicators, and data processing is performed within these windows. A high-frequency indicator window is a data set defined by a time range, used to accommodate high-frequency indicator data points collected within a specific time period. It should be noted that window analysis is a fundamental technique in time series data processing. By dividing a continuous data stream into windows of finite size, data processing becomes more structured and efficient. In a MongoDB multi-node environment, high-frequency indicator windows are typically set between 30 seconds and 5 minutes, depending on stability requirements and decision response speed. The choice of window size requires a trade-off between stability and sensitivity: a larger window will make the system less responsive to performance changes but will provide more stable indicator values; a smaller window will respond quickly to changes but may lead to frequent load adjustments due to short-term fluctuations.

[0120] Specifically, high-frequency indicator windows are typically implemented using a sliding window technique. As time progresses, the window continuously slides forward, removing the oldest data points and adding the newest, while maintaining a relatively constant window size. Key parameters of a sliding window include: window size, defining the length of time the window contains, such as 60 seconds; sliding step, defining the time interval between each window movement, which can be equal to the data acquisition frequency (e.g., 1 second) or an integer multiple thereof; and window overlap, defining the degree of overlap between two adjacent windows, usually determined by both the window size and the sliding step. The data structure design of high-frequency indicator windows needs to consider efficient data access and updates, typically using a circular buffer or a timestamp-indexed hash table to ensure O(1) data insertion and deletion complexity.

[0121] Next, the average calculation within the window involves taking the arithmetic mean of all data points within the window to obtain a value that better represents the overall level of the indicator over that time period. The arithmetic mean effectively reduces the impact of random noise and short-term fluctuations, providing a more stable indicator value. For high-frequency indicators such as CPU utilization and memory utilization, average calculation is particularly important because these indicators may fluctuate significantly in a short period, but what truly matters for system performance is their sustained trend rather than instantaneous values. Specifically, the average calculation process includes: data validation, checking and filtering outliers or erroneous data; aggregation and summation, accumulating the values ​​of all valid data points within the window; dividing by the number of valid data points to obtain the arithmetic mean; and result normalization, ensuring the result is within the expected range (e.g., CPU utilization within the 0-100% range).

[0122] Similarly, based on the low-frequency metric collection task, a low-frequency metric window is configured for the low-frequency metric, and similar data processing is performed. The low-frequency metric window is conceptually the same as the high-frequency metric window, but because low-frequency metrics are collected at a lower frequency, their window size is usually larger to accommodate enough data points for effective analysis. For example, if a low-frequency metric is collected every 10 seconds, and it is desired that the window contains 30 data points to ensure statistical validity, then the low-frequency metric window size needs to be set to 300 seconds (5 minutes). The setting of the low-frequency metric window needs to consider the changing characteristics of the metric: IOPS and disk latency are usually related to database load patterns and may require a larger window to capture the entire workload cycle; network latency may be affected by network congestion and requires a sufficiently long observation window to identify trends in network conditions.

[0123] It should be noted that special considerations for low-frequency indicator windows include: handling data sparsity (due to the low sampling frequency and fewer data points within the window, special attention needs to be paid to data representativeness); balancing timeliness (an excessively large window may cause the indicator to reflect the system state of the distant past, requiring adjustment of the window size according to the timeliness requirements of the indicator); and synchronization with high-frequency indicator windows (ensuring that the evaluation cycles of high and low frequency indicators match, facilitating subsequent aggregation and analysis). The calculation of the average value within the low-frequency indicator window is similar to that of the high-frequency indicator, but may require consideration of data point weighting; that is, newer data points may be assigned higher weights to reflect their stronger timeliness and relevance.

[0124] Secondly, when missing data is detected within high-frequency indicator windows and / or low-frequency indicator windows, appropriate data imputation strategies are implemented. Missing data is a common challenge in time series analysis, which can be caused by various factors, such as network interruptions, temporary node unavailability, data collection timeouts, or data collection task failures. If left untreated, missing data can lead to biased window analysis results, inaccurate statistics, or calculation errors, thereby affecting the accuracy and effectiveness of load balancing decisions. Data imputation is a key technology for solving the missing data problem, estimating the possible values ​​of missing data points through various methods to ensure the continuity and completeness of window analysis.

[0125] Optionally, adjacent data imputation is a simple and direct method that uses the valid data before and after the missing data point to estimate the missing value. Exemplary adjacent imputation strategies may include: Forward Fill, which fills the missing value with the nearest valid value before the missing point, suitable for indicators that do not change significantly in the short term; Backward Fill, which fills the missing value with the nearest valid value after the missing point, suitable for situations where changes have occurred but the timing of the change is uncertain; and Mean Fill, which fills the missing value with the arithmetic mean of the nearest valid values ​​before and after the missing point, providing a smoother transition. Adjacent imputation methods are simple and efficient, suitable for handling a small number of randomly distributed missing points, but may not be accurate enough for a large number of consecutive missing points.

[0126] Optionally, interpolation imputation is a more complex but also more accurate method that estimates missing values ​​based on the mathematical relationships between existing data points. Exemplary interpolation methods can include: linear interpolation, which assumes a linear relationship between data points and calculates the estimated value of the missing point based on the values ​​of preceding and following points and the time interval; suitable for indicators with relatively gentle changes. Spline interpolation uses a piecewise polynomial function (such as cubic splines) to fit the data points, providing a smoother interpolation result; suitable for situations where maintaining curve smoothness is required. Polynomial interpolation uses a single polynomial function to fit all data points, which may lead to overfitting when there are many data points. Interpolation imputation is generally more accurate than adjacent imputation, especially when the indicator shows a clear trend or periodic changes, but it is also more computationally complex.

[0127] It should be noted that the selection of an imputation strategy needs to consider several factors: the missing data pattern (random or consecutive); the characteristics of the indicator (stationary or volatile); timeliness requirements (whether the imputation operation needs to be performed in real time); and computational resource constraints (the available time and processing power for imputation computation). In practical implementation, a hybrid strategy may be adopted: for single-point random missing data, adjacent data may be used for imputation; for a small number of consecutive missing data, interpolation may be used; for a large number of consecutive missing data, it may be necessary to mark them as invalid intervals to avoid making decisions based on unreliable data.

[0128] Furthermore, window data aggregation is performed on the mean of high-frequency indicators, the mean of low-frequency indicators, and the filled data to obtain the final node performance indicators. Window data aggregation is the process of comprehensively processing indicator data from different time windows and different types to form a unified performance evaluation indicator, ensuring that all relevant performance factors are appropriately considered and avoiding biased evaluations that may result from a single indicator. Specifically, the aggregation process typically includes: time alignment, ensuring that the evaluation windows of high-frequency and low-frequency indicators match in time, usually using the low-frequency indicator window as a benchmark to aggregate high-frequency indicator data to the same time point; indicator normalization, converting indicators with different dimensions and ranges into a unified standardized scale to facilitate subsequent comparisons and weighted calculations; weight allocation, assigning different weight coefficients to different indicators based on their impact on overall performance; and weighted combination, synthesizing the indicator values ​​into a comprehensive performance indicator based on the assigned weights.

[0129] Optionally, node performance metrics may be calculated using various mathematical models, such as weighted average models, geometric average models, or more complex nonlinear models. The final node performance metrics typically include several key dimensions: processing capacity metrics, reflecting the node's ability to handle requests, primarily based on CPU utilization and memory usage; storage performance metrics, reflecting the performance of the node's storage subsystem, primarily based on IOPS and disk latency; and network performance metrics, reflecting the quality of the node's network connectivity, primarily based on network latency measurements. These dimensions collectively constitute a comprehensive assessment of the node's overall performance, providing a basis for subsequent weight calculations and request distribution decisions.

[0130] Furthermore, advanced implementations of window data aggregation may include trend analysis and anomaly detection. Trend analysis identifies the changing trends of indicators, such as rising, falling, or stabilizing, by comparing indicator values ​​across multiple consecutive windows. Anomaly detection uses statistical methods to identify data points that significantly deviate from expected patterns, allowing for early detection of potential performance issues. These analytical results can serve as supplementary information, enriching the meaning of node performance indicators and improving the foresight and accuracy of system decisions.

[0131] Data quality control is crucial for ensuring the reliability of results during the aggregation process. This embodiment requires identifying and handling various data issues, such as: outliers, i.e., data points that significantly deviate from the normal range, which may be due to measurement errors or actual performance anomalies; data latency, i.e., some indicator data arriving later than expected, which may lead to incomplete window analysis; and data inconsistency, i.e., contradictions between data from different sources or at different times, which need to be resolved through verification and coordination. A robust data quality control mechanism can effectively improve the accuracy and reliability of aggregation results.

[0132] Therefore, this embodiment transforms the originally collected high-frequency and low-frequency indicator data into more meaningful node performance indicators through window configuration, average calculation, missing data handling, and window data aggregation. Using window-based data not only effectively filters short-term fluctuations and random noise, but also addresses unavoidable data quality issues in real-world environments through data filling and aggregation, ultimately providing a stable and reliable performance evaluation foundation for dynamic multi-node switching in MongoDB. This allows for close integration with the preceding data collection steps, forming a complete performance monitoring subsystem that ensures optimized load balancing decisions are made based on accurate performance data.

[0133] Step S4: Combine the obtained preset node weight values ​​of the MongoDB nodes to convert the node performance indicators of the MongoDB nodes into weight values ​​of the MongoDB nodes.

[0134] It should be noted that the preset node weights are basic weights manually set based on the hardware configuration and expected performance of MongoDB nodes, reflecting the relative processing power of nodes in a static state. High-performance nodes (such as servers configured with high-end CPUs, large amounts of memory, and high-speed SSDs) can be set to preset values ​​of 1.5 to 2.0, standard performance nodes are set to 1.0, and temporary backup or low-performance nodes can be set to 0.8 or lower. Calculating weights using preset node weights ensures that even when all nodes have similar loads, the more powerful nodes are prioritized for processing requests, thereby maximizing overall throughput.

[0135] In this application, the CPU penalty factor is a key weight adjustment parameter used to dynamically adjust node weights based on CPU utilization. As a core component of computing resources, CPU utilization directly reflects the node's workload. Specifically, when CPU utilization reaches a certain threshold, node performance will drop sharply. At this point, the node's weight should be quickly reduced to decrease the number of requests allocated to it.

[0136] The memory usage penalty factor is similar to the CPU penalty factor, but it targets the node's memory resources. MongoDB, as a memory-intensive database, has performance highly dependent on memory conditions. When memory usage exceeds a preset threshold (typically 80%), the node's weight is reduced through the penalty factor. The design of the memory penalty factor needs to consider MongoDB's memory usage characteristics: some memory is used for data caching (which can be reclaimed), and some is used for index and connection management (which is not easily reclaimed). Therefore, memory penalties are usually more gradual than CPU penalties, avoiding excessive penalties for nodes due to normal caching usage.

[0137] The IOPS penalty factor focuses on a node's I / O performance. IOPS is a metric that measures the number of input / output operations a storage system can process per second, directly impacting MongoDB's data read and write speeds. The IOPS penalty factor is calculated based on the ratio of a node's current IOPS to a preset IOPS threshold. The IOPS threshold setting depends on the performance characteristics of the storage hardware: SSDs typically have higher IOPS capacity (e.g., tens of thousands to hundreds of thousands), while traditional hard drives have lower IOPS (hundreds to thousands). When a node's IOPS approaches or exceeds the threshold, it indicates that the storage system may be becoming a performance bottleneck. The penalty factor reduces the node's weight, alleviating its I / O pressure.

[0138] The disk latency penalty factor and network latency penalty factor target storage response speed and network communication efficiency, respectively. Disk latency reflects the average time from I / O request issuance to completion; a longer latency usually indicates storage system overload or problems. Network latency measures the speed at which a node communicates with other system components; high latency can lead to slow query responses. The penalty factors for both metrics are calculated based on a comparison of the current value with a preset threshold. When the metric value exceeds the threshold, the node's weight is reduced accordingly.

[0139] Optionally, a non-linear weighting calculation model is used to calculate the real-time weight of each MongoDB node. This non-linear weighting calculation model is used to calculate the weight value by multiplying the preset value of the node weight by the product of the CPU utilization penalty factor, memory utilization penalty factor, IOPS penalty factor, disk wait time penalty factor, and network latency penalty factor. Specifically, the CPU utilization penalty factor is the smaller of 1 minus the power of α of CPU utilization and 0.9; the memory utilization penalty factor is the smaller of 1 minus the smaller of memory utilization and 0.8; the IOPS penalty factor is the smaller of 1 minus the smaller of the ratio of IOPS to the IOPS threshold and 0.5; the disk wait time penalty factor is the smaller of 1 minus the smaller of the ratio of disk wait time to the wait time threshold and 0.7; and the network latency penalty factor is the smaller of 1 minus the smaller of the ratio of network latency to the latency threshold and 0.3.

[0140] Therefore, this application calculates the final node weight by combining the preset node weight with various penalty factors, ensuring that a severe deterioration in any metric will significantly reduce the node weight. Simultaneously, the maximum penalty for each metric is limited during weight calculation to prevent a single metric from causing the weight to drop to zero, thus preserving the node's ability to handle a small number of requests.

[0141] In some embodiments, step S4 specifically includes:

[0142] When the CPU utilization index is lower than a preset first CPU utilization threshold, the CPU penalty factor is determined as the first CPU penalty factor.

[0143] Alternatively, when the CPU utilization index data exceeds the first CPU utilization threshold but is lower than the preset second CPU utilization threshold, the CPU penalty factor is determined as the second CPU penalty factor, wherein the first CPU utilization threshold is less than the second CPU utilization threshold, and the second CPU penalty factor is greater than the first CPU penalty factor.

[0144] Alternatively, when the CPU utilization index data exceeds the second CPU utilization threshold, the CPU penalty factor is determined as the third CPU penalty factor, wherein the third CPU penalty factor is greater than the second CPU penalty factor.

[0145] Based on the memory utilization rate metrics and a preset memory utilization rate threshold, a memory utilization rate penalty factor is determined; based on the IOPS metrics and a preset IOPS threshold, an IOPS penalty factor is determined.

[0146] Based on the disk wait time metric data and the preset disk wait time threshold, a disk wait time penalty factor is determined;

[0147] Based on the network latency index data and the preset network latency threshold, a network latency penalty factor is determined;

[0148] The weight value of the MongoDB node is calculated based on the preset node weight value of the MongoDB node, the determined CPU penalty factor, the memory utilization penalty factor, the IOPS penalty factor, the disk wait time penalty factor, and the network latency penalty factor.

[0149] In this embodiment, node weight calculation employs a multi-factor nonlinear weight calculation model, converting various indicators into weight values. First, the basic weight of the node (preset node weight value) is set: high-performance nodes can be set to 1.5~2.0, ordinary nodes to 1.0, and temporary backup nodes to 0.8. Then, different CPU penalty factors are applied based on different CPU utilization ranges: when CPU utilization is below 60% (first CPU utilization threshold), 1.0 is used as the CPU penalty factor (first CPU penalty factor); when CPU utilization is between 60% and 80% (second CPU utilization threshold), 1.3 is used as the CPU penalty factor (second CPU penalty factor); and when CPU utilization exceeds 80%, 1.5 is used as the CPU penalty factor (third CPU penalty factor). Similarly, the corresponding penalty factor is determined by comparing memory utilization, IOPS, disk wait time, and network latency with their respective thresholds.

[0150] Specifically, the first step is to determine the corresponding CPU penalty factor by comparing the CPU utilization metric data with a preset threshold. The CPU penalty factor is a performance tuning parameter used to quantify the impact of CPU resource consumption on the overall performance of a node. In distributed database systems, the CPU is typically one of the most critical resources, and its utilization directly reflects the node's processing power and load status. By configuring the penalty factor, when resource utilization exceeds a certain threshold, system performance no longer exhibits a linear relationship with utilization but instead shows a significant decline. Therefore, a non-linear penalty mechanism is needed to accurately reflect this performance change.

[0151] It should be noted that the first CPU utilization threshold is usually set at a low level, such as 50% or 60%, representing the upper limit of the "comfort zone" for CPU load. Below this threshold, the node has sufficient processing capacity to handle sudden load increases, and the system performance is stable and responsive. At this point, the first CPU penalty factor is applied, usually set to a small value (such as 1.0 or 1.1), indicating a small impact on the node weight. The second CPU utilization threshold is usually set at a higher level, such as 80% or 85%, representing the upper limit of the "warning zone" for CPU load. Between the first and second thresholds, the node can still process requests normally, but the reserved processing capacity has been reduced, and the ability to handle sudden loads has decreased. At this point, the second CPU penalty factor is applied, usually set to a medium value (such as 1.5 or 2.0), indicating a moderate impact on the node weight. When CPU utilization exceeds the second threshold, the node enters the "danger zone," and its processing capacity is severely limited, and response time may increase significantly. At this point, the third CPU penalty factor is applied, usually set to a large value (such as 3.0 or 5.0), indicating a significant impact on the node weight.

[0152] In this embodiment, the specific settings of the threshold and penalty factor need to be adjusted based on server hardware characteristics, application load features, and system performance goals. High-performance servers may maintain good performance even under high CPU utilization, so the threshold can be set higher; while for older servers or latency-sensitive applications, the threshold may need to be set lower. The setting of the penalty factor also needs to consider the sensitivity to load balancing: excessively large factor differences can lead to drastic fluctuations in load distribution, potentially triggering a "ping-pong effect"; excessively small factor differences may fail to adequately respond to performance changes. The configuration of the threshold and penalty factor typically needs to be continuously optimized through performance testing and practical operating experience to achieve the best load balancing effect.

[0153] Secondly, based on a comparison of memory usage metrics with preset thresholds, a corresponding memory usage penalty factor is determined. The memory usage penalty factor is a parameter that quantifies the impact of memory resource consumption on node performance. In memory-intensive databases such as MongoDB, memory is a key factor affecting performance. Sufficient available memory ensures that working sets can reside in memory, avoiding frequent disk accesses and significantly improving query performance. Memory usage thresholds are typically set at a high level, such as 75% or 80%, considering that the operating system and database engine usually use idle memory as a cache; a threshold that is too low may lead to wasted memory resources.

[0154] In this embodiment, the memory usage penalty factor is typically set in a stepped structure. For example, when memory usage is below a threshold, the penalty factor is small (e.g., 1.0); when it exceeds the threshold, the penalty factor increases significantly (e.g., 2.5 or 3.0). This design reflects the unique nature of memory usage: system performance typically doesn't change significantly before the usage reaches a certain critical point; however, once this critical point is exceeded, especially when the system starts using swap space, performance can drop sharply. It is particularly important to note that the calculation of memory usage should consider MongoDB's memory management characteristics, especially the caching mechanism of the WiredTiger storage engine. The correct calculation should focus on the actual memory footprint of the database working set, not just the overall system memory usage.

[0155] Next, the IOPS (Input / Output Operations Per Second) metric is compared with a preset threshold to determine the IOPS penalty factor. The IOPS penalty factor quantifies the impact of the storage subsystem's I / O operation load on node performance. IOPS is a key metric for measuring storage system performance, representing the number of I / O requests a storage device can handle per second. In a MongoDB environment, high IOPS typically indicates frequent disk read / write operations, potentially caused by working sets exceeding memory capacity, large write operations, or inadequate indexing. Setting the IOPS threshold requires consideration of the storage device type and performance characteristics: Solid-state drives (SSDs) have significantly higher IOPS capacity than traditional hard disk drives (HDDs), therefore the threshold setting should differ; for storage services in cloud environments, IOPS may be limited by service level, requiring reference to the service provider's specifications.

[0156] In this embodiment, the IOPS penalty factor is typically designed as a continuous or piecewise function, gradually increasing with the increase of IOPS. Low IOPS ranges (e.g., below 50% of the storage device's rated IOPS) generally correspond to smaller penalty factors (e.g., 1.0); medium IOPS ranges correspond to medium penalty factors; and high IOPS ranges (e.g., close to or exceeding the storage device's rated IOPS) correspond to larger penalty factors (e.g., 2.0 or higher). It is worth noting that the IOPS metric should be considered in conjunction with the storage device's queue depth and I / O size, as the same IOPS value may represent completely different system loads under different I / O modes.

[0157] Optionally, a disk wait time penalty factor can be determined by comparing the disk wait time metric data with a preset threshold. Disk wait time refers to the average time from initiation to completion of an I / O request, and is an important indicator for evaluating the response performance of a storage system. Unlike IOPS, which reflects the quantity of I / O operations, disk wait time reflects the quality of I / O operations, directly affecting the response time of database operations. Long disk wait times usually indicate that the storage subsystem has become a performance bottleneck, possibly due to device overload, storage architecture problems, or hardware failures. The setting of the disk wait time threshold also needs to consider the storage type: the typical wait time for SSDs is in the microsecond range, and the threshold can be set to 1-5 milliseconds; the typical wait time for HDDs is in the millisecond range, and the threshold can be set to 10-20 milliseconds.

[0158] In this embodiment, the disk latency penalty factor typically emphasizes penalizing long latency periods, as excessive latency not only affects current operations but can also lead to resource congestion, impacting the overall system throughput. When the latency is below a threshold, the penalty factor is small (e.g., 1.0); as the latency exceeds the threshold, the penalty factor may increase exponentially or superlinearly to reflect the accelerating effect of performance degradation. In practical applications, disk latency is usually considered in conjunction with IOPS, as these two metrics together provide a complete view of storage system performance: high IOPS but low latency indicates that the system is handling a large amount of I / O with good performance; low IOPS but high latency indicates that even with few I / O requests, the response is still slow, potentially indicating serious problems.

[0159] Subsequently, network latency metrics are compared with preset network latency thresholds to determine the network latency penalty factor. Network latency refers to the time required for data packets to travel from transmission to reception. In a distributed database environment, it directly impacts inter-node communication efficiency and client response time. Nodes in a MongoDB cluster need to perform data replication, state synchronization, and coordination operations over the network; therefore, network latency is a crucial factor affecting overall performance. Setting the network latency threshold requires consideration of network topology and physical distance: latency between nodes deployed in the same data center is typically below 1 millisecond, and the threshold can be set to 2-3 milliseconds; latency between nodes deployed across regions may reach tens or hundreds of milliseconds, requiring a correspondingly higher threshold.

[0160] In this embodiment, the network latency penalty factor is typically designed as a stepped or continuous function, increasing with latency. Lower latency ranges (e.g., less than 50% of the threshold) correspond to smaller penalty factors; higher latency ranges (e.g., close to or exceeding the threshold) correspond to larger penalty factors. It is particularly important to consider network latency jitter, as unstable network environments can be more detrimental than consistently high latency. A common approach is to combine average latency with the latency standard deviation, imposing additional penalties on highly volatile network connections. In practical deployments, the network latency threshold and penalty factor may need to be finely adjusted based on the application's network sensitivity and geographical distribution characteristics.

[0161] Finally, based on the preset node weights of the MongoDB nodes and the previously determined penalty factors, the final node weight value is calculated. The preset node weights are the basic weights pre-assigned by the system administrator to each node, reflecting the node's basic processing capacity and expected load. Preset values ​​are typically determined based on factors such as the node's hardware specifications, resource configuration, and deployment location. For example, nodes equipped with high-performance CPUs and large amounts of memory may receive higher preset values, while nodes located in remote data centers or with lower configurations receive lower preset values. This differentiated configuration ensures that, ideally, nodes with better hardware are prioritized when all performance metrics are equal.

[0162] In this embodiment, a multiplicative model is used to calculate the node weight value, which involves multiplying a preset value by various penalty factors to obtain the final weight value. The multiplicative model effectively reflects the combined effect of various factors; a significant deterioration in any one factor will lead to a significant decrease in the weight value. For example, the calculation formula can be expressed as: Node Weight Value = Preset Node Weight Value × CPU Penalty Factor × Memory Usage Penalty Factor × IOPS Penalty Factor × Disk Latency Penalty Factor × Network Latency Penalty Factor. This calculation method ensures that any performance bottlenecks are appropriately reflected in the weight value, guiding the distribution of requests to the optimal node.

[0163] It should be noted that an important consideration in calculating node weights is factor balancing, that is, the degree of influence of different performance metrics on the final weight. In practical applications, different types of database workloads may have different sensitivities to different resources: read-intensive workloads are usually more sensitive to memory and CPU; write-intensive workloads may be more sensitive to disk I / O; and analytical workloads may have high demands on both CPU and memory. Therefore, the design of the penalty factor and the selection of the threshold need to take into account the actual workload characteristics, which may require setting different profiles for different types of applications.

[0164] Furthermore, the dynamic adjustment of node weights is the core mechanism of the multi-node dynamic switching method. Performance metrics are continuously collected, and the latest penalty factors and weight values ​​are calculated to reflect real-time changes in node performance. When a node's performance deteriorates, causing its weight value to decrease, the load balancer reduces the number of requests allocated to that node; conversely, when a node's performance improves, its weight value increases, attracting more requests. This dynamic adjustment ensures efficient utilization of system resources and reasonable load distribution, avoiding the risk of any single point of overload.

[0165] Optionally, advanced implementations of weight calculation may also include historical data analysis and predictive models. By analyzing historical trends in node weight values, future performance fluctuations can be predicted, allowing for proactive adjustments to load balancing strategies. For example, if a node's weight value is detected to be continuously declining, new requests allocated to that node can be proactively reduced to prevent further performance degradation; or if a certain type of operation is observed to cause performance degradation on a specific node, such operations can be preferentially allocated to other more suitable nodes. This proactive adjustment helps improve system stability and responsiveness.

[0166] Therefore, this embodiment implements a dynamic load balancing mechanism in a MongoDB multi-node environment by setting thresholds and penalty factors for various performance indicators and calculating node weights based on these factors. Weight calculation based on multi-dimensional performance indicators comprehensively assesses the actual processing capacity and load status of nodes, providing a scientific and accurate basis for request distribution decisions. Compared to traditional static weighting or single-indicator methods, dynamic multi-dimensional evaluation can more effectively adapt to load changes and performance fluctuations, ensuring optimal utilization of system resources and continuous guarantee of service quality. This allows for close integration with the preceding indicator collection and processing steps, forming a complete and efficient dynamic switching method for MongoDB multi-nodes.

[0167] Step S5: Based on the connection pool management mechanism and the obtained node availability status data of the MongoDB node, determine the request distribution strategy according to the weight value of the MongoDB node. The request distribution strategy is used to indicate the distribution ratio of MongoDB requests to the MongoDB node.

[0168] It should be noted that node availability status data describes whether a MongoDB node is currently available for service. Node availability status is typically categorized into several states, such as "Available," "Warning," and "Unavailable," determined based on a comprehensive assessment of connection tests, response time tests, and metric collection tests. Nodes in the available state can receive and process requests normally; nodes in the warning state, while still operational, show signs of potential problems and their weight can be reduced; nodes in the unavailable state have been confirmed to have serious problems or are completely unconnectable, and are temporarily excluded from routing until recovery testing confirms their renewed availability. Node availability status data is crucial for request distribution, ensuring that requests are only sent to healthy nodes and preventing request failures or delays due to node failures.

[0169] In this embodiment, the selection probability is calculated based on the node weight value and is used to guide the weighted random routing algorithm in selecting the target node. For each available MongoDB node, its selection probability is equal to its weight value divided by the sum of the weight values ​​of all available nodes. For example, if there are three available nodes with weights of 0.8, 0.5, and 0.2, their selection probabilities are 0.8 / (0.8+0.5+0.2)=53.3%, 0.5 / (0.8+0.5+0.2)=33.3%, and 0.2 / (0.8+0.5+0.2)=13.3%, respectively. Therefore, this ensures that the proportion of requests received by a node is proportional to its weight, achieving performance-based load balancing.

[0170] The random number generator is the core component for implementing weighted random routing, randomly selecting a target node based on a pre-calculated selection probability. The random number generator first generates a uniformly distributed random number within the interval [0,1), and then determines the final selected node by comparing this random number with the cumulative distribution of the selection probabilities of each node. For example, using the three nodes example above, this embodiment generates a random number of 0.4. Since 0.4 is less than the selection probability of the first node (0.533), the first node is selected. If the random number is 0.6, since 0.6 is greater than 0.533 but less than 0.533 + 0.333 = 0.866, the second node is selected. This random selection ensures that the request allocation ratio matches the weight under long-term statistics while introducing randomness to avoid fluctuations that might arise from precisely allocating requests according to weight ratios.

[0171] In this embodiment, the request distribution strategy is used to indicate the set of rules for allocating MongoDB requests to different nodes. Weighted random routing is a commonly used distribution strategy that distributes requests proportionally based on the node's weight value, with nodes having higher weights receiving more requests. By configuring the request distribution strategy, the allocation ratio can be dynamically adjusted according to the real-time performance status of the nodes, making the load distribution more reasonable. In addition, the introduction of randomness avoids the performance fluctuations and synchronization effects that may result from allocating requests according to a completely fixed pattern. Besides weighted random, other distribution strategies are also supported, such as weighted round-robin (selecting nodes in turn according to weight ratio) or minimum connection count (prioritizing the node with the fewest active connections). The most suitable strategy can be selected according to actual needs through configuration.

[0172] In some embodiments, step S5 specifically includes:

[0173] Based on the weight value of the MongoDB node and the node availability status data of the MongoDB node, the selection probability of the MongoDB node is calculated, and a target MongoDB node is determined according to the selection probability using a preset random number generator, so as to generate a request distribution strategy based on the target MongoDB node.

[0174] If the identified target MongoDB node fails to respond to the MongoDB request, then based on the node availability status data, a MongoDB node marked as available is selected to respond to the MongoDB request.

[0175] In this embodiment, the request distribution strategy employs a weighted random routing algorithm. The selection probability is calculated based on the real-time weights of each node, and then a random number generator is used to select the target node according to this probability. This ensures that, in the long run, the proportion of requests received by each node is proportional to its weight, while introducing randomness avoids the fluctuation problems caused by precisely allocating requests according to weight proportions. When the selected target node fails to respond, it is automatically reselected from other available nodes to ensure that requests can be processed normally.

[0176] Specifically, the selection probability of each node is first calculated based on its weight value and availability status data. The selection probability is a normalized value representing the relative likelihood of selecting a particular node during request distribution. It is a function of the weight value, ensuring that nodes with better performance (higher weight values) have a greater probability of being selected to process requests. Node availability status data is a binary label indicating whether a node is currently able to accept and process requests. Only nodes marked as available are included in the selection process, ensuring that the system does not send requests to nodes known to be faulty or under maintenance.

[0177] In this embodiment, the selection probability is typically calculated using a weighted proportional allocation method: first, all nodes in available states are selected; then, the proportion of each available node's weight to the total weight is calculated, and this proportion is the selection probability of that node. For example, assuming there are three available nodes A, B, and C with weights of 10, 20, and 30 respectively, their selection probabilities are 10 / (10+20+30)=1 / 6, 20 / 60=1 / 3, and 30 / 60=1 / 2 respectively. By calculating the selection probability, it is ensured that node selection is proportional to its performance level, achieving reasonable load distribution. In actual implementations, the selection probability is usually stored in the form of a cumulative distribution function (CDF) for easy subsequent random selection.

[0178] Furthermore, the maintenance of node availability status data is used for request distribution. Availability status is determined by several factors: node health check results, periodically performed checks (such as heartbeat checks) confirming node operation; explicit maintenance flags, temporary unavailability flags set by administrators for planned maintenance or upgrades; and historical failure records, which may mark frequently failing nodes as unavailable based on recent failure patterns. Availability status data is typically stored in a distributed manner to ensure that all clients or proxy servers can access the latest status information, avoiding sending requests to failed nodes.

[0179] Secondly, a pre-defined random number generator is used to determine the target MongoDB node according to the calculated selection probability. A random number generator is an algorithm that produces random values ​​conforming to a specific distribution (usually a uniform distribution), used here to implement probability-based node selection. Random selection, rather than simple round-robin or fixed allocation, better adapts to dynamically changing load conditions and avoids mismatches between request patterns and system resource allocation. A pre-defined random number generator typically refers to a pseudo-random number generation algorithm pre-configured by the system, such as the Linear Congruential Generator, the Mersenne Twister algorithm, or a cryptographically secure random number generator. Choosing an appropriate random number generator requires considering several factors: randomness quality, ensuring the generated sequence has no obvious patterns or periodicity; performance overhead, as high-quality random algorithms may have higher computational costs; and concurrency safety, avoiding race conditions in multi-threaded environments.

[0180] Furthermore, the process of determining the target MongoDB node typically employs techniques such as Roulette Wheel Selection or the Alias ​​Method. The basic steps of Roulette Wheel Selection are: generating a random number within the interval [0,1); treating the selection probability of each node as a cumulative probability interval; and determining which node's probability interval the random number falls into; that node is the selected target node. For example, for the previously mentioned nodes A, B, and C, their cumulative probability intervals are [0,1 / 6), [1 / 6,1 / 2), and [1 / 2,1), respectively; if the random number is 0.4, it falls into the probability interval of node B, therefore B is selected as the target node. The Alias ​​Method is a more efficient implementation technique that reduces the time complexity of the selection operation to O(1) through preprocessing, making it suitable for high-load systems that frequently perform node selection.

[0181] Therefore, generating a request distribution strategy based on the target node is the process of transforming the node selection result into a specific routing decision. The request distribution strategy includes several aspects: connection parameters of the target node, such as IP address, port number, and authentication information; request timeout settings, specifying the maximum time to wait for a response; retry strategy, defining the number of retries and intervals when a request fails; and additional instructions, such as read / write preference settings and consistency level requirements. Generating a request distribution strategy needs to consider the characteristics of the request: read requests may tend to choose nodes with excellent read performance, such as slave nodes in a replica set; write requests must be sent to nodes with write permissions, such as the master node of a replica set or a specific shard in a sharded cluster. In some complex scenarios, further optimization of the distribution strategy is needed, such as routing analytical queries to dedicated analytical nodes, or routing operations on a specific set to nodes containing data from that set.

[0182] Optionally, this embodiment sets a fault handling mechanism when the determined target MongoDB node fails to respond to a MongoDB request. Response failure refers to the situation where the target node cannot process the request and return a result normally, which may be caused by various reasons: node crash or restart, resulting in connection interruption; node overload, unable to process the request within the expected time; network failure, with the request or response lost during transmission; software errors, such as query parsing errors or execution exceptions. The determination of response failure is usually based on a timeout mechanism and error code analysis: if the request does not receive a response within a preset timeout period, the request is considered to have failed; or if the received response contains a specific error code (such as connection refused, service unavailable, etc.), it is also considered a failure.

[0183] Therefore, handling response failures is a critical capability of the request distribution system, directly impacting reliability and user experience. When a target node's response failure is detected, other MongoDB nodes marked as available are selected to respond to the request based on the node's availability status data. This process typically includes the following steps: marking the failed node, temporarily marking the node that failed to respond as unavailable or reducing its selection probability; updating the availability status data, synchronizing the failure information to a centralized state store; recalculating the selection probability, excluding the failed node and reallocating the probability values; selecting a backup node, using the same random selection mechanism to determine a new target node from the remaining available nodes; and resending the request, forwarding the original request to the newly selected node for processing. Through dynamic adjustments, rapid response to node failures is ensured, minimizing the impact of failures on users.

[0184] In advanced implementations, failure handling may involve more complex mechanisms: cascading failure detection to analyze whether the failure is a single node or a broader issue such as network partitioning; failure mode identification to identify specific types of failure modes and take targeted measures; request classification and processing to adjust retry strategies based on the importance and urgency of requests; service degradation to provide limited but still available service in the event of severe failures; and recovery monitoring to continuously monitor the recovery status of failed nodes and restore their role in load balancing as needed. These mechanisms together constitute a robust failure recovery system, ensuring that the MongoDB cluster can maintain service even if some nodes fail.

[0185] In response, a crucial consideration in fault handling is loopback prevention, which avoids repeatedly sending requests to the same group of failed nodes, thus preventing resource waste and response delays. Common preventative measures include: failure counters, which record the number of consecutive failures for each node and forcibly mark it as unavailable once a threshold is exceeded; cooling-off periods, where failed nodes are excluded from selection for a certain period, giving them a chance to recover; and request marking, which adds a record of already attempted nodes to requests to ensure that duplicate selections are not made. These measures collectively ensure the effectiveness and efficiency of the fault handling mechanism.

[0186] Restoring node availability is the final step in the fault handling cycle. Once a failed node recovers, it needs to be reintegrated into the load balancer. This process typically includes: periodic health checks, periodically probing the status of nodes marked as unavailable; recovery confirmation, requiring multiple consecutive successful health checks to confirm node recovery; smooth integration, gradually increasing the probability of selecting recovered nodes to avoid sudden load surges; and state synchronization, notifying all relevant components of node state changes to ensure system consistency. Through recovery, available resources can be maximized while maintaining stability.

[0187] Therefore, this embodiment achieves intelligent request distribution in a MongoDB multi-node environment by calculating the selection probability based on node weight and availability, determining the target node using a random number generator, and establishing a failure response handling mechanism. Dynamic distribution based on the selection probability, combined with the performance metrics collected and calculated in the previous steps, can effectively address node failures and load changes while ensuring overall performance. Compared to simple round-robin or static allocation methods, this dynamic probability distribution mechanism is more adaptable to complex distributed environments, providing higher availability, more balanced resource utilization, and a better user experience. This step is the final execution stage of the entire MongoDB multi-node dynamic failover method, translating the results of all previous steps into actual load balancing actions, ensuring efficient and stable operation.

[0188] Step S6: The data access layer performs status detection on the MongoDB node. When the MongoDB node is detected to meet the preset abnormal status conditions, the detected MongoDB node is marked as unavailable and the node availability status data is updated.

[0189] In this application, state detection is implemented through a health detector component, which periodically (once every 10 seconds by default) performs health checks on each MongoDB node, including multiple dimensions such as connection testing, simple query testing, and metric collection testing. When a node fails consecutively to a threshold (3 times by default), the node is automatically marked as unavailable and removed from the routing target. A node that has been tripped enters a quiet period (5 minutes by default), after which a recovery check is performed. The recovery check is more stringent than a regular health check, requiring multiple consecutive successes (3 times by default) before the node is marked as available and reinstated into the routing selection scope.

[0190] Status detection is a mechanism for real-time monitoring of the health status of MongoDB nodes. It periodically performs a series of detection operations to assess the availability and performance of nodes. In distributed database systems, node status detection is a key component for ensuring stability and reliability, enabling timely detection of faulty nodes and triggering automatic repair or isolation measures to prevent the spread of failures. MongoDB node status detection typically includes multiple dimensions: connection testing checks network connectivity to confirm that the node can establish TCP connections; simple query testing verifies that the node can correctly respond to basic database operations, such as executing the `serverStatus` command; and metric collection testing confirms the ability to obtain node performance metric data for health status assessment. Status detection is performed by the health detector component, which, as part of the data access layer, can efficiently perform detection tasks using connection pool resources.

[0191] In this application, the abnormal state condition is a predefined set of rules used to determine whether a MongoDB node is in an abnormal state. This abnormal state condition may include: consecutive failure count, referring to the number of consecutive failures of a node's response reaching a threshold (usually 3 times); response timeout, referring to the node's response time exceeding a preset timeout threshold (e.g., 5 seconds); abnormal metrics, referring to key performance indicators such as CPU utilization and memory usage exceeding safe limits; and replication latency (for replica set members), referring to the latency of replicating data from the master node exceeding an acceptable range. The setting of abnormal state conditions needs to balance sensitivity and stability: overly lenient conditions may cause nodes with actual problems to still be considered available; overly strict conditions may frequently mark normal nodes as unavailable due to short-term fluctuations. This embodiment typically allows adjustment of the abnormal state condition parameters through configuration files to adapt to different operating environments and business needs.

[0192] Subsequently, node tagging is the process of recording the state information of MongoDB nodes into the system's internal data structure. Node state tags typically include: the current state value (e.g., "available," "warning," or "unavailable"); a state change timestamp, recording the time of the last state change; a state change reason, describing the specific factors that caused the state change (e.g., "3 consecutive connection timeouts"); and historical state records, saving the history of node state changes for analyzing failure modes and trends. The node tagging mechanism uses atomic operations to ensure data consistency in high-concurrency environments, while simultaneously using an event notification mechanism to propagate state changes to all system components in real time, especially the load balancing engine, enabling it to adjust request distribution strategies promptly.

[0193] Furthermore, the circuit breaker mechanism is a fault isolation technique originating from the field of circuit protection, used to prevent a system from continuously attempting operations that may fail. In a MongoDB multi-node system, the circuit breaker mechanism is used to temporarily isolate nodes that have been detected as abnormal, preventing requests from continuing to be sent to these nodes, thereby avoiding request failures, response delays, and resource waste. The workflow of the circuit breaker mechanism includes: a detection phase, where abnormal nodes are detected through status inspection; a circuit breaker phase, where the node is marked as unavailable and excluded from routing targets; a cooling-off phase, where the circuit-breaker-damaged node enters a quiet period (default 5 minutes), during which it does not participate in regular routing selection; and a recovery detection phase, where a special recovery detection is performed after the quiet period to verify whether the node has returned to normal. The circuit breaker mechanism effectively reduces dependence on faulty nodes, improving overall stability and response speed.

[0194] It should be noted that the quiet period is an important concept in the circuit breaker mechanism. It refers to a specific time period after a node is marked as unavailable, during which the node will not be included in the normal routing selection. The purpose of the quiet period is to prevent frequent fluctuations in node state (i.e., the so-called "jitter" problem), giving the system sufficient time to stabilize and allowing failed nodes enough time to recover. The length of the quiet period is usually dynamically adjusted based on the node's historical reliability: for nodes experiencing their first or occasional failures, a shorter quiet period (e.g., 1-5 minutes) may be used; for nodes experiencing frequent failures or persistent performance issues, a longer quiet period (e.g., 15-30 minutes) may be used to prevent them from prematurely rejoining and affecting system stability again. Quiet period management is a core component of the automatic circuit breaker recovery mechanism; a reasonable quiet period configuration can significantly improve the stability of a distributed system.

[0195] Subsequently, recovery detection is a specialized process to verify whether a circuit-breaker node has returned to normal. It is more stringent than regular state detection, requiring nodes to demonstrate sustained stability before being reintegrated into the routing target. Key characteristics of recovery detection include: continuous success requirements—a node must pass multiple consecutive checks (usually three) to be considered recovered; incremental intervals—the check intervals gradually increase (e.g., initially 10 seconds, then 30 seconds, and finally 60 seconds) to verify the node's stability across different time windows; comprehensive testing—not only checking basic connectivity but also verifying the normality of key indicators such as query performance and resource usage; and gradual load balancing—in the initial recovery phase, a small number of requests may be allocated for actual business testing, and normal load balancing is only implemented after full recovery is confirmed. The recovery detection mechanism ensures that only truly stable nodes re-engage in request processing, avoiding service fluctuations caused by premature recovery.

[0196] In this application, updating node availability status data is the process of synchronizing the latest node status information to all system components, ensuring that functions such as load balancing, request routing, and monitoring and alarms can make decisions based on the latest status. For example, the update process includes: status data persistence, writing the node status to an in-memory data structure, and, if necessary, persisting it to disk or a distributed cache to prevent status loss due to service restarts; event notification, sending node status change events internally to enable timely responses from all components; routing table updates, where the load balancing engine adjusts the node's routing weight based on the status change to ensure requests are not sent to unavailable nodes; and monitoring and alarms, triggering corresponding monitoring alarms when a node becomes unavailable to notify operations personnel to pay attention to and address potential problems. The node availability status data update mechanism adopts a publish-subscribe pattern to ensure that status changes can be efficiently and reliably propagated to all parts of the system.

[0197] Optionally, a state recovery strategy is a set of rules that guides nodes to recover from an unavailable state to an available state. It defines the conditions, procedures, and follow-up processing for recovery. A robust state recovery strategy typically includes the following aspects: gradual recovery, where nodes are initially rejoined with a lower weight and gradually increased to normal levels to avoid sudden increases in load; monitoring period management, where recovered nodes enter a special monitoring period (e.g., 30 minutes) during which the system checks their status more frequently and immediately re-breaks the circuit breaker if a problem occurs; fault statistical analysis, where the system records and analyzes the node's fault history, identifies possible fault modes and root causes, and may take special measures for nodes with recurring faults (e.g., extending the quiet period or reducing the base weight); and recovery notification, where a recovery notification is sent after a node successfully recovers, informing operations personnel of the system status change. The core objective of a state recovery strategy is to ensure that nodes are safely and smoothly reintegrated into the service, while preventing the recurrence of the same problems.

[0198] Therefore, this application constructs an automated health management system by implementing comprehensive functions such as state detection, node marking, circuit breaker mechanism, quiet period management, recovery detection, and state data updating. This system proactively identifies and isolates abnormal nodes to prevent the spread of faults and maintain overall service quality. Simultaneously, it automatically reintegrates nodes into the service after recovery, reducing manual intervention and improving the system's self-healing capabilities and availability. Through automated health management, combined with the data access layer, metric collection, weight calculation, and dynamic routing described in the preceding steps, a comprehensive and efficient method for dynamic multi-node switching in MongoDB is established, enabling stable operation and optimal performance in various complex environments.

[0199] In one specific embodiment, step S5 further includes:

[0200] Based on the weight values, performance metrics, availability status data, and connection relationships of the MongoDB nodes, a MongoDB connection network graph is constructed. The connection relationships indicate the data dependencies between the MongoDB nodes. The vertices in the MongoDB connection network graph represent the MongoDB nodes, and the vertex weights of the vertices are determined by the performance metrics. The edges in the MongoDB connection network graph represent the connection relationships between the MongoDB nodes.

[0201] Based on the MongoDB connection network graph, an NMC sparser model is constructed, and the minimum cut set in the MongoDB connection network graph is identified according to the NMC sparser model. The minimum cut set is used to indicate MongoDB nodes whose node load exceeds a preset load threshold and / or MongoDB nodes whose network connection stability is lower than a preset stability threshold.

[0202] Based on the MongoDB connection network graph and the minimum cut set, determine the target node path from the specified source node to the MongoDB node;

[0203] Based on the minimum cut set and the weight values ​​of the MongoDB nodes, the performance of the request distribution strategy is predicted to obtain the performance prediction results, which include system bottleneck prediction values ​​and performance loss prediction values.

[0204] Based on the weight values ​​of the MongoDB nodes and the path of the target node, the top n MongoDB nodes are selected as candidate nodes.

[0205] The bottleneck risk of the candidate node is evaluated using the NMC sparser model, and the performance evaluation value of the candidate node is determined based on the bottleneck risk of the candidate node, the weight value, and the performance prediction result.

[0206] Candidate nodes whose performance evaluation values ​​are greater than a preset processing capacity threshold are selected as target MongoDB nodes.

[0207] In this embodiment, a more complex request distribution strategy is implemented by constructing a MongoDB connection network graph and an NMC sparser model. First, MongoDB nodes and their connections are abstracted into a network graph. Then, the NMC sparser is used to identify the minimum cut set in the network; these cuts represent potential bottlenecks in the system. Based on the network graph and the minimum cut set, the optimal path from the source node to the target node can be determined, avoiding potential bottlenecks. Next, the top n candidate nodes are selected based on node weights and the target path, and the bottleneck risk of these candidate nodes is evaluated using the NMC sparser. The final target node is determined by combining the weights and performance prediction results. This embodiment considers not only the performance status of individual nodes but also the network structure and traffic distribution of the entire system, enabling more accurate identification and avoidance of potential performance bottlenecks.

[0208] Specifically, a MongoDB connection network graph is first constructed based on the weight values, performance metrics, availability status data, and connectivity relationships of MongoDB nodes. The MongoDB connection network graph is a graph-based data structure used to represent and analyze node relationships and data flow in a MongoDB cluster. In this graph model, vertices represent MongoDB nodes, and edges represent connections between nodes. This intuitive representation provides a more comprehensive understanding of the cluster's topology and performance distribution. Node connectivity relationships refer to the data dependencies and communication patterns between MongoDB nodes, including replication relationships, sharding dependencies, and configuration server connections. These relationships determine how data flows within the system and how node failures affect the overall service.

[0209] In a replica set architecture, node connections primarily manifest as a unidirectional replication flow between the master and slave nodes: the master node receives write operations and replicates the operation log (oplog) to the slave nodes. In a sharded cluster, the connections are more complex: shards may not communicate directly but all rely on the configuration server; the routing server (mongos) needs to establish connections with all shards and the configuration server; and replication relationships exist between replica set members within a shard. These connections are represented in the graph by directed or undirected edges, depending on whether the data flow is unidirectional or bidirectional.

[0210] The weights of vertices in the MongoDB connection network graph are determined by node performance metrics, which quantify the node's processing capacity and load status as vertex attributes in the graph. Vertex weight is typically a comprehensive metric that integrates multiple performance indicators such as CPU utilization, memory usage, IOPS, disk wait time, and network latency. Higher weight values ​​indicate better node performance and suitability for handling requests; lower weight values ​​indicate limited performance, and the number of requests allocated to that node should be reduced. Edge attributes reflect the quality and importance of the connection, potentially including bandwidth capacity, latency level, and data dependency. Critical connections (such as replication links from master to slave nodes) usually have higher edge weights, indicating their significant impact on system stability.

[0211] The process of constructing a MongoDB connection network graph involves several steps: node identification, determining all active nodes in the cluster and their roles (master, slave, configuration server, etc.); relationship mapping, determining the connection relationships between nodes based on the cluster configuration and replication topology; attribute assignment, assigning weight values ​​to vertices and edges based on performance metrics and connection quality; graph structure verification, ensuring that the constructed graph model accurately reflects the actual cluster state; and dynamic updates, updating the graph model in real time based on performance monitoring and cluster changes. This graph-based representation provides an intuitive and powerful foundation for subsequent analysis and decision-making.

[0212] Next, based on the MongoDB network graph, we construct the NMC (Network Minimum Cut) sparser model. The NMC sparser is a graph algorithm model based on the minimum cut theory of networks, used to identify critical bottlenecks and vulnerabilities in a network. In graph theory, a cut is a set of edges that divides a graph into two disconnected parts; a minimum cut is the cut with the smallest sum of edge weights. The NMC sparser model identifies critical nodes or connections in the network whose removal would lead to network splitting or a significant performance degradation by finding the minimum cut.

[0213] It should be noted that the NMC sparser model is built based on classic network flow algorithms, such as the Ford-Fulkerson algorithm, the Edmonds-Karp algorithm, or the Karger algorithm. By simulating traffic transmission in a network, bottlenecks limiting overall traffic are identified. In a MongoDB environment, the NMC sparser needs to consider several special factors: differences in node processing capabilities (different nodes may have different hardware specifications and performance levels); the impact of replication latency (data synchronization delays between master and slave nodes may lead to data inconsistency); and shard balance (uneven data distribution may cause some shards to be overloaded). The construction process includes: graph transformation (converting the MongoDB connected network graph into a format suitable for network flow algorithms); capacity definition (defining appropriate capacity values ​​for edges and vertices to reflect their processing capabilities); algorithm execution (running the selected minimum cut algorithm); and result interpretation (mapping the algorithm results back to the MongoDB cluster context).

[0214] Based on the NMC sparser model, identify the minimum cut set in the MongoDB connection network graph. The minimum cut set is a set of nodes or edges in the graph whose removal would split the graph into disconnected parts, with the total weight of the removed elements being minimized. In a MongoDB cluster environment, the minimum cut set typically corresponds to system bottlenecks or vulnerabilities: MongoDB nodes whose load exceeds a preset load threshold (i.e., servers that have reached or are about to reach their processing capacity limits); and MongoDB nodes whose network connection stability is below a preset stability threshold (i.e., servers with poor connection quality or frequent disconnections). These nodes constitute potential risks to system performance and reliability.

[0215] Optionally, the identification process for the minimum cut set involves multi-dimensional analysis: performance threshold comparison, comparing node performance metrics with preset thresholds to identify overloaded nodes; connection quality assessment, analyzing network connection stability and latency to identify problematic connections; historical failure analysis, considering node historical failure records to identify unstable nodes; and dependency weighting, considering the role and importance of nodes in the cluster, with master nodes or configuration servers typically being more critical. Through this comprehensive analysis, the system can accurately identify nodes and connections that may become system bottlenecks, providing guidance for subsequent path planning and load distribution.

[0216] In this embodiment, based on the MongoDB connection network graph and the minimum cut set, the target node path from the specified source node to the MongoDB node is determined. The target node path refers to the network transmission path from the request initiation point (source node, typically the application server or proxy server) to the potential processing node (target node, the MongoDB server). These paths determine the efficiency and reliability of request transmission, directly impacting the final service quality. When determining the path, nodes and connections within the identified minimum cut set are avoided as much as possible to reduce the risk of requests being affected by bottlenecks.

[0217] For example, path determination typically employs classic graph path algorithms, such as Dijkstra's algorithm, A* algorithm, or multi-constraint path algorithms. These algorithms consider various factors: path length, as shorter paths generally mean lower latency; node performance, as the processing capacity of nodes along the path affects request processing time; connection quality, as high-quality connections provide more stable transmission; and bottleneck avoidance, avoiding known bottlenecks and unstable connections. In complex distributed environments, path determination may also need to consider geographical factors (such as high latency across data center connections) and load balancing requirements (to avoid all requests choosing the same path).

[0218] Subsequently, performance prediction is performed on the request distribution strategy based on the minimum split set and the weight values ​​of the MongoDB nodes, yielding the performance prediction results. Performance prediction is a forward-looking analysis technique that simulates system behavior under a specific request distribution strategy to estimate its potential performance conditions. Through performance prediction, the effectiveness of different strategies can be evaluated before actual implementation, the optimal solution can be selected, and potential performance problems can be avoided. The performance prediction results include two core metrics: a system bottleneck prediction value, which estimates the location and extent of the most likely bottleneck in the system under a specific distribution strategy; and a performance degradation prediction value, which estimates the potential performance reduction due to bottlenecks and other limiting factors.

[0219] Performance prediction is typically based on queuing theory models or discrete event simulation techniques. Queuing theory models treat MongoDB nodes as servers and requests as clients, predicting system performance by analyzing service rates, arrival rates, and queue behavior. Commonly used queuing models include M / M / 1 (single-server exponential distribution), M / M / c (multi-server exponential distribution), or more complex network queuing models. Discrete event simulation, on the other hand, analyzes performance characteristics in more detail by simulating the flow and processing of requests within the system. Regardless of the method used, the prediction process needs to consider several factors: request characteristics, as different types of requests (read / write, simple / complex queries) have different processing requirements; load levels, as behavior may differ significantly under different loads; resource constraints, such as the impact of CPU bottlenecks, memory bottlenecks, or I / O bottlenecks; and failure scenarios, such as the system response when some nodes or connections fail.

[0220] Next, based on the weight values ​​of the MongoDB nodes and the target node path, the top n MongoDB nodes are selected as candidate nodes. Candidate node selection is the process of narrowing down all available nodes to a high-quality subset of manageable size, ensuring that the final target node is selected from these most promising candidates. The selection process is typically based on a multi-factor weighted ranking: node weight, reflecting the node's basic processing capacity and current load status; path quality, considering the efficiency and stability of the network path from the source to the node; historical performance, the node's performance over a past period; and load balancing factors, ensuring that requests are not excessively concentrated on a few nodes. The choice of n needs to balance comprehensiveness and efficiency: n that is too small may exclude some potentially high-quality nodes, while n that is too large increases the computational burden of subsequent evaluations. Typically, n is set as a reasonable proportion of the total number of nodes, such as 20%, or a fixed number (e.g., 5-10).

[0221] Next, the bottleneck risk of candidate nodes is assessed using the NMC sparser model. Bottleneck risk assessment is a process of quantitatively analyzing the performance limitations that candidate nodes may face when processing new requests. The NMC sparser model identifies which candidate nodes are located at or near the minimum cut set, and these nodes have a higher bottleneck risk. The assessment process considers several risk factors: load proximity (how close the node's current load is to its capacity limit); connection dependency (the node's dependence on potentially unstable connections); cluster location (whether the node's position in the cluster topology is critical); and resource sharing (whether the node shares critical resources with other high-load nodes). The bottleneck risk assessment results are typically expressed as a risk score or risk level (e.g., low, medium, high) for subsequent comprehensive evaluation.

[0222] Subsequently, based on the bottleneck risk, weight values, and performance prediction results of the candidate nodes, the performance evaluation value of the candidate nodes is determined. The performance evaluation value is a comprehensive indicator that fully reflects the suitability of a node for handling new requests. By weighted integration of multiple dimensions, a simple and intuitive comparison benchmark is provided. A typical method for calculating the performance evaluation value is a multi-factor weighted model: first, all indicators (bottleneck risk, node weight, prediction performance, etc.) are normalized to the same scale (e.g., 0-1 or 0-100); then, weight coefficients are assigned according to the importance of each indicator; finally, the weighted average is calculated as the final performance evaluation value. Weight allocation needs to be adjusted according to application characteristics and performance goals: for example, for latency-sensitive applications, network latency and processing speed may be given more importance; while for applications with high availability requirements, stability and risk avoidance may be given more importance.

[0223] Finally, candidate nodes with performance evaluation values ​​greater than a preset processing capacity threshold are selected as the final target MongoDB nodes. The processing capacity threshold is a preset minimum performance standard that ensures the selected nodes have sufficient capacity to handle the requests to be distributed. Only nodes with performance evaluation values ​​exceeding this threshold are considered as final targets. This mechanism avoids distributing requests to underperforming nodes, reducing the risk of service quality degradation. Setting the threshold requires a balance between availability and performance: a threshold that is too high may result in too few selectable nodes, or even no nodes meeting the criteria; a threshold that is too low may allow poorly performing nodes to receive requests, impacting user experience. In practical applications, the processing capacity threshold may be dynamically adjusted based on system load and available resources: during low-load periods, the threshold can be increased to optimize performance; during peak periods or when resources are scarce, the threshold may need to be decreased to ensure service availability.

[0224] Optionally, if the performance evaluation values ​​of multiple candidate nodes all exceed the processing capacity threshold, different strategies may be used to select the final target node: optimal selection, choosing the node with the highest performance evaluation value; random weighted selection, randomly selecting nodes according to the proportion of performance evaluation values ​​to avoid all requests being concentrated on a single "best" node; round-robin selection, distributing requests among qualified nodes in turn to ensure load balancing; and a hybrid strategy, combining the above methods and flexibly selecting based on request type and system status. The finally selected target MongoDB node will receive and process the distributed requests, monitor its response, and feed this information back to the performance metric collection and node evaluation system, forming a closed-loop optimization.

[0225] To address this, this embodiment significantly enhances the intelligence of request distribution in MongoDB multi-node environments through target node selection. By constructing a network graph model, applying a minimum cut algorithm to identify bottlenecks, and conducting multi-dimensional performance evaluation, it can make optimal distribution decisions in complex and changing environments. Compared to traditional static weighting or simple rule-based methods, this graph theory- and predictive model-based approach can more accurately grasp the overall state and potential risks of the system, achieving more efficient and reliable load balancing. Especially in large-scale distributed environments, this approach's advantages are even more pronounced, effectively addressing challenges such as a large number of nodes, complex topologies, and variable load patterns, ensuring high system performance and stable operation.

[0226] Therefore, this embodiment constructs a highly intelligent request distribution decision system. Through MongoDB connection network graph construction, NMC sparser model application, minimum cut set identification, target path determination, performance prediction, candidate node screening, bottleneck risk assessment, and comprehensive performance evaluation, it can comprehensively analyze the cluster status, identify potential risks, and make optimal distribution decisions. This method, which deeply integrates graph theory, network flow theory, performance modeling, and risk assessment, represents the cutting edge of load balancing technology and provides powerful performance optimization and stability guarantees for MongoDB multi-node environments.

[0227] In another embodiment, step S3 further includes:

[0228] Based on the performance characteristics of the MongoDB nodes, the node connection relationships, the configuration information, the node performance indicators, and the node availability status data, the MongoDB nodes are clustered.

[0229] MongoDB nodes belonging to the same group are hierarchically structured according to the node performance metrics, a tree structure is created based on the hierarchical relationship, and multiple tree structures are organized into a dynamic forest model. The dynamic forest model is used to indicate the node status that manages the MongoDB nodes.

[0230] When a change in the metric data of any MongoDB node's running metrics is detected, the updated metric data is obtained, the dynamic forest model is updated based on the updated metric data, and the corresponding tree structure change event is determined.

[0231] Based on the dynamic forest model, the tree structure change events, and the operational metrics, a multidimensional index is constructed, which is used to indicate the indexing and retrieval of the operational metrics.

[0232] In this embodiment, efficient node state management is achieved through a dynamic forest data structure. First, MongoDB nodes are clustered based on their performance characteristics and connectivity, grouping similar nodes into the same group. Then, a hierarchical relationship is constructed according to node performance metrics, creating a tree structure, and multiple tree structures are organized into a dynamic forest model. When the performance metrics of any node change, the dynamic forest model is updated, and the corresponding change event is determined. Finally, a multi-dimensional index is built based on the dynamic forest model and performance metrics, supporting efficient indexing and retrieval of node states. This method is particularly suitable for large-scale MongoDB clusters, maintaining efficient state management and query performance even with a large number of nodes.

[0233] Specifically, the first step is to cluster MongoDB nodes based on their performance characteristics, node connectivity, configuration information, performance metrics, and availability status data. Node clustering is an unsupervised learning method that aims to group MongoDB nodes with similar characteristics into logical clusters or clusters. This grouping allows the system to manage a large number of nodes more effectively, simplifies monitoring and decision-making processes, and provides a more granular unit of control for load balancing. Node clustering is particularly valuable in MongoDB environments because different nodes may have different hardware configurations, network environments, data distributions, and usage patterns. Clustering can identify these naturally occurring groups and optimize resource allocation accordingly.

[0234] It should be noted that the feature dimensions used in node clustering are very comprehensive: performance features include hardware specifications (number of CPU cores, memory capacity, disk type, etc.), software configuration (MongoDB version, storage engine type, etc.), and performance benchmark test results; node connectivity reflects the network topology and data dependencies between nodes, such as replica set membership, sharding dependencies, configuration server connections, etc.; configuration information covers node role settings (master node / slave node / arbitrator), read and write permissions, authentication mechanisms, and special function activation status; node performance metrics provide real-time performance data, such as CPU utilization, memory utilization, IOPS, network latency, etc.; node availability status data reflects the health status and service availability of nodes.

[0235] Optionally, the choice of clustering algorithm depends on the data characteristics and objectives: K-means is suitable for spherical clusters and uniformly sized clusters, and is simple and efficient to implement; DBSCAN (density-based spatial clustering) is suitable for identifying clusters of arbitrary shapes and can detect outliers; hierarchical clustering can create nested clusters, suitable for situations requiring multi-level views; and spectral clustering performs excellently when handling complex connections. In a MongoDB environment, the clustering objective is usually to identify groups of nodes with similar performance characteristics and connection patterns for unified management and optimization. Clustering results may reflect various natural groupings: node clusters grouped by geographical location, performance levels classified by hardware configuration, functional groups clustered by data distribution characteristics, or load categories divided by usage patterns.

[0236] For example, the node clustering process includes several steps: feature extraction and standardization, which converts the multidimensional features of nodes into numerical vectors that the algorithm can process and performs normalization; similarity calculation, which uses appropriate distance metrics (such as Euclidean distance, cosine similarity, etc.) to calculate the similarity between nodes; clustering algorithm application, which executes the selected clustering algorithm and determines the optimal parameters (such as the K value in K-means); clustering evaluation, which uses internal metrics (such as silhouette coefficient, Davies-Bouldin index, etc.) and external knowledge to verify the clustering quality; and clustering result interpretation, which maps the abstract clustering results back to the MongoDB management context and assigns meaningful labels and feature descriptions to each cluster. Clustering is not a one-time operation; as node performance and configuration change, the clustering results need to be re-evaluated periodically to ensure they reflect the current state.

[0237] Secondly, MongoDB nodes belonging to the same group are organized into a hierarchical relationship based on node performance metrics. A tree structure is then created based on this hierarchy, and multiple tree structures are organized into a dynamic forest model. A tree structure is a hierarchical data structure, well-suited for representing hierarchical relationships and dependencies between nodes. In this context, the tree structure is used to organize MongoDB nodes within the same cluster, arranging them into a multi-level architecture based on performance metrics. The dynamic forest model is a collection of trees, representing the organizational structure of the entire MongoDB cluster. This forest model is called "dynamic" because it can be updated in real time as the node states change, reflecting the current state of the cluster.

[0238] In this embodiment, the hierarchical structure is built based on a comprehensive evaluation of node performance metrics. High-performance nodes are typically located at the upper levels of the tree, serving as parent or root nodes; low-performance nodes are located at the lower levels, serving as child or leaf nodes. This arrangement reflects the performance differences between nodes and the potential load distribution order: upper-level nodes receive requests first, while lower-level nodes serve as alternatives or auxiliary nodes. The hierarchical structure considers various performance factors: processing power, such as CPU performance and memory capacity; current load, reflecting the node's resource utilization status; connection quality, including network latency and bandwidth; and stability metrics, such as error rate and downtime frequency. Through these multi-dimensional performance evaluations, a hierarchical structure that reflects both hardware differences and actual operating conditions can be created.

[0239] The creation of the tree structure follows specific organizational principles: each node cluster forms an independent tree; the root node is typically the best-performing node within the group, possibly the master or coordinating node; intermediate-level nodes are sorted according to performance metrics, forming secondary node groups; leaf nodes are nodes with lower performance or special roles (such as arbitrators). The depth and branching factor of the tree depend on the size of the clusters and the distribution of node performance: large groups may form deeper trees, while groups with significant performance differences may form wider trees. In some implementations, the tree structure may employ special topologies, such as B-trees, red-black trees, or AVL trees, to optimize the performance of specific operations.

[0240] Furthermore, the dynamic forest model integrates multiple tree structures into a unified management model. Each tree in the forest represents a cluster of nodes, and connections may exist between trees, reflecting the interactions between clusters. The dynamic nature of the forest model is reflected in its responsiveness to changes in node state: changes in node performance may lead to adjustments in its position within the tree; changes in node roles (such as a node being promoted to a master node) may trigger tree structure reorganization; the addition or departure of nodes may trigger the expansion or contraction of the tree. This dynamic adjustment ensures that the forest model always reflects the latest state of the cluster, providing an accurate basis for monitoring and decision-making.

[0241] The dynamic forest model is used to indicate the node status of MongoDB nodes and includes several functions: status visualization, providing an intuitive representation of the cluster status for easy understanding and monitoring by administrators; load allocation guidance, determining request routing strategies based on the tree structure and prioritizing high-performance upper-layer nodes; fault detection and recovery, identifying anomalies by monitoring changes in the tree structure and initiating recovery processes; and resource planning assistance, analyzing the structural characteristics of the forest model to provide a basis for capacity planning and resource allocation. Maintaining the forest model requires efficient update algorithms to ensure consistency and accuracy in environments with frequent changes in node status.

[0242] Next, when a change in the performance metrics of any MongoDB node is detected, the updated metric data is retrieved, and the dynamic forest model is updated based on this updated data, identifying the corresponding tree structure change event. This process demonstrates the system's real-time response capability, ensuring that the forest model always reflects the latest cluster state. A change in metric data refers to a significant change in the performance metrics, availability status, or configuration information of a MongoDB node, which may be caused by various factors: load changes, such as a sudden surge in request volume leading to increased CPU utilization; resource exhaustion, such as memory usage approaching its limit; hardware failures, such as increased disk error rates; network fluctuations, such as increased latency or packet loss rates; and configuration modifications, such as role changes or parameter adjustments.

[0243] Subsequently, the acquisition of metric update data is typically achieved through a monitoring system. A modern MongoDB monitoring architecture may include multiple layers: built-in monitoring modules on nodes, collecting basic performance metrics and status information; dedicated monitoring agents deployed on each node, collecting more detailed system-level metrics; a central monitoring service that aggregates and analyzes monitoring data from all nodes; and an event handling system that identifies abnormal patterns and triggers appropriate processes. This multi-layered monitoring ensures the comprehensiveness and timeliness of metric data, providing reliable input for updating the dynamic forest model.

[0244] Optionally, the update process of the dynamic forest model employs different strategies for different types of changes: local updates, modifying only the affected nodes and their related connections to minimize computational overhead; incremental updates, making incremental adjustments based on the previous state to avoid complete reconstruction; batch updates, merging multiple updates when multiple metrics change within a short period to reduce frequent adjustments; and priority updates, arranging the update order according to the importance and urgency of the changes to ensure that critical changes are handled first. The update algorithm needs to consider the balance between performance and accuracy: overly frequent updates may lead to excessive computational burden, while delayed updates may result in model inaccuracies.

[0245] Tree structure change events refer to changes in the forest model structure caused by metric updates. These events are crucial for understanding cluster dynamics and driving subsequent actions. Common change events include: node upgrades (nodes move up in the tree due to improved performance); node degradation (nodes move down in the tree due to decreased performance); node removal (nodes are removed from the tree due to failure or maintenance); node additions (tree expansion due to new nodes); tree reorganization (major changes in major nodes or a large number of node states leading to significant structural adjustments); and tree merging or splitting (forest structure adjustments due to changes in cluster boundaries). These events not only reflect changes in cluster state but may also trigger a series of management operations, such as load redistribution, replication configuration adjustments, or alert notifications.

[0246] Specifically, the process of identifying change events involves several steps: change detection, comparing the forest model before and after the update to identify structural differences; event classification, categorizing events based on change type and impact scope; event priority assessment, determining the severity and urgency of events; event correlation analysis, identifying causal relationships between related events to avoid handling redundant events; and event notification, disseminating change events to relevant components and administrators to initiate response processes. An effective event handling system can extract meaningful high-level events from numerous low-level metric changes, helping administrators and automation systems better understand cluster dynamics.

[0247] Finally, a multidimensional index is constructed based on the dynamic forest model, tree structure change events, and operational metrics. A multidimensional index is a data structure specifically optimized for complex data queries, enabling efficient data retrieval across multiple dimensions simultaneously. In a MongoDB cluster management environment, multidimensional indexes are used for rapid access and analysis of large amounts of node metric data, supporting real-time monitoring, trend analysis, and anomaly detection. This type of index allows the system to quickly respond to various complex queries, such as "find all nodes whose CPU utilization exceeds 80% and memory usage increases by more than 20% in the past hour."

[0248] Furthermore, the construction of the multidimensional index is based on the structured node representation provided by the dynamic forest model. Each tree in the forest model represents a cluster group, and the tree structure reflects the hierarchical relationship between nodes. This organization provides a natural hierarchical basis for the index. Tree structure change events provide the trigger points and content for index updates, ensuring that the index reflects the latest cluster state. Performance metrics constitute the specific content of the index, including various performance indicators, configuration parameters, and status information.

[0249] For example, multidimensional index structures include: R-trees and their variants (R* trees, R+ trees, etc.), which excel at handling spatial data and range queries; KD trees, suitable for fast retrieval of multidimensional point data; quadtrees / octrees, adept at handling regularly distributed spatial data; and Bloom filters, used for efficient membership queries, especially suitable for pre-filtering large datasets. In a MongoDB cluster environment, index selection needs to consider data characteristics and query patterns: time-series data (such as continuous performance metrics) may be suitable for composite indexes combining time and space; node attribute data (such as configuration parameters) may be suitable for hash indexes or B-trees; and network topology data may be suitable for graph index structures.

[0250] For example, the construction process of a multidimensional index includes several key steps: data preparation, which transforms raw metric data into a format suitable for the index, possibly including normalization, feature extraction, and dimensionality reduction; index structure selection, which determines the most suitable index type based on data characteristics and query requirements; initial index construction, which creates a basic index using the selected algorithm and structure; incremental update mechanism, which designs an efficient update strategy to ensure that the index can keep up with data changes; index optimization, which periodically reorganizes the index structure to maintain query performance; and distributed scaling, which may require distributing the index across multiple nodes for large-scale clusters to ensure scalability.

[0251] Optionally, multidimensional indexes have a wide range of applications: real-time monitoring queries to quickly locate nodes that meet specific conditions, such as "finding all nodes whose I / O wait times exceed a threshold"; historical trend analysis to retrieve performance changes within a specific time period, such as "analyzing memory usage patterns over the past 24 hours"; similarity search to find other nodes with similar performance characteristics to a given node for load balancing or fault diagnosis; anomaly detection to identify node behavior that deviates from normal patterns and discover potential problems in advance; and correlation analysis to explore the relationships between different metrics, such as the correlation between CPU utilization and query response time. These applications collectively support smarter and more efficient MongoDB cluster management.

[0252] It's important to note that implementing multidimensional indexes requires consideration of performance and resource balance: a trade-off between index size and query speed (more complex index structures may offer faster query speeds but consume more storage space); a balance between update frequency and consistency (frequent updates ensure the index accurately reflects the latest state but may incur high computational overhead); memory and disk usage allocation (hot data can be stored in memory for faster access, while cold data can be stored on disk to save resources); and query type optimization (optimizing the index structure based on common query patterns to improve average query efficiency). These considerations are particularly crucial in large-scale MongoDB clusters, directly impacting system responsiveness and resource efficiency.

[0253] Therefore, this embodiment constructs a complex and powerful MongoDB node management framework. Through node clustering, hierarchical tree structure construction, dynamic forest model organization, real-time metric updates, and multi-dimensional index construction, it can efficiently organize and manage large-scale MongoDB clusters, quickly respond to state changes, and support complex monitoring and analysis needs. Its structured management is particularly suitable for handling large-scale distributed environments with hundreds or even thousands of nodes, providing a solid infrastructure for the entire multi-node dynamic failover system. Compared to traditional flat node management methods, this hierarchical and dynamic approach better reflects the natural grouping and performance differences between nodes, providing finer-grained control and more efficient resource utilization, thereby supporting smarter and more adaptive MongoDB request distribution strategies.

[0254] In yet another embodiment, step S4 further includes:

[0255] Each tree structure in the dynamic forest model is considered as a node cluster, and a cluster graph is constructed to indicate multiple node clusters and the communication links between the node clusters.

[0256] The cluster graph is divided into multiple subsets, each subset indicating the bottleneck link between the node clusters. The bottleneck link is used to characterize the communication link when the link performance data reaches the preset abnormal link performance condition.

[0257] The subset is mapped to a cactus structure, and based on the cactus structure and the subset, the target bottleneck point is identified. The cactus structure is used to describe the bottleneck association between the node clusters.

[0258] Based on the multidimensional index, the target bottleneck point, and the weight value of the MongoDB node, load prediction is performed on the MongoDB node to obtain the node load prediction value.

[0259] Based on the predicted node load of the MongoDB node and the target bottleneck, the system state is simulated, and the weight value of the MongoDB node is adjusted according to the predicted node load and the system state.

[0260] In this embodiment, a cactus structure is used to further enhance bottleneck identification and weight adjustment capabilities. First, each tree structure in the dynamic forest model is treated as a node cluster, and a cluster graph is constructed to represent multiple node clusters and their communication links. Then, the cluster graph is minimally chopped to identify bottleneck links, and subsets are mapped to cactus structures to describe bottleneck associations between node clusters. Based on multidimensional indexes, target bottleneck points, and node weight values, node load can be predicted, and system states can be simulated accordingly, ultimately adjusting node weight values. This method proactively adjusts weights before load changes, achieving predictive load balancing rather than passively responding to congestion that has already occurred, significantly improving system stability under dynamic loads.

[0261] First, each tree structure in the dynamic forest model is treated as a node cluster, and a cluster graph is constructed. A node cluster refers to a logical grouping of MongoDB nodes represented by a single tree structure in the dynamic forest model. In this representation, each tree represents an interconnected group of nodes, typically with similar performance characteristics, configuration parameters, or functional roles. The cluster graph is a high-level graph structure where vertices represent node clusters (rather than individual nodes), and edges represent communication links between clusters. This increased level of abstraction allows for a more macroscopic analysis of the topology and performance characteristics of MongoDB clusters, more effectively identifying key bottlenecks in large-scale systems.

[0262] The construction of a cluster graph involves multiple steps: tree structure identification, extracting individual tree structures from a dynamic forest model, with each tree structure corresponding to a node cluster; cluster feature extraction, calculating the aggregate features of each node cluster, such as total processing capacity, average load, and main functions; communication link identification, analyzing data flow and request interactions between different cluster nodes to determine the connection relationships between clusters; link feature calculation, evaluating the performance parameters of inter-cluster communication links, such as bandwidth, latency, and reliability; and graphical representation, converting cluster and link data into a graph structure for easier subsequent analysis. The advantage of cluster graphs lies in their significant simplification of system complexity: abstracting a system that may consist of hundreds of nodes into several or dozens of clusters, making global analysis more feasible.

[0263] Inter-cluster communication links refer to network connections used for data exchange and coordination between different node clusters. These links may correspond to physical network connections (such as dedicated network interfaces or cross-datacenter links) or logical communication channels (such as specific protocol streams or application layer connections). In a MongoDB environment, typical communication scenarios include: data replication streams within replica sets, such as the transmission of operation logs from master nodes to slave nodes; data migration between shards, such as block movement during data rebalancing; configuration synchronization streams, such as metadata exchange between cluster members and the configuration server; and query routing communication, such as request-response interactions between application servers or routing servers and data nodes. The performance characteristics of these communication links directly affect the overall operating efficiency of the cluster and are important targets for bottleneck analysis.

[0264] Secondly, a minimum cut is performed on the cluster graph to generate multiple subsets. Minimum cut is a classic concept in graph theory, referring to removing the minimum number (or minimum weight) of edges to split the graph into disconnected parts. Performing a minimum cut on a cluster graph means finding the critical communication links that, if interrupted, would cause the system to split. These links are often potential bottlenecks in the system, and their performance limitations can affect the overall system efficiency. A subset is the disconnected graph portion formed after the minimum cut; each subset contains a group of interconnected nodes, but the connections between it and other subsets are severed.

[0265] For example, there are various implementations of minimum cut algorithms, including: the Ford-Fulkerson algorithm, based on the maximum flow minimum cut theorem, which determines the minimum cut by calculating the maximum flow of the network; the Stoer-Wagner algorithm, a more efficient global minimum cut algorithm; the Karger algorithm, a randomized minimum cut approximation algorithm suitable for large-scale graphs; and spectral clustering methods, which use the eigenvalue decomposition of the graph's Laplacian matrix for cutting. In a MongoDB cluster environment, the choice of minimum cut algorithm needs to consider factors such as the graph's size, density, and weight distribution. For smaller cluster graphs, exact algorithms can be used; while for large and complex cluster topologies, approximate algorithms or heuristic methods may be necessary to ensure computational efficiency.

[0266] In this embodiment, a bottleneck link refers to a communication link whose performance data reaches a preset abnormal link performance condition. These links are usually "weak links" in the system, and their performance limitations may become the bottleneck of the entire system. Link performance data includes various indicators: bandwidth utilization, the ratio of data transmission volume to the maximum capacity of the link; latency level, the transmission time of data packets from source to destination; packet loss rate, the proportion of data packets lost during transmission; jitter, the degree of latency variation; and error rate, the frequency of transmission errors. Abnormal link performance conditions refer to preset thresholds or patterns used to determine when link performance is considered abnormal or a bottleneck. These conditions may be static fixed thresholds (such as bandwidth utilization exceeding 90%) or dynamic benchmarks based on historical data (such as latency exceeding twice the recent average).

[0267] In a MongoDB cluster, different types of bottlenecks can lead to various performance issues: replication latency bottlenecks, where limited replication links between master and slave nodes increase data synchronization delays; inter-shard communication bottlenecks, affecting data migration and distributed query performance; configuration synchronization bottlenecks, impacting the propagation speed of cluster configuration updates; and client connection bottlenecks, limiting the system's request processing capacity. By identifying these bottlenecks through minimal sharding analysis, system administrators can perform targeted network optimizations or resource adjustments to improve the overall cluster performance and reliability.

[0268] Next, the subsets generated by the minimum cut are mapped to a cactus structure, and the target bottleneck is identified based on this structure and subsets. A cactus structure is a special graph structure in graph theory, characterized by any edge appearing in at most one cycle. In other words, a cactus graph is an undirected connected graph where any two cycles share at most one vertex. In network analysis, the cactus structure provides a simplified way to represent key connection patterns in complex networks, and is particularly suitable for describing systems with multiple potential bottlenecks.

[0269] The subsequent process of mapping subsets to cactus structures involves graph transformation and simplification: loop identification, recognizing all loop structures within each subset; shared vertex analysis, identifying shared vertices between different loops; structural reorganization, reorganizing the original subset graph into a structure conforming to the cactus definition; and compressed representation, combining non-critical links to highlight potential bottlenecks. This mapping is not a simple one-to-one transformation, but rather a process of structural simplification and feature extraction, aiming to retain key features in the original network topology relevant to bottleneck analysis while reducing complexity for easier subsequent analysis.

[0270] The cactus structure offers unique advantages in describing bottleneck relationships between node clusters: It provides a clear structure, with each edge belonging to at most one cycle, making bottlenecks easier to identify; it has a distinct hierarchy, visually representing the bottleneck's level and scope of influence; it simplifies analysis, reducing computational complexity by simplifying complex networks into more easily analyzable structures; and it preserves relationships, retaining crucial bottleneck relationship information despite the simplified structure. In bottleneck analysis of MongoDB clusters, the cactus structure can help identify critical connection points where performance degradation would affect multiple clusters, or fragile links that could lead to system partitioning.

[0271] Target bottlenecks are the most critical performance limiting points identified in the cactus structure and original subset analysis. These points are typically located at the intersection of multiple rings or bridging locations connecting different substructures. Once performance degrades or fails, it can have a widespread impact on the entire system. Identification of target bottlenecks is based on a comprehensive analysis of multiple factors: topological importance (the node's location and connectivity characteristics within the network structure); traffic load (the amount of data traffic and requests passing through the point); performance volatility (the instability of historical performance data); redundancy (the availability of alternative paths or backup mechanisms); and business criticality (the degree of support for core business functions).

[0272] The process of identifying target bottlenecks typically involves several stages: initial screening, selecting candidate points based on topology characteristics; load analysis, evaluating the historical and current load levels of each candidate point; impact simulation, simulating the scope and extent of performance degradation or failure for each candidate point; comprehensive scoring, combining multiple factors to score and rank candidate points; and threshold screening, determining the final target bottleneck based on preset importance thresholds. The identified target bottlenecks become the focus of system performance optimization and resource allocation, and changes in their performance directly trigger corresponding adjustment measures.

[0273] Then, based on the multidimensional index, the target bottleneck point, and the weight values ​​of the MongoDB nodes, load prediction is performed on the MongoDB nodes to obtain the node load prediction values. Load prediction refers to estimating the workload that a node may face in the future time period by analyzing historical data, current status, and future trends. This predictive capability allows the system to respond to possible load changes in advance, proactively adjust resource allocation, and avoid performance problems. The node load prediction value is a quantitative indicator representing the expected level of resource usage, typically including multiple dimensions such as CPU utilization, memory consumption, disk I / O, and network traffic.

[0274] The core of load forecasting lies in its data sources and analytical methods. Multidimensional indexes provide efficient access to historical performance data, enabling the system to quickly retrieve and analyze large amounts of historical load patterns; target bottleneck analysis provides structural insights, helping to predict the propagation path and scope of load changes; node weight values ​​reflect the processing capacity of nodes and current allocation strategies, serving as an important reference for predicting future load distribution. These data collectively support more accurate and comprehensive load forecasting.

[0275] For example, commonly used load forecasting methods include various techniques: time series analysis, such as the ARIMA (Autoregressive Integrated Moving Average) model, suitable for loads with obvious time patterns; machine learning methods, such as regression analysis, neural networks, or random forests, capable of capturing complex nonlinear relationships; pattern recognition, identifying recurring load patterns, such as daily peaks or periodic fluctuations; seasonality analysis, considering the impact of business cycles, seasonal variations, or special events; and causal models, analyzing the impact of user behavior, application updates, or business changes on load. In practical applications, these methods may be combined, or appropriate methods may be selected based on different time scales and forecasting objectives.

[0276] Optional, specific considerations for load forecasting include: the forecast timeframe (short-term forecasts (minutes to hours) are generally more accurate, while long-term forecasts (days to weeks) require consideration of more variables); handling of unexpected events (how to forecast and respond to unconventional load surges); data quality issues (how to handle incomplete or noisy historical data); the impact of system changes, such as the impact of architecture adjustments and version upgrades on the forecasting model; and the integration of external factors (how to incorporate business events, marketing campaigns, or market changes into the forecasting model). Proper handling of these factors is crucial for improving forecast accuracy.

[0277] Therefore, the results of load forecasting, namely node load forecasts, provide the expected load level for each MongoDB node over a future period. These forecasts typically include: baseline load, the expected load level under normal conditions; peak estimate, the expected maximum load level and duration; trend, the load's growth or decline trend; fluctuation range, the expected magnitude of load fluctuations; and anomaly probability, the likelihood of the load significantly deviating from expectations. These forecasts provide the data foundation for system resource planning and optimization, enabling management measures to shift from reactive response to proactive prevention.

[0278] Subsequently, based on the predicted node load of MongoDB nodes and the target bottleneck, the system state is simulated, and the weight values ​​of the MongoDB nodes are adjusted according to the predicted node load and the system state. System state simulation is a technique that uses computer models to simulate the future behavior of a system, predicting the system's performance under different conditions, evaluating the effectiveness of potential optimization measures, and identifying possible problem scenarios. In this process, the predicted load and bottleneck analysis results are used as input to generate predictions of the future system operating conditions.

[0279] In this embodiment, system state simulation typically employs methods such as discrete event simulation or system dynamics models, selecting appropriate techniques based on system complexity and simulation objectives. The simulation process considers multiple factors: node processing capacity (the maximum number of requests and data each node can handle); network topology (the connection structure and communication characteristics between nodes); load distribution (the distribution pattern of requests and data within the system); bottleneck impact (the limiting effect of identified bottlenecks on system performance); and fault scenarios (the impact of potential node failures or network outages). Simulation results typically include various system performance metrics: response time (the time from request initiation to completion); throughput (the number of requests or data the system can process per unit time); resource utilization (the utilization level of CPU, memory, disk, and network resources for each node); queue length (the number of requests waiting to be processed); and stability metrics, such as the system's ability to maintain normal operation and its adaptability to load surges.

[0280] Based on simulation results and load prediction, the weight values ​​of MongoDB nodes are dynamically adjusted. Weight adjustment is the core mechanism of a load balancing system. By changing the weight values ​​of nodes, it influences request distribution decisions, optimizing resource utilization and performance. Adjustment strategies are typically based on several considerations: load balancing, ensuring even distribution of requests and avoiding node overload or idleness; bottleneck mitigation, reducing the load on target bottleneck points and alleviating system bottlenecks; performance optimization, improving overall system response time and throughput; stability assurance, enhancing the ability to cope with sudden loads or partial failures; and resource efficiency, maximizing the utilization of hardware resources and reducing operating costs.

[0281] Specific methods for weight adjustment may include various algorithms: linear adjustment, which adjusts weights proportionally based on the difference between the predicted load and the target load; threshold-triggered adjustment, which adjusts weights when the predicted load exceeds a specific threshold; gradual adjustment, which gradually reaches the target through multiple small adjustments to avoid system oscillations; predictive adjustment, which adjusts weights in advance based on predicted future load trends to prevent problems before they occur; and differentiated adjustment, which adopts different adjustment strategies based on node roles, performance characteristics, and business importance. Regardless of the method used, the adjustment process is usually continuous and gradual to avoid system instability caused by drastic changes.

[0282] The execution of weight adjustments needs to consider several practical factors: adjustment frequency (too frequent adjustments may cause system fluctuations, while too delayed adjustments may not respond to changes in a timely manner); adjustment magnitude (large adjustments may cause sudden load shifts, while small adjustments may have little effect); adjustment verification (the effect of each adjustment needs to be verified to ensure that the expected goals are achieved); adjustment constraints (some nodes may have minimum or maximum weight limits, reflecting their role or capability limitations); and adjustment coordination (in a multi-layer load balancing system, weight adjustments at different levels need to be coordinated to ensure overall consistency).

[0283] The result of weight adjustments is the new weight values ​​for MongoDB nodes, which directly impact subsequent request distribution decisions. After adjustment, the system continuously monitors the difference between actual load and forecasts, evaluates the adjustment's effectiveness, and performs further optimization as necessary. Through this closed-loop forecast-simulation-adjustment-verification process, the system can continuously adapt to changing load patterns and performance requirements, maintaining optimal operating conditions.

[0284] Therefore, this embodiment constructs a highly intelligent MongoDB cluster performance optimization system. Through cluster graph construction, minimum partitioning analysis, cactus structure mapping, target bottleneck identification, load prediction, system state simulation, and dynamic weight adjustment, the system can accurately identify performance bottlenecks, predict future load trends, and proactively optimize resource allocation. This graph theory-based and predictive analytics approach is more advanced and efficient than traditional reactive load balancing strategies, providing more stable and balanced performance in complex and dynamic distributed environments. Especially for large-scale MongoDB clusters, this predictive and adaptive optimization method can significantly improve system reliability, performance, and resource utilization efficiency, while reducing operational complexity and costs.

[0285] like Figure 2 As shown in the figure, this application provides a MongoDB multi-node dynamic switching system, including:

[0286] The data access layer creation module 201 is used to obtain the configuration information of multiple MongoDB nodes and establish a data access layer based on the configuration information. The data access layer is configured with a connection pool management mechanism for indicating the management of the connection pool.

[0287] The metrics acquisition module 202 is used to acquire metrics data of the MongoDB node about multiple operating metrics through the data access layer according to the hierarchical frequency acquisition task configured for each MongoDB node. The acquisition task is used to indicate the multiple acquisition frequencies executed by the MongoDB node and the operating metrics to be acquired at each acquisition frequency. The operating metrics include CPU utilization, memory utilization, IOPS, disk wait time and network latency.

[0288] The data aggregation module 203 is used to aggregate the indicator data of the MongoDB nodes to obtain the node performance indicators of each MongoDB node.

[0289] The weight calculation module 204 is used to combine the obtained preset value of the node weight of the MongoDB node to convert the node performance index of the MongoDB node into the weight value of the MongoDB node.

[0290] The routing and distribution module 205 is used to determine a request distribution strategy based on the connection pool management mechanism and the obtained node availability status data of the MongoDB node, according to the weight value of the MongoDB node. The request distribution strategy is used to indicate the distribution ratio of MongoDB requests allocated to the MongoDB node.

[0291] The health management module 206 is used to perform status detection on the MongoDB node through the data access layer. When the MongoDB node is detected to meet the preset abnormal status conditions, the detected MongoDB node is marked as unavailable and the node availability status data is updated.

[0292] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for dynamic multi-node switching in MongoDB, characterized in that, include: Obtain configuration information for multiple MongoDB nodes, establish a data access layer based on the configuration information, and configure a connection pool management mechanism within the data access layer to indicate the management connection pool; Through the data access layer, the MongoDB node acquires metric data about multiple operating metrics by collecting tasks at the hierarchical frequency configured for each MongoDB node. The collection tasks are used to indicate the multiple collection frequencies executed by the MongoDB node and the operating metrics to be collected at each collection frequency. The operating metrics include CPU utilization, memory utilization, IOPS, disk wait time, and network latency. The metric data of the MongoDB nodes are aggregated to obtain the node performance metrics for each MongoDB node. Based on the obtained preset values ​​of the node weights of the MongoDB nodes, the node performance metrics of the MongoDB nodes are converted into the weight values ​​of the MongoDB nodes. Based on the connection pool management mechanism and the obtained node availability status data of the MongoDB node, a request distribution strategy is determined according to the weight value of the MongoDB node. The request distribution strategy is used to indicate the distribution ratio of MongoDB requests allocated to the MongoDB node. The data access layer performs status detection on the MongoDB node. When the MongoDB node is detected to meet the preset abnormal status conditions, the detected MongoDB node is marked as unavailable and the node availability status data is updated. The step of determining the request distribution strategy based on the connection pool management mechanism and the obtained node availability status data of the MongoDB nodes, according to the weight value of the MongoDB nodes, includes: Based on the weight values, performance metrics, availability status data, and connection relationships of the MongoDB nodes, a MongoDB connection network graph is constructed. The connection relationships indicate the data dependencies between the MongoDB nodes. The vertices in the MongoDB connection network graph represent the MongoDB nodes, and the vertex weights of the vertices are determined by the performance metrics. The edges in the MongoDB connection network graph represent the connection relationships between the MongoDB nodes. Based on the MongoDB connection network graph, an NMC sparser model is constructed, and the minimum cut set in the MongoDB connection network graph is identified according to the NMC sparser model. The minimum cut set is used to indicate MongoDB nodes whose node load exceeds a preset load threshold and / or MongoDB nodes whose network connection stability is lower than a preset stability threshold. Based on the MongoDB connection network graph and the minimum cut set, determine the target node path from the specified source node to the MongoDB node; Based on the minimum cut set and the weight values ​​of the MongoDB nodes, the performance of the request distribution strategy is predicted to obtain the performance prediction results, which include system bottleneck prediction values ​​and performance loss prediction values. Based on the weight values ​​of the MongoDB nodes and the path of the target node, the top n MongoDB nodes are selected as candidate nodes. The bottleneck risk of the candidate node is evaluated using the NMC sparser model, and the performance evaluation value of the candidate node is determined based on the bottleneck risk of the candidate node, the weight value, and the performance prediction result. Candidate nodes whose performance evaluation values ​​are greater than a preset processing capacity threshold are selected as target MongoDB nodes.

2. The method according to claim 1, characterized in that, The step of obtaining configuration information for multiple MongoDB nodes and establishing a data access layer based on the configuration information includes: Based on the interface specifications of the native driver of the MongoDB node and the business requirements of multiple MongoDB nodes, the overall architecture of the data access layer is determined. Based on the overall architecture of the data access layer and the connection string, maximum number of connections, connection timeout time and idle connection hold time in the configuration information of the MongoDB node, a corresponding connection pool is created for each MongoDB node, and a connection pool management mechanism is configured. A data access layer is established based on the connection pool management mechanism and the overall architecture of the data access layer.

3. The method according to claim 2, characterized in that, The hierarchical frequency acquisition task includes high-frequency indicator acquisition tasks and low-frequency indicator acquisition tasks; the acquisition of indicator data for multiple operational indicators of the MongoDB node through the data access layer according to the hierarchical frequency acquisition task configured for each MongoDB node includes: Based on the aforementioned operational metrics, high-frequency metric collection tasks and low-frequency metric collection tasks are determined to obtain tiered frequency collection tasks. A corresponding tiered frequency collection task is configured for each MongoDB node. The high-frequency metric collection task is used to indicate a first collection frequency for high-frequency metrics, and the low-frequency metric collection task is used to indicate a second collection frequency for low-frequency metrics. The first collection frequency is greater than the second collection frequency. The high-frequency metrics include the CPU utilization and the memory utilization, and the low-frequency metrics include the IOPS, the disk wait time, and the network latency. Based on the high-frequency indicator collection task and the connection pool management mechanism, a high-frequency collection thread is established for each MongoDB node, and the high-frequency indicator collection task is executed according to the high-frequency collection thread to obtain the high-frequency indicator data. Based on the low-frequency indicator collection task and the connection pool management mechanism, a low-frequency collection thread is established for each MongoDB node, and the low-frequency indicator collection task is executed according to the low-frequency collection thread to obtain the low-frequency indicator data. Configure a timeout mechanism and a retry strategy for the high-frequency index data and the low-frequency index data. The timeout mechanism is used to indicate that the current hierarchical frequency acquisition task is marked as acquisition failure when the data acquisition time exceeds a preset acquisition time threshold. The retry strategy is used to indicate that the corresponding hierarchical frequency acquisition task is re-executed when acquisition failure is detected.

4. The method according to claim 3, characterized in that, The aggregation of the metric data of the MongoDB nodes to obtain the node performance metrics for each MongoDB node includes: Based on the high-frequency indicator collection task, a high-frequency indicator window is configured for the high-frequency indicator, and the average value of the high-frequency indicator data collected within the high-frequency indicator window is calculated to obtain the average value of the high-frequency indicator. Based on the low-frequency index collection task, a low-frequency index window is configured for the low-frequency index, and the average value of the low-frequency index data collected within the low-frequency index window is calculated to obtain the average value of the low-frequency index. When missing data is detected in the high-frequency indicator window and / or the low-frequency indicator window, the missing data is filled with adjacent data or the missing data is filled by interpolation. Window data aggregation is performed on the average of the high-frequency indicators, the average of the low-frequency indicators, and the filled data to obtain the node performance indicators.

5. The method according to claim 4, characterized in that, The step of combining the obtained preset node weight values ​​of the MongoDB nodes to convert the node performance metrics of the MongoDB nodes into weight values ​​includes: When the CPU utilization index is lower than a preset first CPU utilization threshold, the CPU penalty factor is determined as the first CPU penalty factor. Alternatively, when the CPU utilization index data exceeds the first CPU utilization threshold but is lower than the preset second CPU utilization threshold, the CPU penalty factor is determined as the second CPU penalty factor, wherein the first CPU utilization threshold is less than the second CPU utilization threshold, and the second CPU penalty factor is greater than the first CPU penalty factor. Alternatively, when the CPU utilization index data exceeds the second CPU utilization threshold, the CPU penalty factor is determined as the third CPU penalty factor, wherein the third CPU penalty factor is greater than the second CPU penalty factor. Based on the memory usage rate index data and the preset memory usage rate threshold, a memory usage rate penalty factor is determined; Based on the IOPS metric data and the preset IOPS threshold, an IOPS penalty factor is determined; Based on the disk wait time metric data and the preset disk wait time threshold, a disk wait time penalty factor is determined; Based on the network latency index data and the preset network latency threshold, a network latency penalty factor is determined; The weight value of the MongoDB node is calculated based on the preset node weight value of the MongoDB node, the determined CPU penalty factor, the memory utilization penalty factor, the IOPS penalty factor, the disk wait time penalty factor, and the network latency penalty factor.

6. The method according to claim 1, characterized in that, The aggregation of the metric data of the MongoDB nodes to obtain the node performance metrics for each MongoDB node also includes: Based on the performance characteristics of the MongoDB nodes, the node connection relationships, the configuration information, the node performance indicators, and the node availability status data, the MongoDB nodes are clustered. MongoDB nodes belonging to the same group are hierarchically structured according to the node performance metrics, a tree structure is created based on the hierarchical relationship, and multiple tree structures are organized into a dynamic forest model. The dynamic forest model is used to indicate the node status that manages the MongoDB nodes. When a change in the metric data of any MongoDB node's running metrics is detected, the updated metric data is obtained, the dynamic forest model is updated based on the updated metric data, and the corresponding tree structure change event is determined. Based on the dynamic forest model, the tree structure change events, and the operational metrics, a multidimensional index is constructed, which is used to indicate the indexing and retrieval of the operational metrics.

7. The method according to claim 6, characterized in that, The step of combining the obtained preset node weight values ​​of the MongoDB nodes to convert the node performance metrics of the MongoDB nodes into weight values ​​also includes: Each tree structure in the dynamic forest model is considered as a node cluster, and a cluster graph is constructed to indicate multiple node clusters and the communication links between the node clusters. The cluster graph is divided into multiple subsets, each subset indicating the bottleneck link between the node clusters. The bottleneck link is used to characterize the communication link when the link performance data reaches the preset abnormal link performance condition. The subset is mapped to a cactus structure, and based on the cactus structure and the subset, the target bottleneck point is identified. The cactus structure is used to describe the bottleneck association between the node clusters. Based on the multidimensional index, the target bottleneck point, and the weight value of the MongoDB node, load prediction is performed on the MongoDB node to obtain the node load prediction value. Based on the predicted node load of the MongoDB node and the target bottleneck, the system state is simulated, and the weight value of the MongoDB node is adjusted according to the predicted node load and the system state.

8. A MongoDB multi-node dynamic switching system, characterized in that, include: The data access layer creation module is used to obtain the configuration information of multiple MongoDB nodes and establish a data access layer based on the configuration information. The data access layer is configured with a connection pool management mechanism for instructing the management of the connection pool. The metrics acquisition module is used to acquire metrics data of the MongoDB node about multiple operational metrics through the data access layer according to the hierarchical frequency acquisition task configured for each MongoDB node. The acquisition task is used to indicate the multiple acquisition frequencies executed by the MongoDB node and the operational metrics to be acquired at each acquisition frequency. The operational metrics include CPU utilization, memory utilization, IOPS, disk wait time and network latency. The data aggregation module is used to aggregate the indicator data of the MongoDB nodes to obtain the node performance indicators of each MongoDB node. The weight calculation module is used to combine the obtained preset values ​​of the node weights of the MongoDB nodes to convert the node performance indicators of the MongoDB nodes into the weight values ​​of the MongoDB nodes. A routing and distribution module is used to determine a request distribution strategy based on the connection pool management mechanism and the obtained node availability status data of the MongoDB nodes, according to the weight value of the MongoDB nodes. The request distribution strategy indicates the proportion of MongoDB requests allocated to the MongoDB nodes. The determination of the request distribution strategy based on the connection pool management mechanism and the obtained node availability status data of the MongoDB nodes, according to the weight value of the MongoDB nodes, includes: Based on the weight values, performance metrics, availability status data, and connection relationships of the MongoDB nodes, a MongoDB connection network graph is constructed. The connection relationships indicate the data dependencies between the MongoDB nodes. The vertices in the MongoDB connection network graph represent the MongoDB nodes, and the vertex weights of the vertices are determined by the performance metrics. The edges in the MongoDB connection network graph represent the connection relationships between the MongoDB nodes. Based on the MongoDB connection network graph, an NMC sparser model is constructed, and the minimum cut set in the MongoDB connection network graph is identified according to the NMC sparser model. The minimum cut set is used to indicate MongoDB nodes whose node load exceeds a preset load threshold and / or MongoDB nodes whose network connection stability is lower than a preset stability threshold. Based on the MongoDB connection network graph and the minimum cut set, determine the target node path from the specified source node to the MongoDB node; Based on the minimum cut set and the weight values ​​of the MongoDB nodes, the performance of the request distribution strategy is predicted to obtain the performance prediction results, which include system bottleneck prediction values ​​and performance loss prediction values. Based on the weight values ​​of the MongoDB nodes and the path of the target node, the top n MongoDB nodes are selected as candidate nodes. The bottleneck risk of the candidate node is evaluated using the NMC sparser model, and the performance evaluation value of the candidate node is determined based on the bottleneck risk of the candidate node, the weight value, and the performance prediction result. Candidate nodes whose performance evaluation values ​​are greater than a preset processing capacity threshold are selected as target MongoDB nodes; The health management module is used to perform status detection on the MongoDB node through the data access layer. When the MongoDB node is detected to meet the preset abnormal status conditions, the detected MongoDB node is marked as unavailable and the node availability status data is updated.

Images