A dynamic load balancing method and system based on Redis and memory collaboration

Abstract

This invention discloses a dynamic load balancing method and system based on Redis and memory collaboration, belonging to the field of intelligent resource scheduling technology. It includes obtaining complete profile data and generating a hot group list and link matrix through unified collection of multi-level indicators and synchronous monitoring of node links; obtaining a link migration execution report by detecting the link health of migration commands and node parameters; verifying and replaying the migration execution report using contradictory verification correction to generate an adoption report and an anomaly list; and dynamically adjusting the order of different levels and nodes and updating the migration execution report based on the adoption report and the anomaly list using an adaptive parameter tuning method. This invention achieves hot / cold zone perception and congestion prediction of migration commands by combining a cross-layer remapping candidate list with a link matrix for bandwidth hot zone projection and cold zone priority sorting, effectively improving the accuracy of resource scheduling under Redis and memory collaboration.

Description

Technical Field

[0001] This invention relates to the field of intelligent resource scheduling technology, and in particular to a dynamic load balancing method and system based on Redis and memory collaboration. Background Technology

[0002] In distributed caching and high-concurrency service scenarios, load balancing technology is widely used to improve system throughput and resource utilization. Existing methods usually rely on static weight allocation and threshold triggering mechanisms based on single-dimensional indicators, combined with a centralized scheduler to distribute traffic to Redis nodes. To a certain extent, it can alleviate local hotspot pressure, is suitable for deployment environments with relatively stable structures, and has been verified in practice in multiple large-scale online systems.

[0003] However, conventional methods lack fine-grained perception of cross-layer resource coordination when facing dynamically changing node loads and link states in multi-layer architectures, making it difficult to accurately identify hot zones and efficiently activate cold zones. At the same time, parameter adjustments are mostly performed offline or periodically, and cannot be optimized in a closed loop based on real-time feedback of migration effects, resulting in delayed or fluctuating load redistribution, affecting overall stability and response efficiency. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a dynamic load balancing method based on Redis and memory collaboration to solve the problem of cross-layer collaborative optimization.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: Firstly, this invention provides a dynamic load balancing method based on Redis and memory collaboration, comprising: obtaining complete profile data and generating a hot group list and link matrix through unified collection of multi-level indicators and synchronous monitoring of node links; performing data analysis on the target node set of each hot group at different levels based on the hot group list and link matrix to obtain a cross-level remapping candidate list; based on the cross-level remapping candidate list, using the link matrix to perform bandwidth hot zone projection, congestion determination, and cold zone priority sorting to obtain migration instructions and node parameters for link hot and cold distribution; obtaining a link migration execution report by performing link health detection on the migration instructions and node parameters; verifying and replaying the migration execution report using proof by contradiction to generate an adoption report and an anomaly list; and dynamically adjusting the order of different levels and nodes and updating the migration execution report based on the adoption report and the anomaly list using an adaptive parameter tuning method.

[0007] As a preferred embodiment of the dynamic load balancing method based on Redis and memory collaboration described in this invention, the steps for obtaining complete profile data and generating a hot group list and link matrix through unified collection of multi-level metrics and synchronous monitoring of node links are as follows: Based on multi-source indicator collection, performance and status data at all levels are uniformly aggregated to obtain a multi-level indicator set covering the entire link. Based on the multi-level indicator set and the real-time status data of the links between nodes, the nodes and connection relationships are dynamically tracked through synchronous monitoring to generate complete profile data; By utilizing complete profile data, clustering and analysis are used to identify highly active node groups and interaction links, resulting in a list of hot groups and a link matrix.

[0008] As a preferred embodiment of the dynamic load balancing method based on Redis and memory collaboration described in this invention, the dynamic tracking refers to continuously collecting multi-level indicator data and synchronously monitoring the status of highly active nodes and links during operation.

[0009] As a preferred embodiment of the dynamic load balancing method based on Redis and memory collaboration described in this invention, the step of analyzing the target node sets of each hot group at different levels according to the hot group list and the link matrix to obtain a cross-level remapping candidate list involves the following specific steps: Based on the hot group list and link matrix, the analytic hierarchy process is used to perform weight partitioning and hierarchical data analysis on the target node set of each hot group at different levels, and to obtain the cross-level node importance ranking of each hot group. Based on the importance ranking of nodes across layers, the path consistency verification and mapping feasibility screening of the ranking results are performed using the connection relationships in the link matrix, generating a candidate list for cross-layer remapping.

[0010] As a preferred embodiment of the dynamic load balancing method based on Redis and memory collaboration described in this invention, the specific steps are as follows: based on the cross-layer remapping candidate list, bandwidth hot zone projection and congestion determination are performed using the link matrix, and cold zone priority is ranked to obtain the migration instructions and node parameters for the link hot and cold distribution. Based on the cross-layer remapping candidate list, the load thermal distribution of each link is obtained by hot zone projection of bandwidth usage using the link matrix. Based on the load thermal distribution, the congestion status in the link matrix is ​​determined, and the links in the cold zone are prioritized to obtain the migration instructions and node parameters for the hot and cold distribution of the links.

[0011] As a preferred embodiment of the dynamic load balancing method based on Redis and memory collaboration described in this invention, the step of obtaining a migration execution report by performing link health checks on migration commands and node parameters is as follows: By probing the link connectivity and load status of migration commands and node parameters, basic link health data can be obtained. Based on the basic health data of the link, a time-series consistency verification method is used to compare and analyze the link status fluctuations and abnormal responses to obtain the link migration execution report.

[0012] As a preferred embodiment of the dynamic load balancing method based on Redis and memory collaboration described in this invention, the specific steps for verifying and replaying the migration execution report using counter-evidence correction to generate an adoption report and an anomaly list are as follows. Based on the migration execution report, the current link behavior is compared with the expected behavior to obtain a set of deviation features; Based on the set of deviation features, reverse verification is used to comprehensively judge the adoption feasibility scale and the distribution of multi-level anomalies, and an adoption report and an anomaly list are generated.

[0013] As a preferred embodiment of the dynamic load balancing method based on Redis and memory collaboration described in this invention, the adaptive parameter tuning method refers to dynamically adjusting the parameters and node order at each level based on feedback information generated during operation, and continuously updating the execution strategy.

[0014] As a preferred embodiment of the dynamic load balancing method based on Redis and memory collaboration described in this invention, the specific steps of dynamically adjusting the order of different levels and nodes and updating the migration execution report using an adaptive parameter tuning method based on the adoption report and the exception list are as follows. Based on the adoption report and the anomaly list, consistency checks are used to dynamically adjust the node order at different levels to obtain the updated hierarchical node configuration. Based on the updated hierarchical node configuration, the migration execution report is synchronously corrected using an adaptive parameter tuning method to generate an optimized migration execution report.

[0015] Secondly, this invention provides a dynamic load balancing system based on Redis and memory collaboration, comprising: a data acquisition and detection module, which obtains complete profile data and generates a hot group list and a link matrix through unified collection of multi-level indicators and synchronous monitoring of node links; an analysis and mapping module, which performs data analysis on the target node set of each hot group at different levels according to the hot group list and the link matrix to obtain a cross-level remapping candidate list; a projection and sorting module, which, based on the cross-level remapping candidate list, uses the link matrix to perform bandwidth hot zone projection, congestion determination, and cold zone priority sorting to obtain migration instructions and node parameters for link hot and cold distribution; a detection and execution module, which obtains a link migration execution report by detecting the link health of the migration instructions and node parameters; a verification and comparison module, which verifies and replays the migration execution report using proof by contradiction correction to generate an adoption report and an anomaly list; and an adjustment and update module, which dynamically adjusts the order of different levels and nodes and updates the migration execution report according to the adoption report and the anomaly list using an adaptive parameter tuning method.

[0016] The beneficial effects of this invention are as follows: by uniformly collecting multi-level indicators and synchronously monitoring node links, real-time dynamic tracking of the entire link status is realized, and highly active hot groups and link dependencies are accurately identified, providing a complete data foundation for subsequent load scheduling; by combining the cross-layer remapping candidate list with the link matrix to project bandwidth hot zones and prioritize cold zones, the cold and hot perception and congestion prediction of migration instructions are realized, effectively improving the accuracy of resource scheduling under the collaboration of Redis and memory. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a dynamic load balancing method based on Redis and memory collaboration.

[0019] Figure 2 This is a schematic diagram of a dynamic load balancing system based on Redis and memory collaboration.

[0020] Figure 3 This is a flowchart of the cross-layer remapping candidate list.

[0021] Figure 4 This is a flowchart for link health detection and migration execution report. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides a dynamic load balancing method based on Redis and memory collaboration, comprising the following steps: S1: By collecting multi-level indicators in a unified manner and monitoring node links synchronously, complete profile data is obtained and a hot group list and link matrix are generated.

[0026] S1.1: Based on multi-source indicator collection, the performance and status data of each level are uniformly aggregated to obtain a multi-level indicator set covering the entire link.

[0027] Furthermore, based on multi-dimensional metrics including access frequency, response latency, memory usage, connection count, and bandwidth utilization synchronously acquired from Redis instances, memory buffers, and inter-node communication links, the collected metrics are time-aligned and structure-normalized according to their hierarchical affiliation. The processed metric data is then aggregated and classified into control plane, data plane, and storage plane through unified aggregation, forming a multi-layered metric set covering the entire link.

[0028] It should be noted that time alignment and structure normalization processing refers to the following: When collecting multi-source data, all multi-source data carries a timestamp with a unified time base. Before aggregation, the acquisition and detection module adds two fields, acquisition time and event time, to the acquisition records and retains a sequence identifier in the streaming channel for playback alignment. This unifies the timestamp alignment of multi-source data at different time points and within different time windows, ensuring correct matching and comparison of multi-source data in the time dimension. Simultaneously, multi-source data with different formats and structures are standardized and converted into a unified structural format, ensuring effective integration and analysis of multi-source data on the same platform. Hierarchical attribution refers to classifying different nodes and connections into corresponding layers based on their hierarchical relationship within the network. During multi-source data analysis and load balancing, performance data, load conditions, and network status are classified and processed hierarchically based on node hierarchical attribution. Cross-layer remapping and bandwidth scheduling are performed to optimize overall load balancing and resource allocation efficiency.

[0029] S1.2: Based on the multi-level indicator set and the real-time status data of the links between nodes, the nodes and connection relationships are dynamically tracked through synchronous monitoring to generate complete profile data.

[0030] Furthermore, based on the multi-level indicator set and the real-time status data of the links between nodes, the multi-level indicator set and the real-time status data of the links between nodes enter the time alignment and structure normalization joint analysis process under a unified time benchmark. Window alignment and sequence identification are used to ensure consistent comparison of records across sources. During the operation, the multi-level indicator set covering the entire link is continuously collected to obtain the real-time status data of the links between nodes. The multi-level indicator set and the real-time status data are subjected to time alignment and structure normalization joint analysis. Dynamic tracking is used to continuously record and correlate node activity, link load changes and interaction frequency to generate complete profile data reflecting the current operating status.

[0031] It should be noted that association mapping refers to identifying the interaction between performance, bandwidth, and consistency by analyzing the dependencies and data flow between different layers. Association mapping maps the node activity in the hot group list to the connection relationship in the link matrix one-to-one with the window timestamp, forming a node-to-link mapping. This can associate the node activity in the hot group list with the network topology in the link matrix, ensuring that when performing cross-layer remapping and data migration, resource scheduling and data allocation can be optimized reasonably according to the connection relationship and performance requirements between nodes, thereby improving the efficiency and stability of overall load balancing.

[0032] S1.3: Using complete profile data, identify highly active node groups and interaction links through clustering and analysis to obtain a list of hot groups and a link matrix.

[0033] Furthermore, based on the performance and status information of each node under multi-level indicators and the frequency and timing characteristics of link interactions contained in the complete profile data, cluster analysis is used to group the nodes, and nodes with high access frequency, high data interaction density and strong correlation are aggregated into highly active node groups. It should be noted that clustering analysis refers to analyzing the characteristic data of nodes and links to group nodes and links with high similarity into a hot group list. Based on the similarity measure between nodes, clustering analysis classifies highly active nodes and frequently interacting links into the same category, identifies nodes and links in the network that require key attention, optimizes data transmission and resource allocation, ensures that high-load areas are effectively scheduled and processed, and improves overall performance and stability.

[0034] Based on the real-time connection relationship and communication strength between nodes, a set of frequently interacting links is extracted to form a hot group list and a link matrix.

[0035] S1.4: Dynamic tracking refers to continuously collecting multi-level indicator data and simultaneously monitoring the status of highly active nodes and links during operation.

[0036] Furthermore, dynamic tracking refers to the continuous collection and real-time monitoring of multi-level indicator data of nodes and links during operation, including node load, bandwidth usage, latency, and packet loss, and the ability to adjust load balancing strategies when network status and resource consumption change. Among them, the dynamic tracking mechanism synchronously monitors the status of nodes and links, dynamically builds and updates a complete picture of the network, continuously senses hot zones, cold zones, congestion status and migration effects, and makes real-time adjustments based on feedback information to ensure the response speed and stability of the load balancing strategy and optimize overall performance and resource utilization efficiency.

[0037] It should be noted that load balancing strategy refers to dynamically adjusting the distribution of data traffic in a network system by comprehensively analyzing the performance, status, and resource requirements of different nodes and links to ensure load balance and efficient resource utilization. Load balancing strategy includes real-time monitoring and dynamic tracking of multi-layer indicators, combined with network topology and node performance, to migrate, distribute, and schedule data between high-load areas and cold areas. Through cross-layer remapping, bandwidth scheduling, and adaptive parameter tuning, performance is continuously optimized to avoid network congestion, improve response speed and stability, and ensure high efficiency and reliability in data transmission.

[0038] S2: Based on the hot group list and link matrix, perform data analysis on the target node set of each hot group at different levels to obtain a cross-level remapping candidate list.

[0039] S2.1: Based on the hot group list and link matrix, the analytic hierarchy process (AHP) is used to perform weight division and hierarchical data analysis on the target node set of each hot group at different levels, and to obtain the importance ranking of cross-level nodes of each hot group.

[0040] Furthermore, based on the highly active node groups identified in the hot group list and combined with the node connection relationships recorded in the link matrix, the target node set of each hot group at each level is obtained. Furthermore, for each set of target nodes and the access frequency, connection density, and link load status of the target nodes in the current level, the relative weight of each node in the current level is analyzed using the analytic hierarchy process (AHP). The weight results of each node in the same hot group under different levels are normalized and integrated to form a ranking sequence that reflects the comprehensive cross-level influence, thus obtaining the cross-level node importance ranking of each hot group.

[0041] It should be noted that the Analytic Hierarchy Process (AHP) is a decision-making method that combines quantitative and qualitative approaches for multi-criteria decision-making. It primarily decomposes complex problems into multiple levels for structured analysis. In AHP, the problem is divided into multiple hierarchical structures, typically including an objective level, a criterion level, and a solution level. The importance of each element is compared pairwise, weights are assigned to each pair, and consistency checks are performed to ensure the rationality of the comparison results. By synthesizing the weights of each level, the priority ranking of decision solutions is obtained. In this invention, AHP is used to perform weight partitioning and hierarchical data analysis on sets of objective nodes at different levels, helping to determine the priority of cross-level nodes.

[0042] S2.2: Based on the importance of cross-layer nodes, the path consistency of the sorting results is verified and the mapping feasibility is screened using the connection relationship in the link matrix, and a cross-layer remapping candidate list is generated.

[0043] Furthermore, based on the importance ranking of cross-layer nodes obtained by the analytic hierarchy process, and combined with the actual connection topology between nodes represented by the link matrix, the path consistency of each group of cross-layer node mapping paths is checked to determine the effective path of each group of cross-layer nodes in the link matrix. It should be noted that path consistency verification refers to analyzing the connection relationships in the link matrix during cross-layer remapping to ensure that the paths between each node will not cause link breaks or path inconsistencies during migration, thus meeting the requirements of load balancing. Path consistency verification is based on the feasibility screening of confirmed paths to ensure that data migration and distribution do not exceed the bandwidth limits of the links, avoid the generation of new congestion points, and ensure that the overall performance and stability can be effectively guaranteed when performing cross-layer remapping.

[0044] Based on link stability, the mapping path is screened for feasibility, and only cross-layer node combinations that meet the path consistency requirements and have feasible connection relationships are retained to generate a cross-layer remapping candidate list.

[0045] Feasibility screening refers to the process of generating a cross-layer remapping candidate list, and then performing path consistency verification on the candidate nodes based on the connection relationships in the link matrix and the importance ranking of nodes at each level. This process determines whether each candidate node has reachability and communication stability with the target node. By analyzing parameters such as network bandwidth, latency, and load between nodes, the process ensures that the mapping of each candidate node is feasible under performance and network conditions, and eliminates infeasible paths that are congested or do not meet performance requirements, thus selecting a suitable set of nodes.

[0046] S3: Based on the cross-layer remapping candidate list, the link matrix is ​​used to perform bandwidth hot zone projection and congestion determination and cold zone priority sorting to obtain the migration instructions and node parameters of link hot and cold distribution.

[0047] S3.1: Based on the cross-layer remapping candidate list, the load thermal distribution of each link is obtained by hot zone projection of bandwidth usage using the link matrix.

[0048] Furthermore, the mapping relationships of each candidate node included in the cross-layer remapping candidate list are associated with the real-time bandwidth usage data of the corresponding links in the link matrix. The bandwidth usage intensity is distributed and mapped on the link topology using a spatial mapping method to form a load heat distribution that reflects the load density of each link, thus obtaining the load heat distribution of each link.

[0049] It should be noted that spatial mapping refers to analyzing nodes and their connections to map nodes and resources at different levels to appropriate locations in the network topology, thereby optimizing data flow and resource allocation. Spatial mapping combines the physical and logical relationships between nodes and performs bandwidth hotspot projection and congestion assessment to prioritize the allocation of cold-spot nodes to idle network areas, improving data transmission efficiency and overall network performance. S3.2: Based on the load thermal distribution, determine the congestion status in the link matrix, and prioritize the cold zone links to obtain the migration instructions and node parameters for the link hot and cold distribution.

[0050] Furthermore, based on the load heat distribution, the bandwidth usage of each link in the link matrix is ​​analyzed to identify links with high load and mark them as congested. Combined with the cold zone links in the link matrix, which have low bandwidth usage and light load, the cold zone links are prioritized and selected for data transfer during the migration process. The link matrix is ​​then used to project the bandwidth usage into hot zones to obtain the load heat distribution of each link. It should be noted that bandwidth usage analysis refers to analyzing throughput, utilization, round-trip latency, packet loss, jitter, queuing signs, and burstiness around the occupancy level of links in different time windows and directions. This involves correlating the timing characteristics of each link with the traffic between nodes to form bandwidth heat map distribution and cold zone location, and drawing conclusions on the support of migration commands and node parameters.

[0051] Based on the load heat distribution, the congestion status of each link in the link matrix is ​​determined, and the determination result is combined with the availability information of cold zone links. The links are prioritized according to the cold zone priority principle, and migration instructions and node parameters for link hot and cold distribution are generated.

[0052] The judgment result refers to the standardized set of conclusions formed after the completion of the bandwidth hot zone projection and congestion judgment, path consistency verification and mapping feasibility screening, link health detection time sequence consistency verification and counter-evidence correction stages. The content includes whether the cross-layer remapping candidate is approved, whether the migration command and node parameters are executable, whether the directory and route are feasible to adopt, the link health level and the attribution of anomalies, and the parameter tuning scheme for adaptive parameter tuning. The judgment result is used to drive the generation of migration execution report, adoption report and anomaly list, and serves as the basis for dynamically adjusting the order of different layers and nodes and updating the migration execution report.

[0053] It should be noted that the adaptive parameter tuning method identifies parameter deviations and performance fluctuations under different states by analyzing the adoption reports and anomaly lists monitored in real time, automatically adjusts parameters, and continuously corrects and updates the execution strategy by verifying the effect. This ensures that the strategy can be adaptively adjusted when the load and network environment change, avoiding manual intervention and achieving dynamic optimization of the load balancing strategy.

[0054] S4: Obtain a migration execution report by performing link health checks on migration commands and node parameters.

[0055] S4.1: Obtain basic link health data by detecting link connectivity and load status of migration commands and node parameters.

[0056] Furthermore, based on the source and target nodes specified in the migration instruction, and combined with the link identification information of the node parameters, connectivity probe requests are initiated sequentially to confirm whether the link is feasible. Simultaneously, real-time load status data of the link during the probe period is collected, including bandwidth utilization and response latency. The connectivity results and load status data are correlated and integrated to obtain basic link health data.

[0057] It should be noted that the standard for link reachability refers to a qualified conclusion formed after multi-dimensional verification of the connection. This requires end-to-end route resolution and successful bidirectional transmission; ensuring normal session establishment and maintenance without frequent resets or timeouts; ensuring read / write probes continuously return complete and valid data during round trips without prolonged periods of no response or error code accumulation; ensuring stable latency and data loss performance within the observation window, without significant jitter or sudden blockages; and ensuring the access path is not blocked by access control policies, isolation policies, and rate limiting policies (using network security policies and flow control mechanisms to restrict or block non-compliant communication requests, traffic, and services to ensure the security, stability, and fairness of network resources and avoid unauthorized access, overload, and abuse). If the control channel and data channel are blocked or temporarily unable to remain stable within the observation window, it is considered unreachable.

[0058] Specifically, the standard means that the link must be able to resolve end-to-end routes, achieve successful bidirectional transmission, maintain stable sessions, and return complete and valid data. It must also be stable in terms of latency, packet loss, and bandwidth usage, and be unaffected by access control, isolation, or rate limiting policies.

[0059] S4.2: Based on the basic health data of the link, the timing consistency verification method is used to compare and analyze the link status fluctuations and abnormal responses to obtain the link migration execution report.

[0060] Furthermore, by arranging the basic health data of the links in a time series, the link connectivity indicators and load status change trends within a continuous time window are extracted; and based on the time series consistency verification method, the link status fluctuation pattern within the current time window is aligned with the historical normal response pattern point by point and deviation processing is performed to identify abnormal response segments that deviate from normal behavior. It should be noted that the time sequence consistency verification method refers to, after performing time alignment and structural normalization on multi-source data under a unified time benchmark, comparing and replaying the key processes of migration instructions, block warm-up, directory update, route adoption, and performance fallback in chronological order according to the event causal chain; determining the consistency between the behavior of the links and nodes and the expected process by checking whether the order within the window is maintained, whether cross-layer propagation is complete, whether the time interval is reasonable, whether duplication or missing occurs, and whether records from different sources corroborate each other; (the expected process refers to ensuring that each step, data transmission, and state change is consistent with the initial plan during execution, in accordance with the predetermined order, rules, and goals, avoiding abnormal interruptions and deviations from the expected behavior), and outputting the judgment conclusion of pass or fail and the corresponding anomaly location information.

[0061] By associating and integrating abnormal response segments with basic link health data, a migration execution report of the link is generated, which includes the link execution status, abnormal location, and health change trajectory.

[0062] S5: Use the proof of contradiction to verify and replay the migration execution report for comparison, and generate an adoption report and an anomaly list.

[0063] S5.1: Based on the migration execution report, perform consistency verification feature comparison between the current link behavior and the expected behavior to obtain a set of deviation features.

[0064] Furthermore, by extracting the actual response timing, connectivity status, and load fluctuation characteristics of the links during the migration process from the migration execution report, and generating corresponding expected response timing, connectivity status, and load fluctuation characteristics based on the expected link behavior implied by the migration instructions and node parameters, the actual response timing is aligned and compared point by point with the expected response timing, the actual connectivity status is matched and verified with the expected connectivity status, and the actual load fluctuation characteristics are compared and verified with the expected load fluctuation characteristics to check for trend consistency. Based on the consistency verification results of the comparison, inconsistent feature items are identified and aggregated to form a deviation feature set.

[0065] It should be noted that consistency verification feature comparison refers to the process of aligning the migration execution report and the expected behavior description with time and normalizing their structure, and then extracting a set of comparable features from the dimensions of directory adoption, read / write order, causal relationship, replication freshness, link round trip, session stickiness, and chunk completion. Each feature is then checked against rules of sequential consistency, content consistency, temporal consistency, and location consistency to identify the location, scope, direction, and persistence of inconsistencies. These are then merged into a set of deviation features, and the judgment results are output, providing a basis for reverse verification and the generation of adoption reports and anomaly lists.

[0066] Specifically, expected behavior includes (expected link behavior, expected response timing, expected connectivity status, and expected load fluctuation characteristics). It refers to the ideal operating mode that is predicted and defined in advance based on the target, network topology, node configuration, and external conditions during operation and migration. The expected timing of data transmission, the connectivity status of the link, and the load fluctuation characteristics are used as standards to measure the actual performance of the system.

[0067] S5.2: Based on the deviation feature set, reverse verification is used to comprehensively judge the adoption feasibility scale and the distribution of multi-level anomalies, and an adoption report and an anomaly list are generated.

[0068] Furthermore, based on the deviation feature set, a reverse verification method is used to conduct a retrospective analysis on the differences between the actual behavior and the expected behavior of each link in the migration execution report, and obtain the adoption feasibility scale results of the unified collection of multi-level indicators. It should be noted that the reverse verification method refers to starting with the migration execution report, replaying the link and routing behavior according to the migration instructions and node parameters that were in effect at that time, and comparing the expected conclusions when comparing bandwidth hotspot projection and congestion judgment, path consistency verification and mapping feasibility screening (the expected conclusions refer to the ideal results and target states predicted and set according to the migration instructions and node parameters, which are used to compare and verify with the actual execution results), to achieve a step-by-step tracing from the result to the cause; through multi-source observation and cross-verification of event timelines, it verifies whether the directory has been adopted, whether the read and write paths have been hit, and whether replication and consistency have been achieved, and forms an adoption report and an anomaly list based on the output adoption feasibility conclusions and anomaly attribution.

[0069] The feasibility scale for adoption is comprehensively judged, and the spatial distribution characteristics of multi-level anomalies are identified based on the synchronous monitoring data of node links, generating an adoption report and an anomaly list.

[0070] The comprehensive judgment is based on a set of evidence that gathers three types of evidence to support the conclusion under the premise of time alignment. These evidences are: evidence of consistency between multi-layer indicators and the link matrix; evidence of consistency between migration instructions and node parameters and the migration execution report; and evidence of causal closure between the adoption report and the anomaly list. By coordinating the matching of latency, loss, bandwidth usage, read / write hits, replication freshness and cross-layer placement relationships, the feasibility and landing point matching in bandwidth hotspot projection, congestion judgment, path consistency verification and mapping feasibility screening are compared. Combined with the verification of whether multi-layer anomalies meet the predetermined goals and complete the anomaly attribution, a standardized basis for adopting feasible conclusions and anomaly location is formed.

[0071] S6: Based on the adoption report and the anomaly list, use the adaptive parameter tuning method to dynamically adjust the order of different levels and nodes and update the migration execution report.

[0072] S6.1: Adaptive parameter tuning refers to dynamically adjusting the parameters and node order at each level based on feedback information generated during operation, and continuously updating the execution strategy.

[0073] Furthermore, adaptive parameter tuning refers to automatically adjusting key parameters at each level by collecting feedback information in real time, adopting reports and anomaly lists, analyzing parameter deviations and performance fluctuations through adaptive parameter tuning, dynamically adjusting configurations, avoiding manual intervention, verifying the effect after each adjustment, and continuously correcting and updating the execution strategy.

[0074] The execution strategy refers to the dynamic adjustment of different levels and node sequences based on the adoption report and the anomaly list using an adaptive parameter tuning method, and the generation and updating of migration execution reports, which serve as dynamic decision-making rules and implementation arrangements for load migration and resource scheduling.

[0075] S6.2: Based on the adoption report and the exception list, a consistency check is used to dynamically adjust the node order at different levels to obtain the updated hierarchical node configuration.

[0076] Furthermore, based on the link behavior adoption results recorded in the adoption report and the multi-level anomaly point distribution marked in the anomaly list, the current order of nodes in each level is compared for consistency to identify the node positions that deviate from the expected behavior; and based on the node activity and link status reflected in the complete profile data continuously updated by the dynamic tracking mechanism, combined with the cross-level node importance ranking obtained by the analytic hierarchy process, the order of the deviating nodes in their respective levels is rearranged to obtain the updated level node configuration.

[0077] S6.3: Based on the updated hierarchical node configuration, the migration execution report is synchronously corrected using an adaptive parameter tuning method to generate an optimized migration execution report.

[0078] Furthermore, based on the deviations in link behavior reflected in the adoption report and the anomaly list, consistency verification is performed on the node order at different levels to determine the level node configuration that needs to be adjusted. The updated level node configuration is then used as input, combined with the link health data recorded in the migration execution report, to dynamically adjust the parameters and node order at each level through an adaptive parameter tuning method, so that the migration execution report and the node order are consistent, and an optimized migration execution report is generated.

[0079] This embodiment also provides a dynamic load balancing system based on Redis and memory collaboration, including a data acquisition and detection module, which obtains complete profile data and generates a hot group list and link matrix through unified collection of multi-level indicators and synchronous monitoring of node links; an analysis and mapping module, which analyzes the target node set of each hot group at different levels according to the hot group list and link matrix to obtain a cross-level remapping candidate list; a projection and sorting module, which uses the link matrix to perform bandwidth hot zone projection, congestion judgment, and cold zone priority sorting based on the cross-level remapping candidate list to obtain migration instructions and node parameters for link hot and cold distribution; a detection and execution module, which obtains a link migration execution report by detecting the link health of the migration instructions and node parameters; a verification and comparison module, which verifies and replays the migration execution report using proof by contradiction correction to generate an adoption report and an anomaly list; and an adjustment and update module, which dynamically adjusts the order of different levels and nodes and updates the migration execution report according to the adoption report and the anomaly list using an adaptive parameter tuning method.

[0080] In summary, this invention achieves real-time dynamic tracking of the entire link status through unified collection of multi-layer indicators and synchronous monitoring of node links, accurately identifying highly active hot groups and link dependencies, and providing a complete data foundation for subsequent load scheduling; by combining a cross-layer remapping candidate list with a link matrix to project bandwidth hot zones and prioritize cold zones, it realizes hot / cold perception and congestion prediction of migration instructions, effectively improving the accuracy of resource scheduling under Redis and memory collaboration.

[0081] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A dynamic load balancing method based on Redis and memory collaboration, characterized in that: include, By collecting multi-level indicators in a unified manner and monitoring node links synchronously, complete profile data is obtained and a hot group list and link matrix are generated. Based on the hot group list and link matrix, the target node set of each hot group at different levels is analyzed to obtain a cross-level remapping candidate list. Based on the cross-layer remapping candidate list, the link matrix is ​​used to perform bandwidth hot zone projection and congestion determination and cold zone priority sorting to obtain the migration instructions and node parameters of link hot and cold distribution. By performing link health checks on migration commands and node parameters, a migration execution report is obtained. The migration execution report is verified and replayed for comparison using the evidence-reverse correction method, generating an adoption report and an anomaly list; Based on the adoption report and the anomaly list, an adaptive parameter tuning method is used to dynamically adjust the order of different levels and nodes and update the migration execution report.

2. The dynamic load balancing method based on Redis and memory collaboration as described in claim 1, characterized in that: The process involves unified collection of multi-level indicators and synchronous monitoring of node links to obtain complete profile data and generate a hotspot list and link matrix. The specific steps are as follows: Based on multi-source indicator collection, performance and status data at all levels are uniformly aggregated to obtain a multi-level indicator set covering the entire link. Based on the multi-level indicator set and the real-time status data of the links between nodes, the nodes and connection relationships are dynamically tracked through synchronous monitoring to generate complete profile data; By utilizing complete profile data, clustering and analysis are used to identify highly active node groups and interaction links, resulting in a list of hot groups and a link matrix.

3. The dynamic load balancing method based on Redis and memory collaboration as described in claim 2, characterized in that: The dynamic tracking refers to the continuous collection of multi-level indicator data and the synchronous monitoring of highly active nodes and link status during operation.

4. The dynamic load balancing method based on Redis and memory collaboration as described in claim 3, characterized in that: The step involves analyzing the target node sets of each hot group at different levels based on the hot group list and link matrix to obtain a cross-level remapping candidate list. The specific steps are as follows. Based on the hot group list and link matrix, the analytic hierarchy process is used to perform weight partitioning and hierarchical data analysis on the target node set of each hot group at different levels, and to obtain the cross-level node importance ranking of each hot group. Based on the importance ranking of nodes across layers, the path consistency verification and mapping feasibility screening of the ranking results are performed using the connection relationships in the link matrix, generating a candidate list for cross-layer remapping.

5. The dynamic load balancing method based on Redis and memory collaboration as described in claim 4, characterized in that: The process involves using a cross-layer remapping candidate list, leveraging the link matrix to project bandwidth hot zones, determine congestion, and prioritize cold zones to obtain migration instructions and node parameters for link hot and cold zone distributions. The specific steps are as follows: Based on the cross-layer remapping candidate list, the load thermal distribution of each link is obtained by hot zone projection of bandwidth usage using the link matrix. Based on the load thermal distribution, the congestion status in the link matrix is ​​determined, and the links in the cold zone are prioritized to obtain the migration instructions and node parameters for the hot and cold distribution of the links.

6. The dynamic load balancing method based on Redis and memory collaboration as described in claim 5, characterized in that: The process of obtaining a migration execution report by performing link health checks on migration commands and node parameters involves the following steps: By probing the link connectivity and load status of migration commands and node parameters, basic link health data can be obtained. Based on the basic health data of the link, a time-series consistency verification method is used to compare and analyze the link status fluctuations and abnormal responses to obtain the link migration execution report.

7. The dynamic load balancing method based on Redis and memory collaboration as described in claim 6, characterized in that: The process of verifying and replaying the migration execution report using evidence-by-counterfeiting to generate an adoption report and an anomaly list involves the following steps: Based on the migration execution report, the current link behavior is compared with the expected behavior to obtain a set of deviation features; Based on the set of deviation features, reverse verification is used to comprehensively judge the adoption feasibility scale and the distribution of multi-level anomalies, and an adoption report and an anomaly list are generated.

8. The dynamic load balancing method based on Redis and memory collaboration as described in claim 7, characterized in that: The adaptive parameter tuning method refers to dynamically adjusting the parameters and node order at each level based on feedback information generated during operation, and continuously updating the execution strategy.

9. The dynamic load balancing method based on Redis and memory collaboration as described in claim 8, characterized in that: The process involves dynamically adjusting the order of different levels and nodes and updating the migration execution report based on the adoption report and the anomaly list, using an adaptive parameter tuning method. The specific steps are as follows: Based on the adoption report and the anomaly list, consistency checks are used to dynamically adjust the node order at different levels to obtain the updated hierarchical node configuration. Based on the updated hierarchical node configuration, the migration execution report is synchronously corrected using an adaptive parameter tuning method to generate an optimized migration execution report.

10. A dynamic load balancing system based on Redis and memory collaboration, based on the dynamic load balancing method based on Redis and memory collaboration as described in any one of claims 1 to 9, characterized in that: include, The data acquisition and detection module obtains complete profile data and generates a hot group list and link matrix through unified acquisition of multi-level indicators and synchronous monitoring of node links; The analysis and mapping module performs data analysis on the target node set of each hot group at different levels based on the hot group list and the link matrix to obtain a cross-level remapping candidate list. The projection sorting module, based on the cross-layer remapping candidate list, uses the link matrix to perform bandwidth hot zone projection and congestion determination and cold zone priority sorting to obtain the migration instructions and node parameters of link hot and cold distribution. The detection and execution module performs link health checks on migration commands and node parameters to obtain a migration execution report for the link. The verification and comparison module uses counter-evidence correction to verify and replay the migration execution report, generating an adoption report and an anomaly list; The adjustment and update module dynamically adjusts the order of different levels and nodes and updates the migration execution report based on the adoption report and the exception list using an adaptive parameter tuning method.

Images