Communication state monitoring method and device based on intelligent computing cluster, and storage medium

Abstract

The invention provides a communication state monitoring method and device based on an intelligent computing cluster and a storage medium, and the method comprises the steps: firstly obtaining information of an intelligent computing node and a communication link, constructing a communication topology model, carrying out the quality optimization of a candidate path through employing an ant colony algorithm, and forming a stable optimal communication path set; then, link-level, node-level and end-to-end path-level communication monitoring data are collected, dynamic adjustment of sampling points is realized by using an ant colony algorithm, a communication monitoring result with excellent coverage and timeliness is obtained, a communication state evaluation index is calculated through a multi-scale time window, the weight of the evaluation index is optimized in combination with the ant colony algorithm, and a communication state evaluation result is obtained. And obtaining a node communication risk level, constructing computing power availability measurement of communication state perception, optimizing a scheduling decision by adopting an ant colony algorithm, and outputting a task optimal execution node. By introducing task execution feedback, threshold adjustment and global pheromone updating mechanisms, closed-loop adaptive optimization of communication monitoring and scheduling decision is realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent cluster management technology, and in particular to a communication status monitoring method, device and storage medium based on intelligent computing clusters. Background Technology

[0002] With the rapid growth of large-scale artificial intelligence model training, scientific computing, and high-performance parallel rendering, intelligent computing clusters have become large-scale computing infrastructures with multiple nodes, routes, and links intertwined. In such clusters, the performance of computing tasks is not only limited by the capabilities of the computing chips themselves, but also highly dependent on the stability of communication links between nodes, changes in topology paths, and bandwidth contention.

[0003] Existing technologies generally separate communication quality monitoring from computing power scheduling: the communication system is only responsible for link measurement, and the scheduling system only allocates tasks based on computing resources. This results in a structural contradiction where communication fluctuations are imperceptible and scheduling decisions are decoupled from network status. Especially in intelligent computing cluster environments, link fluctuations are not abnormal but normal. A small jitter at a node port, a brief congestion in path traffic, or a change in cross-switch forwarding strategy can instantly trigger a decrease in training throughput, gradient synchronization delay, model iteration stall, or even a complete task rollback. Existing scheduling strategies, based on static link evaluation, average bandwidth, or offline test metrics for resource matching, are difficult to reflect the real dynamic communication situation. This often leads to situations where scheduling appears optimal but execution is actually the worst. Therefore, we propose a communication status monitoring method, device, and storage medium based on intelligent computing clusters.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a communication status monitoring method, device, and storage medium based on intelligent computing clusters, thereby resolving the technical problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The communication status monitoring method based on intelligent computing clusters includes the following steps: S1. Configure multiple intelligent computing nodes and corresponding network interfaces in the intelligent computing cluster, obtain the connection relationship between the intelligent computing nodes and the external communication network, construct a communication topology model, and identify the mapping relationship between each communication path and the corresponding intelligent computing node in the communication topology model. S2. In the intelligent computing cluster, continuously collect communication status monitoring data for each communication path and each intelligent computing node's network interface, and collect and store the communication status monitoring data according to the mapping relationship between the communication path and the intelligent computing node to obtain a communication status monitoring data set. S3. Perform statistical analysis on each communication path within a preset time window to obtain the communication status evaluation index of the communication path; based on the mapping relationship between the communication path and the intelligent computing node, aggregate the communication status evaluation index of multiple communication paths to the corresponding intelligent computing node to obtain the aggregated communication status evaluation index. S4. Classify the communication risk of each intelligent computing node and generate a corresponding communication status-aware computing power availability metric. Intelligent computing nodes with good communication status are assigned a higher computing power availability metric, while intelligent computing nodes with fluctuating communication status are assigned a lower computing power availability metric, thus forming a communication status-aware computing power availability metric result. S5. Match and evaluate the intelligent computing tasks to be executed, and generate communication status awareness scheduling suggestion information. This gives higher priority to intelligent computing nodes with better communication status and higher computing power availability metrics in the scheduling suggestion information, and explicitly maps the communication status monitoring results into information that can be used for task scheduling linkage. S6. Output the communication status awareness computing power availability measurement results and communication status awareness scheduling suggestions to the operation and maintenance interface or intelligent computing task scheduling module; based on the feedback of communication anomalies and task completion quality, adaptively adjust the calculation parameters of communication status assessment indicators and the boundary thresholds of communication risk classification, and update the communication topology model and communication status monitoring data set collection strategy.

[0007] S1 specifically includes: Obtain the processing power, network interface type, and link connection information of each intelligent computing node in the intelligent computing cluster to provide basic node data for communication topology construction; The link structure between the intelligent computing node and the external communication network is analyzed to extract link bandwidth, link structure hierarchy and link direction information, forming a basic link dataset. Based on the node data and link data, an initial version of the communication topology model is constructed, using intelligent computing nodes as topology nodes and communication links as topology edges. In the communication topology model, depth-first search is used to enumerate all possible communication paths, and a mapping relationship is established between each communication path and the corresponding intelligent computing node to form a candidate set of communication paths; The ant colony algorithm is used to optimize the path quality of the candidate communication paths. A pheromone function is constructed based on link length, bandwidth stability, and historical availability to generate an optimized communication topology model.

[0008] S2 specifically includes: Determine the core monitoring indicators for communication status monitoring, including latency, bandwidth utilization, packet loss rate, jitter, and retransmission ratio, and establish indicator collection templates. Based on the communication topology model, data is collected from the network interface of each intelligent computing node to obtain node-level communication status monitoring data. Based on the set of communication paths, real-time monitoring data for each communication path is collected, including the time series data of the aforementioned indicators. Based on the mapping relationship between communication paths and intelligent computing nodes, node-level monitoring data and path-level monitoring data are uniformly collected to generate a communication status monitoring data set. Using the ant colony algorithm, the data uncertainty of each link is evaluated from the communication status monitoring data set. The monitoring sampling frequency and sampling point distribution are dynamically adjusted through the pheromone feedback mechanism, so that the collection resources are given priority to the communication status fluctuation area.

[0009] S3 specifically includes: Based on preset time windows of seconds, minutes, and hours, the communication status monitoring data set is segmented and organized to perform multi-scale evaluation; For each communication path, the mean delay, jitter variance, packet loss ratio, bandwidth utilization change rate, and retransmission ratio change rate are calculated in multi-scale time windows to obtain path-level communication status evaluation indicators. Based on the mapping relationship between communication paths and intelligent computing nodes, the evaluation indicators of multiple communication paths are aggregated by node to obtain intelligent computing node-level communication status evaluation indicators. The communication status assessment indicators are scaled to ensure that indicators with different dimensions can be compared under a unified assessment system. The ant colony algorithm is used to dynamically optimize the weights of each monitoring indicator, and the sensitivity to key indicators is improved through pheromone gain or decay mechanisms.

[0010] S4 specifically includes: Based on communication status assessment indicators, a communication risk classification rule is constructed, defining threshold ranges for low risk, medium risk, and high risk. Based on the communication risk classification rules, a communication risk analysis is performed on each intelligent computing node to generate a node-level communication risk label; Sensitivity analysis is performed on the bandwidth requirements, latency tolerance, and task priority of the target intelligent computing task, and a mapping relationship is established between communication risk labels and task sensitivity. Based on communication risk labels and task sensitivity, a communication status-aware computing power availability metric is generated for each intelligent computing node. By using the ant colony algorithm to update the pheromone concentration of nodes based on the historical task execution results, the availability metric of nodes with stable communication is dynamically increased, while the availability metric of nodes with fluctuating communication is decreased, making the availability metric closer to the real network performance.

[0011] S5 specifically includes: The communication requirements, computing power requirements, and real-time requirements for performing intelligent computing tasks are analyzed to form a task requirement vector; Based on the communication status-aware computing power availability metric, a set of candidate intelligent computing nodes that meet the minimum communication requirements is selected from all intelligent computing nodes. For each candidate node, a node scheduling preference score is calculated based on its availability metric, task requirement vector, and historical task completion rate. Candidate nodes are sorted according to scheduling preference scores to generate a communication state-aware task scheduling candidate sequence; The ant colony algorithm is used to optimize the scheduling path of candidate node sequences, and the optimal task allocation path is found through a joint mechanism of pheromone and heuristic function.

[0012] S6 specifically includes: Output the communication status-aware computing power availability metric and the task scheduling candidate sequence to the scheduling system and operation and maintenance interface; During the execution of tasks at the target intelligent computing node, information on communication anomalies, task delay deviations, and task completion quality feedback is collected. Based on task execution feedback, the communication risk classification parameters, scheduling preference calculation parameters, and evaluation index weight parameters are adaptively adjusted. When a change in the communication topology, a sudden drop in link quality, or a node state migration is detected, the topology update process is triggered to ensure the timeliness of the topology model. By using the ant colony algorithm to adjust the global pheromone distribution based on task execution feedback and topology update results, a positive closed-loop optimization is formed between monitoring, evaluation, scheduling, and feedback, thereby improving the stability of the entire intelligent computing cluster's communication status monitoring.

[0013] An electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps in the communication status monitoring method based on intelligent computing clusters.

[0014] A computer-readable storage medium storing program instructions that can be executed by a processor to implement the steps in the communication status monitoring method based on a smart computing cluster.

[0015] The beneficial effects of this invention are as follows: This invention quantifies communication risk level, link stability and remaining computing resources of nodes in a unified way by constructing a computing power availability metric that is aware of communication status. This makes scheduling strategies no longer dependent on traditional "pure computing power optimization" or "static bandwidth" judgments, thereby avoiding high computing power nodes from becoming performance bottlenecks due to communication fluctuations and significantly reducing the scheduling misjudgment rate.

[0016] This invention uses an ant colony algorithm to guide monitoring sampling points to focus on areas of communication fluctuation. It reduces monitoring overhead when the link is stable and increases sampling density when the link is unstable, enabling on-demand allocation of monitoring resources. While ensuring the accuracy of the overall communication status, it allows for the timely capture of short-period phenomena such as sudden jitter, instantaneous congestion, and routing changes, thus improving the ability to detect communication anomalies.

[0017] This invention uses the ant colony algorithm to adaptively optimize the weights of multi-scale evaluation indicators. It does not rely on manual experience parameters and can dynamically adjust the importance of different communication indicators (latency, bandwidth, packet loss, jitter, retransmission) according to actual operation results to adapt to different business scenarios such as model training, gradient synchronization, inference services, and real-time rendering, thereby achieving a highly generalizable communication status evaluation capability.

[0018] This invention introduces task execution feedback and maps it to global pheromone updates, enabling the system to continuously adjust risk thresholds, scheduling parameters, and availability metrics based on actual task execution results. This creates a scheduling trend where "failed nodes gradually decrease in priority, while stable nodes are gradually strengthened," avoiding the systemic inertia of traditional systems that continuously select incorrect nodes during long-term operation.

[0019] This invention implements a closed-loop optimization mechanism for communication monitoring, communication evaluation, availability measurement, and scheduling decisions, enabling the scheduling system to select the optimal node despite network fluctuations. Even under conditions of limited link resources or frequent topology changes, it maintains high throughput, low latency fluctuations, and high task completion rates, thereby improving the overall computing power utilization efficiency of the cluster. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the communication status monitoring method based on intelligent computing clusters according to the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1: As Figure 1 As shown, this embodiment provides a communication status monitoring method based on an intelligent computing cluster, including the following steps: S1. Configure multiple intelligent computing nodes and their corresponding network interfaces in the intelligent computing cluster, obtain the connection relationship between the intelligent computing nodes and the external communication network, construct a communication topology model based on the connection relationship, identify the mapping relationship between each communication path and the corresponding intelligent computing node in the communication topology model, and provide a topology basis for subsequent monitoring of the communication status of intelligent computing nodes. S2. Based on the communication topology model, communication status monitoring data, including latency, bandwidth utilization, packet loss rate, jitter and retransmission ratio, are continuously collected for each communication path and network interface of each intelligent computing node in the intelligent computing cluster. The communication status monitoring data is then collected and stored according to the mapping relationship between the communication path and the intelligent computing node to obtain a set of communication status monitoring data for intelligent computing nodes. S3. Based on the communication status monitoring data set, perform statistical analysis on each communication path within a preset time window, and calculate the communication status evaluation index of the communication path by combining at least two of the following indicators: latency, bandwidth utilization, packet loss rate, jitter, and retransmission ratio; and aggregate the communication status evaluation index of multiple communication paths to the corresponding intelligent computing node according to the mapping relationship between the communication path and the intelligent computing node to obtain the aggregated communication status evaluation index for each intelligent computing node. S4. Based on the aggregated communication status evaluation index and the sensitivity of the intelligent computing tasks currently running by the intelligent computing nodes to communication quality, each intelligent computing node is classified into communication risks, and a corresponding communication status-aware computing power availability metric is generated. Intelligent computing nodes with good communication status are given a higher computing power availability metric, while intelligent computing nodes with fluctuating or deteriorating communication status are given a lower computing power availability metric, thus forming a communication status-aware computing power availability metric result for intelligent computing clusters. S5. Based on the communication status-aware computing power availability measurement results and the bandwidth requirements, latency tolerance, and task priority of the intelligent computing tasks to be executed, the intelligent computing tasks to be executed are matched and evaluated, and communication status-aware scheduling suggestion information containing a candidate set of target intelligent computing nodes and their priority order is generated. This makes intelligent computing nodes with better communication status and higher computing power availability measurement values ​​have higher priority in the scheduling suggestion information, thereby explicitly mapping the communication status monitoring results into linkage information that can be used for task scheduling. S6. Output the communication status awareness computing power availability measurement results and communication status awareness scheduling suggestions to the operation and maintenance interface and / or intelligent computing task scheduling module for operation and maintenance personnel to view and the scheduling module to call; and based on the communication anomaly events and task completion quality fed back during the actual operation of intelligent computing tasks, adaptively adjust the calculation parameters of communication status evaluation indicators and the boundary thresholds of communication risk classification, update the collection strategy of communication topology model and communication status monitoring data set, and realize continuous monitoring, scheduling linkage and closed-loop optimization of the communication status of intelligent computing cluster; S1 specifically includes the following sub-steps: S110, Steps for obtaining intelligent computing node information: Obtain the set of intelligent computing nodes in the intelligent computing cluster, denoted as... ,in This represents the nth intelligent computing node; simultaneously, each intelligent computing node is retrieved. The processing capability parameters (including but not limited to the number of CPU cores, video memory capacity, etc.) as well as the network interface type and interface identifier provide basic node data for subsequent communication topology construction.

[0023] S120, Communication Link Structure Parsing Steps: Parse the set of communication links between the intelligent computing node and the external communication network, denoted as... ,in This represents the i-th communication link; for each link... Extract its link bandwidth limit (Indicates link) Maximum available bandwidth), link physical length (Indicates link) The physical transmission distance and link direction information are used to form a set of link feature data.

[0024] S130. Initial steps for constructing the communication topology: Construct a communication topology graph based on the node set V and the link set E. , where G is a directed or undirected graph model describing the intelligent computing nodes and their communication link connections; and provides a unified topology basis for subsequent path enumeration and path quality optimization.

[0025] S140. Communication path enumeration steps: In the initial topology graph G, the depth-first search (DFS) algorithm is used as the path enumeration strategy to exhaustively enumerate the reachable paths between node pairs, thus obtaining the communication path set. ,in Represents the k-th communication path; for each communication path... Establish its contained link sequence The mapping relationship is used for subsequent path-level communication state aggregation and optimization.

[0026] S150. Topology path quality optimization steps based on ant colony algorithm: For each path Distribute pheromone concentration ,in Representing a path The pheromone concentration on the path; and define heuristic information for each path. ,in Representing a path The heuristic expectation value is used to guide ants in prioritizing their path selection during the path search process.

[0027] Heuristic information Specific computational form: In a specific implementation, heuristic information It can be constructed based on path quality as follows: ; in It is a path The total physical length (the sum of the physical lengths of all links in the path); It is a path The average latency; It is the average packet loss rate of the path; These are path quality weight parameters (representing the contribution weights of physical length, average latency, and average packet loss rate to path quality, respectively); the larger this heuristic information is, the better the path quality, and the more likely it is to be selected by ants.

[0028] The probability formula for an ant choosing a path: When the m-th ant chooses a path, its chosen path... The probability of is given by the following formula: ; in It is a pheromone importance parameter (representing the degree of importance of pheromones in path selection); It is a parameter representing the importance of heuristic information (indicating the degree to which heuristic information is important for path selection).

[0029] Pheromone update mechanism: After each iteration, the path pheromones are updated according to the following rules: ; in It is the pheromone evaporation coefficient (representing the proportion of pheromones retained from the previous round). This is the pheromone increment for this round, used to reward high-quality paths; to ensure algorithm executability, in one specific implementation, the pheromone increment can be defined as: ; in It is the path quality enhancement constant; It is the path comprehensive cost, defined as the weighted sum of the path's average delay, average packet loss rate, and physical length, and is used to evaluate the quality of path communication.

[0030] Ant Colony Algorithm Initialization and Termination Conditions: To ensure the optimization process is fully implementable, the ant colony algorithm must satisfy the following conditions at the start and end: Initialization conditions: The initial pheromone concentration for all paths is uniformly set to [value missing]. ,in The value is a preset positive number, representing the initial pheromone intensity.

[0031] Stopping condition: Iteration can be stopped when any of the following conditions are met: The number of iteration rounds reaches the preset maximum value. ,in Indicates the maximum number of iterations; continuous The optimal path remains unchanged in each iteration, where This represents the threshold for determining path stability.

[0032] Final optimized output: After stopping the iteration, the paths with the highest pheromone concentration are considered as optimized paths, resulting in the optimized set of communication paths. Based on this, the communication topology model is updated to form the final optimized communication topology diagram. ,in It represents the topology containing the optimal communication path and the optimal link quality distribution.

[0033] S2 specifically includes the following sub-steps: S210. Monitoring indicator initialization steps: Define a unified monitoring indicator system at the link level, node level, and path level for communication status monitoring.

[0034] For the link set Each link Set the following time series metrics: :link At any moment Round-trip delay; :link At any moment Bandwidth utilization; :link At any moment The packet loss rate; :link At any moment The time delay jitter; :link At any moment retransmission ratio; The sampling period between two consecutive samples.

[0035] To ensure the stability of the sampling frequency, the sampling time series is set as follows: ; where t represents the sampling time index.

[0036] S220, Node-level Monitoring Data Collection Steps: For each intelligent computing node The network interface collects node-level communication metrics at each sampling time t, forming a node-level metric sequence: ; ; ; ; .

[0037] Node-level metrics are calculated using a weighted average of the corresponding link metrics, for example: ; ; .

[0038] S230, Path-level Monitoring Data Acquisition Steps: Based on optimized communication topology Path set in For each path Collect path-level metrics using end-to-end probe messages: :path Round-trip delay at sampling time t; :path End-to-end bandwidth utilization; :path End-to-end packet loss rate; :path End-to-end jitter; :path The end-to-end retransmission ratio.

[0039] Path-level metrics can all be derived from the set of links within the path. For example: ; .

[0040] Path-level packet loss rate can be calculated as follows: ; in Indicates link The packet loss rate.

[0041] S240. Monitoring data collection and multidimensional index establishment steps: Collect all link-level indicators, node-level indicators, and path-level indicators into a unified monitoring data set. ,in: The complete collection of communication status monitoring data; Data is organized using a four-dimensional index structure: ; represent the sampling time, link number, node number, and path number, respectively.

[0042] To support fast queries, in collections Multidimensional R-Tree indexes or hash indexes can be built to enable fast extraction by time window, link, node or path.

[0043] S250, Ant Colony Algorithm-Based Adaptive Adjustment Steps for Sampling Points: To ensure that sampling resources are allocated to the most valuable links, the link set... Each link in the algorithm is considered a candidate collection point for the ant colony algorithm.

[0044] Pheromon concentration definition: for each link Set the pheromone concentration for sampling. ,in Indicates link The degree of preference for being sampled first.

[0045] Link sampling uncertainty index: Calculate link Uncertainty in the most recent N samples: ; in It is a link Uncertainty indicators; This represents variance; the greater the uncertainty, the more necessary it is to increase the sampling frequency.

[0046] Heuristic information definition: Define the heuristic information for this link as follows: This represents the sampling value of the link.

[0047] The probability formula for an ant choosing a path: The m-th ant chooses a path. The probability of being a key sampling point is: ; in It is a parameter related to the importance of pheromones; It is a parameter for the importance of heuristic information.

[0048] Pheromones update formula: After each iteration, the pheromones are updated as follows: ; in It is the pheromone evaporation coefficient; It is a link The increase in the sampled pheromone.

[0049] In one specific implementation, we have: ; in It is the uncertainty gain constant (meaning that the higher the uncertainty, the greater the pheromone increment).

[0050] Initialization and stopping conditions, to ensure the algorithm is feasible: initialization: ,in is the initial pheromone constant.

[0051] Stopping condition: The iteration terminates when any of the following conditions are met: The number of iterations reached ;continuous The link order remained unchanged; the final adaptive sampling strategy gives higher sampling priority to links with high uncertainty.

[0052] S3 specifically includes the following sub-steps: S310, Multi-scale time window division steps: To improve the timeliness and stability of communication status assessment, the monitoring data set is divided into... Divided according to multi-scale time windows.

[0053] set up: A time window length in seconds (e.g., 5 seconds) is used to capture rapid, sudden anomalies; Minute-level time windows (e.g., 1 minute) are used to capture stable trends; Hourly time windows (e.g., 1 hour) are used to extract long-cycle fluctuations. A collection of multi-scale time windows.

[0054] For the sampling time sequence By window length Perform sliding partitioning to form a window set: ,in: It is the first One time window; This indicates that t belongs to the time window. This step provides a time basis for subsequent multi-scale assessments.

[0055] S320, Path-level Communication Status Evaluation Indicator Calculation Steps: For each communication path (where P is the set of optimized paths), in each window Internal calculation of multidimensional statistical characteristics.

[0056] Window average of path-level metrics: For path Calculate the window average for latency, bandwidth utilization, packet loss rate, jitter, and retransmission ratio: ; in: It is a path Round-trip time delay at time t; It is path bandwidth utilization; ; ; ; .

[0057] Path volatility metric: Calculates the window variance of path jitter and delay to measure instability. ; Path-level multidimensional evaluation vector: ultimately forming a path-level communication state evaluation vector. ; in It is a path In the window The multidimensional state evaluation vector.

[0058] S330, Node-level communication status index aggregation steps: Based on the mapping between paths and intelligent computing nodes, aggregate the evaluation indicators of multiple paths to the node level.

[0059] set up: For the nodes involved The set of paths; This is the path-level evaluation vector; Node-level evaluation vector.

[0060] Node-level aggregation uses a weighted average method: ; in It is a path For nodes The contribution weight satisfies The weights can be determined based on the actual business traffic of the path, the path bandwidth, or the path reliability; ultimately, the weights are obtained. :node In the window The communication state synthesis vector.

[0061] S340, Normalization steps for communication status assessment indicators: Since the dimensions of indicators in each dimension are different, a unified scale is required.

[0062] set up: :node In the window The dth indicator; : The normalized indicator.

[0063] Linear normalization is used: ; in ; Normalized node-level evaluation vector; This step ensures that different dimensions are comparable during weight optimization.

[0064] S350. Evaluation index weight optimization steps based on ant colony algorithm: To enhance the reliability of communication status evaluation, the optimal weights are assigned to each evaluation dimension.

[0065] Let: Weight vector: ; Where D is the dimension of the evaluation vector (D=7 in this example). This represents the weight of the d-th indicator; Weight constraints: ; for each candidate weight combination distribute: : Pheromon concentration (representing weighted combination) (priority) Heuristic information (based on historical assessment accuracy, such as fault detection rate).

[0066] The probability formula for an ant choosing a weighted combination: The m-th ant chooses a weighted combination The probability is: ; in It is a parameter related to the importance of pheromones; It is a parameter for the importance of heuristic information.

[0067] Pheromone Update: After the iteration ends, update the pheromones of the weighted combinations. ; in: It is the pheromone evaporation coefficient; It is an increase in pheromones.

[0068] set up Weighted combination The accuracy rate of the assessment in the previous round (e.g., the rate at which the prediction matches the actual risk level) can then be defined as: ;in Weight optimization gain constant.

[0069] Initialization and Stopping Conditions: Initialization: ;in The initial pheromone concentration.

[0070] Stopping condition: The iteration ends when any of the following conditions are met: The number of iterations reached ;continuous The optimal weight combination remained unchanged in the next iteration.

[0071] The optimal weight vector is finally obtained: .

[0072] S4 specifically includes the following sub-steps: S410, Communication Risk Classification Rule Construction Steps: Based on the normalized node-level evaluation vector obtained in the previous stage: ; in ; : The d-th normalized evaluation index; D: The number of evaluation dimensions (consistent with S310–S350).

[0073] Define a node communication risk scoring function: ; in :node Communication risk score; The d-th evaluation metric weight (i.e., the optimal weight vector) obtained by ant colony optimization in S350. (the dth dimension).

[0074] Set risk level thresholds: : Upper limit of low-risk threshold (meaning: the highest risk score that is judged as low risk); : Lower limit of the high-risk threshold (meaning: the lowest risk score required to classify a situation as high-risk); meets the following requirements .

[0075] Based on the relationship between risk scores and thresholds, risk levels are divided into: Low risk (L0): ; Medium risk (L1): ; High risk (L2): ; This step establishes a unified set of rules for determining communication risk levels.

[0076] S420, Node Communication Risk Label Generation Steps: Based on the risk score in S410 With threshold , For each intelligent computing node Generate its communication risk label ,in: The node is at a low risk level (communication is stable). The node is at a medium-risk level (potential volatility exists). The node is at a high risk level (poor communication quality).

[0077] Risk Label This will serve as a key parameter for subsequent "availability metric calculation" and "task scheduling linkage".

[0078] S430, Computing Power Task Communication Sensitivity Matching Steps: For each task to be scheduled... Extract task communication and computing power requirements, and define a task communication sensitivity vector: ; ; :Task Priority (the larger the value, the more important); :Task Minimum computing resources required (such as the number of GPUs and video memory usage).

[0079] This vector is used to measure the sensitivity of a task to communication metrics, providing input parameters for availability metrics.

[0080] S440, Node computing power availability metric calculation steps: Combine the node communication risk label with the node's remaining computing power resources to calculate the "communication status awareness computing power availability metric" for each node.

[0081] set up: node Current remaining computing resources (such as the number of idle GPUs and available video memory); Node communication risk labels; node The computing power availability metric.

[0082] Constructing availability metrics: ; in Availability calculation weight parameters (representing the weights of computing resources and communication risks, respectively); The computing power resource utility function, which satisfies the condition that the more resources available, the larger the output, can be defined as: ; Risk penalty function, used to reflect that the higher the communication risk, the lower the availability, can be defined as: ; Finally obtained : A node-level computing power availability quantification indicator that can be directly used for task scheduling.

[0083] S450. Dynamic optimization steps for availability metrics based on ant colony algorithm: To ensure the continuous accuracy of availability metrics in complex and dynamic network environments, the availability metrics are dynamically optimized using the ant colony algorithm.

[0084] For each node set up: Node availability pheromone concentration (indicating the tendency of a node to be a task execution location); Node heuristics (prior expectations used to measure a node’s historical performance and availability).

[0085] The heuristic information is set as follows: ; in This is the current availability metric for the node.

[0086] The probability of an ant choosing a node: the probability of the m-th ant choosing a node. The probability is: ; in : Pheromone importance coefficient; Heuristic information importance coefficient.

[0087] Pheromones update rule: Update node pheromones after each iteration: ; in : Pheromones evaporation coefficient (representing the proportion of pheromone decay); node The increase in pheromones.

[0088] To ensure executability, the increment is defined as: ; in: : Optimize the gain constant; : Communication stability score of the node's most recent task execution, for example: ; in The deviation between actual and predicted latency; The deviation between the actual packet loss rate and the estimated value; It is the weighting coefficient of the two deviations.

[0089] Initialization and Stopping Conditions: Initialization: Set the initial pheromone concentration for each node: : in It is a positive constant.

[0090] Stopping condition: The iteration ends when any of the following conditions are met: The number of iterations reached continuous The optimal node composition remained unchanged during the round of iterations.

[0091] Finally, we obtain the dynamically optimized set of availability metrics. Used for task scheduling.

[0092] S5 specifically includes the following sub-steps: S510, Task Requirement Analysis Steps: Set of Tasks to be Scheduled Each task The computing power requirements, communication requirements, and latency constraints are analyzed.

[0093] For each task Define its communication and computing power requirements vector: ; in :Task Minimum bandwidth required; The maximum tolerable latency for the task; The maximum packet loss rate that the task can tolerate; Minimum computing power requirements for the task; Task priority (the higher the value, the more important the task).

[0094] Task requirement vector It will be used to filter and sort nodes.

[0095] S520, Candidate Node Selection Steps: Node availability metrics obtained from S440–S450 above Combined with task requirement vector The task is obtained by filtering the set of intelligent computing nodes V. The set of candidate nodes: , in: :Task The set of candidate nodes; :Task The minimum node availability threshold; if a node meets the communication metrics, computing resources, and priority requirements, it can enter the candidate set. .

[0096] S530, Steps for calculating candidate node scheduling preference score: For each candidate node Calculate scheduling preference score This score is used to measure the node's contribution to the task. The degree of compatibility.

[0097] Define the scheduling preference scoring function: ; in: :node The final availability metric; Node estimated latency (lowest possible path latency); Node estimation of packet loss rate; Task priority; Scheduling preference weight parameters (satisfying) The higher the score, the more suitable the node is for performing tasks.

[0098] S540, Scheduling order generation step: Based on the scheduling preference score from the previous step, generate a sorted sequence of candidate nodes: , in :Task The sequence of scheduling candidate nodes (by (Sorted from high to low); this sequence serves as the initial path for further optimization of the ant colony algorithm.

[0099] S550. Task scheduling path optimization steps based on ant colony algorithm: Treat task scheduling as an optimization problem of "node path selection", and use the ant colony algorithm to optimize the candidate node sequence. Perform in-depth optimization.

[0100] Pheromones and heuristic information definitions: for candidate nodes Set two types of quantities: Pheromones: ; indicates a node The degree of historical preference in scheduling optimization.

[0101] Heuristic information: The scheduling preference score in S530.

[0102] The probabilistic model for ants choosing nodes: the m-th ant is the task... Select node The probability is: ; in : Pheromone importance coefficient; Heuristic information importance coefficient.

[0103] Task scheduling path quality function: Defines path quality (i.e., node selection quality): ; in: It is a path quality weight parameter that satisfies The larger this function is, the better the quality of the scheduling path.

[0104] Pheromone update rule: After each iteration, update the scheduling pheromone for all nodes: ; in : Pheromones volatile coefficient; : Schedule pheromone increment.

[0105] In order for the algorithm to be executable, let: ; in: : Scheduling pheromone gain constant.

[0106] Initialization and stopping conditions: Initialization: Set the concentration of all scheduled pheromones to: It is a preset positive number.

[0107] Stopping condition: The iteration ends when any one of the following conditions is met: the maximum number of iterations has been reached. ;continuous The optimal scheduling node remained unchanged in the next iteration.

[0108] Optimal Node Decision Output: The final optimal node selection is: ; The node with the highest pheromone concentration is used as the task. The optimal scheduling position.

[0109] S6 specifically includes the following sub-steps: S610, Communication Monitoring Result Output Step: Output the final set of node-level computing power availability metrics generated in the preceding steps: ; And the optimal task scheduling decision obtained based on the ant colony algorithm: The output is sent to the scheduling system control terminal and the visual monitoring interface.

[0110] in: :node Final availability metric; :Task The system will identify the optimal scheduling node and display the availability, communication risk level, and scheduling results of each node in real time.

[0111] S620, Task Execution Feedback Collection Steps: In the task... Assign to the optimal node Then, real-time task execution feedback is collected and defined as: ; in: Actual task delay; Actual packet loss rate of the task; Real-time jitter during task execution; : Task success rate (whether it was completed on time, whether any anomalies occurred); this feedback vector is used to check the accuracy of the model's predictions and the reliability of the scheduling decisions.

[0112] S630, Parameter Adaptive Adjustment Steps: Compare and evaluate the predicted values ​​with the actual feedback values, and construct the deviation vector: ; in: : The predicted index value used to calculate scheduling preferences in the S530 phase; : The actual deviation quantization vector after the node executes the task.

[0113] Based on the deviation vector, the system parameters are adaptively updated: Risk threshold adaptive update: ; ; in The average value of the deviation vector; Threshold adjustment coefficient; ensures that the risk threshold is updated adaptively as the actual communication environment changes.

[0114] Scheduling preference parameter update: Adaptive update of scheduling weight parameters: ; in : Scheduling preference weights defined in S530; Weighted adjustment gain coefficient.

[0115] Availability metric function update: Update risk penalty function parameters: ; in To adaptively update the coefficients; this process ensures that the availability metric model gradually approximates the actual performance; the deviation vector in this step This is a three-dimensional vector containing delay bias, packet loss bias, and jitter bias. During threshold updates, its "average bias value" is used in the calculation; that is, the arithmetic mean of the absolute values ​​of each component of the three-dimensional bias vector is calculated, and this average value is used as a scalar input for updating. This ensures that the update formula is mathematically computable and logically consistent.

[0116] S640, Topology Update Triggering Steps: When the communication status becomes abnormal or the topology changes (such as a sudden drop in link quality, node offline, or sudden bandwidth change), a topology update is triggered.

[0117] Define link state change quantities: ; in: :link The benchmark indicators; Link status deviation index.

[0118] If the following conditions are met: This triggers a topology update, where... The threshold for triggering topology updates.

[0119] The topology update process will re-execute the following steps: re-evaluate link metrics (corresponding to S210–S250); recalculate path status (corresponding to S320); and re-optimize the path (corresponding to S150).

[0120] Finally, a new optimized topology is generated. .

[0121] S650. Global pheromone update steps based on ant colony algorithm: In order to achieve closed-loop optimization of monitoring-evaluation-scheduling-feedback, all pheromones are updated globally.

[0122] Define three levels of pheromone sets: Path pheromones (from S150): ; Sampled pheromones (from S250): ; Scheduling pheromones (from S550): ; Construct a unified pheromone vector: ; Global pheromone update formula: ; in Global pheromone evaporation coefficient; : Global pheromone increment.

[0123] Increments are built based on communication status and task feedback: ; in : Global gain constant; System stability (the smaller the deviation, the larger the increment).

[0124] Ultimately, the system achieves the following in actual operation: positive pheromone feedback strengthens stable links and high-performance nodes; suppresses fluctuating links and low-reliability nodes; and adaptive dynamic scheduling combined with self-learning optimization.

[0125] Example 2: This example provides an electronic device, which includes a memory and a processor. The processor is used to execute program instructions stored in the memory to implement the steps in the communication status monitoring method based on intelligent computing cluster.

[0126] This embodiment also provides a computer-readable storage medium storing program instructions that can be executed by a processor to implement the steps in the communication status monitoring method based on intelligent computing clusters.

[0127] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0128] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0129] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for monitoring the state of communication based on a cluster of intelligent algorithms, characterized by, Comprise the following steps: S1, configure a plurality of intelligent calculation nodes and corresponding network interfaces in the intelligent calculation cluster, obtain the connection relationship between the intelligent calculation nodes and the external communication network, build a communication topology model, and identify the mapping relationship between each communication path and the corresponding intelligent calculation node in the communication topology model; S2, continuously collect communication state monitoring data for each communication path and the network interface of each intelligent calculation node in the intelligent calculation cluster, store the communication state monitoring data according to the communication path and intelligent calculation node mapping relationship, and obtain a communication state monitoring data set; S3, statistical analysis is performed on each communication path within a preset time window to obtain a communication state evaluation index of the communication path; according to the communication path and intelligent calculation node mapping relationship, the communication state evaluation indexes of multiple communication paths are aggregated to the corresponding intelligent calculation node to obtain an aggregated communication state evaluation index; S4, the communication risk of each intelligent calculation node is graded, and a corresponding communication state perception algorithm power availability measurement value is generated; the intelligent calculation node with good communication state is given a higher algorithm power availability measurement value, and the intelligent calculation node with fluctuating communication state is given a lower algorithm power availability measurement value, thereby forming a communication state perception algorithm power availability measurement result; S5, the matching evaluation of the to-be-executed intelligent calculation task is performed, the communication state perception scheduling suggestion information is generated, the intelligent calculation node with better communication state and higher algorithm power availability measurement value has higher priority in the scheduling suggestion information, and the communication state monitoring result is explicitly mapped to the task scheduling linkage information that can be used. 2.The method according to claim 1, wherein, Further comprising S6, outputting the communication state perception algorithm power availability measurement result and the communication state perception scheduling suggestion information to an operation and maintenance interface or an intelligent calculation task scheduling module; according to the feedback communication abnormal event and the task completion quality, the calculation parameters of the communication state evaluation index and the demarcation threshold of the communication risk grading are adaptively adjusted, and the communication topology model and the communication state monitoring data set collection strategy are updated. 3.The method of claim 1, wherein, S1 specifically comprises: Obtain the processing capacity, network interface type and link connection information of each intelligent calculation node in the intelligent calculation cluster to provide basic node data for communication topology construction; Analyze the link structure between the intelligent calculation nodes and the external communication network, extract the link bandwidth, link structure hierarchical relationship and link direction information, and form a link basic data set; Based on the node data and link data, the intelligent calculation node is taken as a topology node and the communication link is taken as a topology edge to construct an initial version of the communication topology model; In the communication topology model, all possible communication paths are enumerated using a depth-first search, and a mapping relationship between each communication path and the corresponding intelligent calculation node is established to form a communication path candidate set; An ant colony algorithm is used to optimize the path quality of the communication path candidate set, an information function is constructed according to the link length, bandwidth stability and historical availability, and an optimized communication topology model is generated. 4.The method of claim 1, wherein, S2 specifically comprises: Determine the time delay, bandwidth utilization rate, packet loss rate, jitter degree and retransmission ratio core monitoring indexes for communication state monitoring, and establish an index collection template; Based on the communication topology model, data collection is performed on the network interface of each intelligent calculation node to obtain node-level communication state monitoring data; Based on the communication path set, real-time monitoring data of each communication path is collected, including time series data of the indicators; According to the mapping relationship between the communication path and the intelligent calculation node, the node-level monitoring data and the path-level monitoring data are unified and collected to generate a communication state monitoring data set; Using an ant colony algorithm, the data uncertainty of each link is evaluated from the communication state monitoring data set, and the monitoring sampling frequency and sampling point distribution are dynamically adjusted through the pheromone feedback mechanism, so that the collection resources are preferentially invested in the communication state fluctuation area. 5.The method of claim 1, wherein, S3 specifically includes: Based on the preset second-level, minute-level and hour-level time windows, the communication state monitoring data set is segmented and arranged to perform multi-scale evaluation; For each communication path, the average delay, jitter variance, packet loss ratio, bandwidth utilization rate change rate and retransmission ratio change rate are calculated within the multi-scale time window to obtain the path-level communication state evaluation indicators; According to the mapping relationship between the communication path and the intelligent calculation node, the evaluation indicators of multiple communication paths are aggregated by node to obtain the intelligent calculation node-level communication state evaluation indicators; The communication state evaluation indicators are subjected to scale normalization processing to ensure that different dimension indicators can be compared under a unified evaluation system; An ant colony algorithm is used to dynamically optimize the weights of each monitoring indicator to improve the sensitivity to key indicators through the pheromone gain or decay mechanism. 6.The method of claim 1, wherein, S4 specifically includes: According to the communication state evaluation indicators, a communication risk grading rule is constructed to define the threshold intervals of low risk, medium risk and high risk; Based on the communication risk grading rule, the communication risk of each intelligent calculation node is analyzed to generate a node-level communication risk label; The sensitivity of the bandwidth demand, time delay tolerance and task priority of the target intelligent calculation task is analyzed, and a mapping relationship between the communication risk label and the task sensitivity is established; Based on the communication risk label and the task sensitivity, a communication state perception computing power availability measurement value is generated for each intelligent calculation node; Through the ant colony algorithm, the node pheromone concentration is updated according to the historical task execution effect to dynamically adjust the availability measurement value of the communication stable node and the availability measurement value of the communication fluctuation node, so that the availability measurement is closer to the real network performance. 7.The method of claim 1, wherein, S5 specifically includes: The communication demand, computing power demand and real-time demand of the to-be-executed intelligent calculation task are analyzed to form a task demand vector; According to the communication state perception computing power availability measurement value, a candidate intelligent calculation node set that meets the minimum communication requirement is selected from all intelligent calculation nodes; For each candidate node, a node scheduling preference score is calculated according to its availability measurement value, task demand vector and historical task completion rate; The candidate nodes are sorted according to the scheduling preference score to generate a communication state perception task scheduling candidate sequence; An ant colony algorithm is used to optimize the scheduling path of the candidate node sequence to find the optimal task allocation path through the joint mechanism of pheromone and heuristic function. 8.The method of claim 2, wherein, S6 specifically includes: The communication state perception computing power availability measurement value and the task scheduling candidate sequence are output to the scheduling system and the operation and maintenance interface; During the execution of the target intelligent calculation task, communication abnormal events, task delay deviation and task completion quality feedback information are collected; Based on the task execution feedback, the communication risk grading parameters, the scheduling preference calculation parameters, and the evaluation index weight parameters are adaptively adjusted; When a link change, a link quality sudden drop, or a node state transition is detected in the communication topology, a topology update process is triggered to ensure the timeliness of the topology model; The global pheromone distribution is adjusted according to the task execution feedback and the topology update result by using an ant colony algorithm, so that the monitoring, evaluation, scheduling, and feedback form a positive closed-loop optimization, and the stability of the communication state monitoring of the whole intelligent algorithm cluster is improved.

9. An electronic device, comprising: The electronic device includes a memory and a processor, and the processor is configured to execute program instructions stored in the memory to implement the steps in the communication state monitoring method based on the intelligent algorithm cluster according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores program instructions, and the program instructions can be executed by the processor to implement the steps in the communication state monitoring method based on the intelligent algorithm cluster according to any one of claims 1-8.

Images