Distributed system performance evaluation method for deep learning workloads in CENI networks

Abstract

The present invention discloses a distributed system performance evaluation method for deep learning loads in a CENI network, provides a user-configurable customized deep learning load configuration method, supports users to make targeted adjustments and optimizations to the test load configuration according to the system usage scenario; implements a deep learning load deployment and testing framework, supports efficient compression and automatic distribution of deep learning test images, and adaptive generation of gradual loads and sudden loads; based on the basic monitoring indicators of the system, comprehensively evaluates the distributed system from multiple dimensions to obtain the performance of the system in the face of the set deep learning tasks and loads. The present invention can achieve distributed system performance evaluation that is highly adaptable to deep learning computing scenarios from three aspects: customized deep learning test load setting, deep learning load deployment and testing, and comprehensive evaluation of distributed system performance.

Description

Technical Field

[0001] The present invention belongs to the field of cloud computing and edge computing, and specifically relates to a distributed system performance evaluation method for deep learning loads in a CENI network. Background Art

[0002] With the continuous advancement of information technology, distributed systems have become an essential component of modern computing architectures. By distributing computing tasks across multiple nodes, distributed systems not only improve computing efficiency and system scalability, but also enhance fault tolerance and reliability. Distributed systems are now widely used in cloud computing, the Internet of Things, big data processing, edge computing, and other fields. These systems involve large numbers of computing nodes and complex resource scheduling, placing higher demands on system performance and resource management.

[0003] The Future Network Experimental Facility (CENI) is my country's first major national scientific and technological infrastructure in the communications and information sector, providing users with low-latency, low-jitter, high-throughput, and highly reliable network transmission services. By deploying high-performance network nodes and fiber-optic backbone links nationwide, the CENI network boasts flexible resource scheduling and advanced network virtualization capabilities, enabling efficient support for a wide range of innovative application workloads, including artificial intelligence, deep learning, cloud computing, and big data analytics. The CENI network enables the construction of cross-regional, high-throughput distributed computing systems, eliminating network transmission bottlenecks between remote computing nodes and significantly improving the collaborative computing efficiency of distributed applications. It also provides critical network support for the connection, operation, and expansion of remote computing nodes.

[0004] Performance testing and load testing are essential components of distributed system development and maintenance. However, existing load testing techniques have limitations, especially when dealing with modern, complex distributed systems. Traditional load testing methods typically evaluate system performance by simulating large numbers of user requests, but these methods often fail to truly reflect the system's performance under high concurrency, complex data transmission, and computational demands. For example, in scenarios requiring large-scale data processing and complex computations, traditional load testing often lacks sufficient testing depth and accuracy, making it difficult to accurately identify potential bottlenecks in the system.

[0005] With the rapid development of deep learning technology, deep learning models have demonstrated powerful application capabilities in a wide range of fields. From image recognition and speech processing to natural language processing and recommendation systems, deep learning has become an important tool for solving complex problems. The training and inference processes of deep learning models typically require extensive computing resources. These computing tasks often involve high concurrency, high data transmission, and computational complexity, placing high demands on the system's computing and communication performance.

[0006] In distributed systems such as cloud computing and edge computing, containerization has become a common deployment method. Container technology allows applications to be packaged together with their dependent environments, ensuring consistent operation in any environment. A container image typically contains the application itself, along with all necessary dependencies and libraries, making it an ideal way to deploy and run applications across multiple environments. However, as distributed systems continue to scale, container image distribution issues are becoming increasingly prominent. In edge computing and large-scale distributed systems, the transmission and distribution of container images are often limited by network bandwidth and storage space, resulting in slow distribution and impacting system deployment efficiency. Improving the efficiency of container image distribution, particularly between edge nodes, has become a major technical challenge in distributed systems.

[0007] Furthermore, load changes in distributed systems are often unpredictable, especially under high load and high concurrency conditions. Therefore, effectively responding to fluctuations in system load becomes crucial for system expansion. Traditional expansion mechanisms primarily include proactive expansion and responsive expansion. Proactive expansion relies on load forecasting, predicting load changes based on historical data or trends and increasing computing resources in advance. While this approach ensures that the system prepares resources before the load increases, its drawback is that inaccurate forecasts can lead to wasted resources. Responsive expansion initiates expansion operations when the load actually reaches a certain threshold. However, due to the long response time of expansion, the system may not be able to respond to load changes in a timely manner, resulting in performance fluctuations and other issues. Summary of the Invention

[0008] The purpose of this invention is to propose a distributed system performance evaluation method for deep learning loads in CENI networks. The method customizes deep learning tests according to user usage scenarios, provides efficient compression and automatic distribution of test images, and the ability to adaptively generate gradual and sudden loads. Based on the basic system monitoring indicators, the method comprehensively evaluates the distributed system from multiple dimensions such as task completion, resource allocation, and load balancing.

[0009] The technical solution for achieving the purpose of the present invention is:

[0010] A distributed system performance evaluation method for deep learning workloads in a CENI network includes the following steps:

[0011] S1: Set up a deep learning model file, parse the deep learning model file, extract the deep learning model name and version information, select the corresponding deep learning model on the cloud, and build an image based on the extracted deep learning model name and version information;

[0012] S2: Dynamically delete and hierarchically compress the image, and distribute the compressed image;

[0013] S3: Generates corresponding loads based on different evaluation requirements;

[0014] S4: Collect indicator data after load generation and conduct a comprehensive evaluation of distributed system performance based on the indicator data.

[0015] Preferably, setting the deep learning model file specifically includes setting the model name, model version, running rounds, number of samples processed at a single time, sample data set, load type, cycle time, initial load quantity, load increment quantity, burst load quantity, burst load occurrence time, total test time, and number of nodes.

[0016] Preferably, the types of loads include periodic loads, gradual loads and sudden loads; the periodic load refers to a load that repeatedly runs the same number of times at a specified time interval; the gradual load refers to a load that repeatedly runs a gradually increasing amount of data at a specified time interval; the sudden load refers to a load that runs a sample number and a processing number that increases by more than a set threshold within a limited time interval.

[0017] Preferably, the specific method for dynamic deletion and hierarchical compression of the image is:

[0018] Start a temporary container for the image and monitor file access in real time during the temporary container's operation; after the temporary container ends, package the accessed files into a new image;

[0019] Merge all image layers in the new image and use the final merged file view as the only image layer of the new image;

[0020] The merged new image is divided into logical blocks that can be downloaded in parallel and distributed to multiple points synchronously.

[0021] Preferably, the specific method of dividing the merged new image into logical blocks that can be downloaded in parallel and performing multi-point synchronous distribution is:

[0022] Pre-caching image layers or data blocks on distributed system nodes;

[0023] When there is a high-speed interconnection link between the nodes of the distributed system, the node that has completed the download will share the image blocks with other nodes.

[0024] Preferably, according to different evaluation requirements, the specific method of generating corresponding load is:

[0025] Periodic load generation: A timer is set in the distributed system to start a specified number of new Pods. Containers are started in these new Pods, which run the test load of a specified deep learning model. Whenever a new time period arrives, the load that was not completed in the previous period continues to execute, without stopping. The scheduler distributes the newly started Pods with the specified number of loads to the distributed system nodes. When the distributed system node task is completed, the Pod is destroyed, freeing up the occupied computing resources. Subsequent test loads use the released computing resources.

[0026] Gradual load generation: A timer is set in the distributed system to start a new Pod with a specified initial load. Containers are then started in the new Pods, which run the test load of a specified deep learning model. At specified intervals, the number of newly started deep learning model test loads gradually increases, and the load increases by the specified incremental load compared to the previous interval. Simultaneously, the scheduler selects nodes to run the newly added deep learning load tasks based on node resource usage.

[0027] Burst load generation: A timer is set in the distributed system to launch a specified number of new pods with an initial load. These pods are then used to start containers running a test workload using a specified deep learning model. New pods with a specified initial load are launched at specified intervals. When the evaluation reaches the specified burst load time, the specified number of new pods are launched all at once.

[0028] Compared with the prior art, the present invention has the following significant advantages:

[0029] (1) The deep learning load deployment and testing framework designed by the present invention can support the efficient compression and automatic distribution of deep learning test images, as well as the adaptive generation of gradual loads and sudden loads. It breaks through the limitations of traditional distributed system testing, such as the difficulty of test program deployment and the single form of test requests, and is highly original.

[0030] (2) The comprehensive evaluation index of distributed system performance proposed in this invention can comprehensively evaluate the distributed system from multiple dimensions such as task completion, resource allocation, and load balancing on the basis of the basic monitoring indicators of the system. It solves the limitations of a single evaluation index in judging the performance of cloud-edge hybrid distributed computing systems and is highly innovative.

[0031] (3) The user-configurable customized deep learning load generation method adopted by the present invention supports users to make targeted adjustments and optimizations to the test load configuration according to the system usage scenario, significantly improving the matching degree between the test environment and the generation environment, and provides a method for combining user-defined configuration with system default configuration, which simplifies the difficulty of user configuration and has high practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of the distributed system performance evaluation method for deep learning loads in the CENI network of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0034] like Figure 1 As shown, as an embodiment, a distributed system performance evaluation method for deep learning load in a CENI network includes the following steps:

[0035] Step S1: Customize deep learning test load settings:

[0036] Users set up deep learning model files based on load requirements, including the required model name, model version, run rounds, number of samples processed at a time, sample dataset, load type, cycle time, initial load quantity, load increment quantity, burst load quantity, burst load occurrence time, total test time, and number of nodes. A distributed system node refers to an independent computing unit or device in a distributed system. Each node can operate independently and communicate and collaborate with other nodes through the network to jointly complete system tasks.

[0037] In a further embodiment, the load type is determined according to the test content, specifically:

[0038] In order to test the stability of the system in a normal scenario, a periodic load is selected;

[0039] To test the system's elastic expansion and contraction capabilities, select a gradual load change.

[0040] In order to test the system's response efficiency and fault recovery level when facing instantaneous high concurrency or resource preemption, burst load is selected.

[0041] In a further embodiment, the load types include periodic load, gradually changing load and sudden load.

[0042] Periodic load: Repeatedly runs the same amount of load at specified time intervals;

[0043] Gradual load: Repeatedly run a load with gradually increasing data volume at specified time intervals;

[0044] Bursty load: A load that runs with a sharp increase in the number of samples and processing times within a short time interval.

[0045] Parse the deep learning model file, extract the deep learning model name and version information, select the corresponding deep learning model in the cloud, and build an image based on the extracted deep learning model name and version information. The load setting file is described in JSON format.

[0046] S2: Efficient compression and fast distribution of payload images;

[0047] Traditional image builds often package the entire Linux distribution and all its dependencies, resulting in bulky image files and time-consuming distribution and deployment. To effectively address this issue, this paper proposes a triple reduction and transmission solution for large workloads such as deep learning models: a combination of "dynamic analysis and pruning," multi-layer image merging, and distributed rapid distribution. This solution not only accurately eliminates unused image dependencies but also enables rapid image distribution to distributed computing nodes, significantly improving overall system transmission and startup efficiency.

[0048] First, the present invention accurately identifies the actual necessary files through the "dynamic analysis and tailoring" method. Specifically, a temporary container is started and the access to files during the operation of the temporary container is monitored in real time; after the temporary container ends, only the accessed files are packaged into the new image, thereby removing redundant content that has never been called or is only used for testing purposes. Compared with traditional solutions that rely on static lists or manually configured black and white lists, dynamic analysis can more accurately capture the library files, model parameters and data dependencies required for deep learning reasoning or training, thereby improving the accuracy of image tailoring.

[0049] Secondly, the present invention implements container multi-layer storage and a copy-on-write mechanism through a "mirror multi-layer merging" method, merging new mirror layers into a unique layer, thereby further cleaning up redundant information caused by modifying or deleting lower-layer files. Specifically, since the container's modification of lower-layer files does not actually change the underlying files, but instead generates incremental layers or placeholders to mark deletions, multi-layer stacking will generate a large amount of duplicate data. By unifying and merging into a single layer, redundant files can be effectively eliminated and the final consistent file view can be retained, significantly reducing the image size.

[0050] Finally, to further shorten image distribution time in a distributed environment, the present invention also proposes a multi-node "distributed fast distribution" method. This method can divide the merged new image into logical blocks that can be downloaded in parallel and supports multi-point synchronization in the distribution network. Specifically, it includes the following two aspects: first, layered or block pre-pull, that is, pre-caching some commonly used image layers or key data blocks on the distributed system nodes to achieve "local pull to start" rapid parallel deployment; second, peer-to-peer transmission (P2P) or hierarchical relay: When there is a high-speed interconnection link between the nodes of the distributed system, the node that has completed the download can share the image blocks with other nodes, reducing dependence on the central image repository and reducing bandwidth and request pressure.

[0051] Through the above three-pronged solution, the present invention not only significantly compresses the file size during the image building stage, but also accelerates distribution by utilizing multi-point parallel and peer-to-peer transmission in the distribution link, greatly reducing resource usage and time overhead during distributed deployment.

[0052] Step S3, customized generation of multiple types of evaluation loads;

[0053] During the actual operation of distributed systems, they often face diverse task scenarios such as real-time inference and batch training. The differentiation of task fields will lead to significant differences in load scale, concurrency characteristics, and resource requirements. Therefore, customized generation of multiple types of evaluation loads is particularly critical for the comprehensive evaluation of distributed systems.

[0054] By allowing users to customize and flexibly configure deep learning models, load generation timing, and volume, we can more closely align with real-world business needs, thereby comprehensively evaluating the adaptability of the system's scheduling algorithms, resource management strategies, and fault-tolerance mechanisms under different load profiles. Periodic loads can verify the system's stability under typical scenarios, while gradual loads measure its elastic scaling capabilities. Burst loads are used to evaluate the system's response efficiency and fault recovery capabilities when faced with transient high concurrency or resource preemption. The combination of multiple load types and monitoring feedback provides a reliable basis for system optimization and upgrades, comprehensively improving the stability and high availability of distributed environments.

[0055] (1) Periodic load generation

[0056] The characteristic of a periodic load is that it repeatedly initiates the same amount of load at a specified time period within the distributed system being evaluated. The cycle time (cycle_time), initial load quantity (initial_load), and test load (model_name, model_version) are set by the system evaluation user in step S1.

[0057] Specifically, at the beginning of the evaluation, a timer is set in the distributed system to start a new Pod with a specified load number, and a container is started in the new Pod, in which the test load of the specified deep learning model is running. Whenever a new time period is reached, on the one hand, the load that was not completed in the previous period will not stop running, but will continue to execute. On the other hand, the scheduler will distribute the newly started Pod with a specified load number to the appropriate distributed system node. When the distributed system node task is completed, the Pod will be destroyed, releasing the occupied computing resources, and subsequent test loads will use the released computing resources. Among them, Pod is the smallest deployable computing unit in the distributed container management platform Kubernetes, which contains one or more containers that share network and storage resources. Through such a load generation mechanism, the performance of the distributed system under periodic stable load conditions can be evaluated.

[0058] (2) Gradual load generation

[0059] The characteristic of a gradual load is that the deep learning computation load launched to the distributed system under evaluation gradually increases at specified intervals. The cycle time (cycle_time), initial load (initial_load), incremental load (load_increment), and test load (model_name, model_version) are set by the system evaluation user in step S1.

[0060] Specifically, at the start of the evaluation, a timer is set in the distributed system to launch a specified number of new pods with an initial load. Containers are then started within these pods, running the test workload of a specified deep learning model. Subsequently, at specified intervals, the number of newly launched deep learning workloads increases gradually, with the specified incremental load increase compared to the previous interval. Simultaneously, the system scheduler selects appropriate nodes to run the newly added deep learning workload tasks based on node resource usage. This load generation mechanism allows the performance of the distributed system to be evaluated under increasing load pressure.

[0061] (3) Burst load generation

[0062] The characteristic of burst load is that, in the distributed system being evaluated, a large amount of deep learning computing load is launched at a specified time point based on the periodic load. The cycle time (cycle_time), initial load amount (initial_load), burst load amount (burst_load), burst load occurrence time (burst_time), and test load (model_name, model_version) are set by the system evaluation user in step S1.

[0063] Specifically, at the start of the evaluation, a timer is set in the distributed system to launch a new Pod with a specified initial load. Containers are then started in these new Pods, running the test workload of a specified deep learning model. Subsequently, new Pods with a specified initial load are launched to the system at specified intervals. Furthermore, when the system evaluation time reaches the specified burst load occurrence time, a new Pod with a specified burst load is launched all at once. This load generation mechanism allows the performance of the distributed system to be evaluated under large burst loads.

[0064] Step S4: Comprehensive evaluation of distributed system performance

[0065] Step S401: Obtaining basic indicators of the distributed system

[0066] This step defines and designs methods for obtaining seven basic indicators of distributed systems: task response time, completion time, node resource utilization, number of node tasks, number of successfully run tasks, total number of tasks, and total number of system nodes. This allows for comprehensive monitoring of task and node performance during distributed system evaluation, providing support for subsequent system comprehensive indicator analysis and optimization suggestion generation. The specific basic indicator symbols, meanings, and acquisition methods are as follows:

[0067]

[0068] Step S402: Distributed system comprehensive index analysis and optimization suggestion generation

[0069] The purpose of comprehensive indicator analysis and optimization suggestion generation for distributed systems is to fully understand the system operation status and locate potential bottlenecks. This step is based on the distributed system foundation obtained in step S401, and further calculates seven comprehensive indicators of the distributed system, including system response time variance, task completion time variance, node resource utilization balance, load task completion rate, load task completion consistency, cluster load balancing index, and load task balancing index, to comprehensively evaluate system performance and stability. Based on the analysis results, flexible and efficient optimization solutions are proposed for possible problems such as communication delays, node performance differences, resource imbalances, and failure risks. This process helps system analysts quickly locate and repair deep-seated problems in distributed systems, and provides a scientific basis for system expansion, upgrades, and overall architecture design. Through continuous monitoring and feedback loops, distributed systems can continue to evolve, maintain high availability and scalability, and achieve stable, efficient, and sustainable service capabilities.

[0070] System response time variance :

[0071]

[0072] in, is the number of tasks, represents the response time of the i-th task, is the mean of the average response time of all tasks. Determine whether the system's task distribution and node response are stable under different load requirements. More than 50ms 2 , indicating significant fluctuations in response time, possibly due to unstable communication delays. System optimization recommendations for this issue include reducing communication delay fluctuations by optimizing network topology, introducing load balancing, and reducing cross-node communication. Caching and retry mechanisms are also recommended to improve response stability.

[0073] Task completion time variance :

[0074]

[0075] in, is the number of tasks, is the time required to complete the i-th task, is the average time required to complete all tasks. More than 20s 2 , there may be significant differences in node performance and unstable computing power. System optimization recommendations for this issue include using adaptive load balancing and real-time monitoring of node performance to dynamically allocate tasks and redundancy for critical tasks; and quickly migrating computing to nodes with surplus resources to ensure overall stability.

[0076] Node resource utilization balance B:

[0077]

[0078] in, is the number of nodes, is the resource utilization of the jth node, B is the average resource utilization of all nodes. If B is less than 0.8, it indicates a significant disparity in node resource utilization and an unbalanced task distribution. System optimization recommendations for this issue include adopting a resource-aware scheduling strategy that monitors real-time node load to migrate and balance tasks, and implementing moderately redundant parallel execution to reduce overload on individual nodes.

[0079] Load task completion rate CTR:

[0080]

[0081] Where T s Indicates the number of tasks that get results, T t is the total number of tasks, if If the percentage is less than 90%, there may be issues with task configuration errors, insufficient resources, or system failures. System optimization suggestions for this issue include strengthening configuration verification and automated testing, dynamically expanding capacity to address resource shortages, and establishing monitoring and failover mechanisms to promptly detect and recover abnormal nodes.

[0082] Load task completion consistency CI:

[0083]

[0084] in, is the number of tasks, is the time required to complete the i-th task, It is the average time required to complete all tasks. When the system completes tasks with the same data set and model information but different nodes, the load execution time is used as a parameter to judge the stability of the system. If the CI is lower than 0.8, it means that the consistency of task completion is poor, and there may be problems with improper dynamic distribution of resources and disordered task execution order. The system optimization suggestions generated for this problem are to use a centralized scheduler to uniformly distribute tasks, monitor the load of each node and adjust it dynamically; use priority or FIFO queues to ensure execution order consistency and reduce waiting. Through automated resource elastic expansion and contraction and node health detection, ensure reasonable resource allocation, avoid task preemption and disordered order, and improve overall execution consistency.

[0085] Cluster load balancing index (CL):

[0086]

[0087] in, is the number of nodes, is the number of tasks for the jth node, N is the average number of tasks across all nodes. This measures the uniformity of the load in a distributed cluster. If the result is greater than N / 10, it indicates an uneven distribution of tasks across nodes. System optimization recommendations for this issue include utilizing a monitoring system to obtain real-time node load information and using a load balancing algorithm to dynamically schedule and migrate tasks, preventing overload on individual nodes and improving resource utilization.

[0088] Load task balance index TLBI

[0089]

[0090] in, is the number of nodes, is the number of tasks for the jth node, is the average number of tasks for all nodes. If the value is less than 0.8 or greater than 1.2, some nodes may be overloaded or underloaded. System optimization recommendations for this issue include real-time monitoring of node load and applying a dynamic scheduling algorithm to properly migrate tasks between overloaded and underloaded nodes, maintaining overall system load balance.

Claims

1. A distributed system performance evaluation method for deep learning workloads in a CENI network, characterized by: The following steps are involved: S1: Set up a deep learning model file, parse the deep learning model file, extract the deep learning model name and version information, select the corresponding deep learning model on the cloud, and build an image based on the extracted deep learning model name and version information; S2: Dynamically delete and hierarchically compress the image, and distribute the compressed image. The specific method is as follows: Start a temporary container for the image and monitor file access in real time during the temporary container's operation; after the temporary container ends, package the accessed files into a new image; Merge all image layers in the new image and use the final merged file view as the only image layer of the new image; Divide the merged new image into logical blocks that can be downloaded in parallel and distribute them synchronously to multiple locations; S3: Generates corresponding loads based on different evaluation requirements; S4: Collect indicator data after load generation and conduct a comprehensive evaluation of distributed system performance based on the indicator data.

2. The method for evaluating distributed system performance for deep learning loads in a CENI network according to claim 1, wherein: Setting up the deep learning model file specifically includes setting the model name, model version, run rounds, number of samples processed at a single time, sample data set, load type, cycle time, initial load quantity, load increment quantity, burst load quantity, burst load occurrence time, total test time, and number of distributed system nodes.

3. The method for evaluating distributed system performance for deep learning loads in a CENI network according to claim 1, wherein: The types of loads include periodic loads, gradual loads and sudden loads; the periodic load refers to a load that repeatedly runs the same number of times at a specified time interval; the gradual load refers to a load that repeatedly runs a gradually increasing amount of data at a specified time interval; the sudden load refers to a load that runs a sample number and a processing number that increases by more than a set threshold within a limited time interval.

4. The method for evaluating distributed system performance for deep learning workloads in a CENI network according to claim 1, wherein: The specific method of dividing the merged new image into logical blocks that can be downloaded in parallel and distributing them to multiple points synchronously is as follows: Pre-caching image layers or data blocks on distributed system nodes; When there is a high-speed interconnection link between the distributed system nodes, the distributed system nodes that have completed the download are allowed to share the image blocks with other distributed system nodes.

5. The method for evaluating distributed system performance for deep learning loads in a CENI network according to claim 1, wherein: According to different evaluation requirements, the specific method of generating corresponding load is as follows: Periodic load generation: A timer is set in the distributed system to start a specified number of new Pods. Containers are started in these new Pods, which run the test load of a specified deep learning model. Whenever a new time period arrives, the load that was not completed in the previous period continues to execute, without stopping. The scheduler distributes the newly started Pods with the specified number of loads to the distributed system nodes. When the distributed system node task is completed, the Pod is destroyed, freeing up the occupied computing resources. Subsequent test loads use the released computing resources. Gradual load generation: A timer is set in the distributed system to start a new Pod with a specified initial load, and a container is started in the new Pod, which runs the test load of the specified deep learning model. At every specified time period, the number of newly started deep learning model test loads gradually increases, and the load increases by the specified incremental load compared to the previous time period. At the same time, the scheduler selects the corresponding distributed system node to run the newly added deep learning load tasks based on the distributed system node resource usage. Burst load generation: Set a timer in the distributed system to start a new Pod with a specified initial load, and start a container in the new Pod, which runs the test load of the specified deep learning model. At a specified time interval, a new Pod with a specified initial load is launched. When the evaluation time reaches the specified burst load occurrence time, a new Pod with a specified burst load is launched at once.

6. The method for evaluating distributed system performance for deep learning workloads in a CENI network according to claim 1, wherein: Comprehensively evaluate the performance of distributed systems based on indicator data, including: Determining system response time variance : ； in, is the number of tasks, represents the response time of the i-th task, is the mean of the average response time of all tasks; according to Determine whether the system's task distribution and node response are stable under different load requirements. If the first threshold is exceeded, it indicates that there is a problem of unstable communication delay; Calculating task completion time variance : ； in, is the number of tasks, is the time required to complete the i-th task, is the mean time required to complete all tasks, if If it is greater than the second threshold, there will be problems such as large differences in node performance and unstable computing power performance; Compute node resource utilization balance B: ； in, is the number of nodes, is the resource utilization of the jth node, is the mean of all node resource utilization. If B is lower than the third threshold, it indicates that there is a large gap in node resource utilization and unbalanced task distribution. Load task completion rate CTR: ； Among them, T s Indicates the number of tasks that get results, T t is the total number of tasks, if If it is less than the fourth threshold, there is a problem of task configuration error, insufficient resources, or system failure; Calculate the load task completion consistency CI: ； in, is the number of tasks, is the time required to complete the i-th task, It is the average time required to complete all tasks. When completing tasks with the same dataset and model information but different nodes, the load execution time is used as a parameter to judge the stability of the system. If the CI is lower than the fifth threshold, it indicates that there is an improper dynamic distribution of resources and a disordered task execution order. Calculate the cluster load balancing index: ； in, is the number of nodes, is the number of tasks for the jth node, is the average number of tasks across all nodes, which is used to measure the uniformity of the load in the distributed cluster. If the result is greater than N / 10, it indicates that there is an uneven distribution of tasks among the nodes. Calculate the load task balance index TLBI ； in, is the number of nodes, is the number of tasks for the jth node, is the mean number of tasks for all nodes, if If it is less than the sixth threshold or greater than the seventh threshold, it means that some nodes are overloaded or underloaded, and the sixth threshold is less than the seventh threshold.

Images