Large-scale quick downloading digital analysis method

Abstract

The invention discloses a large-scale rapid downloading digital analysis method, which relates to the technical field of digital analysis, and comprises the following steps: data collection and downloading performance benchmark test; analyzing the bandwidth utilization rate according to the performance data; performing network bottleneck identification according to the bandwidth data; performing data flow optimization scheduling based on the bottleneck information; performing node selection and load balancing according to a scheduling result; performing server performance monitoring according to a load balancing result; fault detection and repair are carried out based on the monitoring data; and generating a fault recovery report according to the recovery data. According to the large-scale fast downloading digital analysis method, the real-time performance of the dynamic data flow scheduling algorithm is improved by setting the dynamic data flow scheduling algorithm and combining the deep reinforcement learning (DRL) and the network bottleneck prediction result, and the capacity of the dynamic data flow scheduling algorithm for adapting to the complex network environment change is enhanced. In addition, the scheduling algorithm can be adaptively adjusted, and the problem that a traditional static scheduling method is not timely in response to a rapidly changing network environment is avoided.

Description

Technical Field

[0001] The invention relates to the technical field of digital analysis, in particular to a large-scale fast download digital analysis method. Background Art

[0002] In large-scale fast download systems, the efficiency of data transmission is affected by many factors, including network bandwidth, latency, packet loss rate, and traffic load. In order to improve download efficiency and avoid the impact of network bottlenecks, modern digital analysis methods use a variety of advanced algorithms and technologies. In particular, with the combination of deep learning models and dynamic data flow scheduling algorithms, these methods have shown great potential in processing large-scale data flows, predicting network bottlenecks, and achieving real-time adjustments.

[0003] In the existing technology, although the dynamic data flow scheduling algorithm can adjust the allocation of data flow based on the current network status, it often ignores the real-time and dynamic changes of network status. The existing scheduling algorithm fails to effectively integrate the prediction results from the deep learning model and lacks an adaptive adjustment mechanism, resulting in the algorithm being unable to respond quickly in a rapidly changing network environment. For example, when a network bottleneck occurs, the scheduling algorithm may not be able to adjust the bandwidth allocation strategy in time, thus affecting the overall download efficiency. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a large-scale rapid download digital analysis method to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: In a first aspect, an embodiment of the present invention provides a large-scale rapid download digital analysis method, comprising the following steps: S1,Data collection and download performance benchmark; Collect network performance data and benchmark the network to obtain performance data; S2. Analyze bandwidth utilization based on performance data; Based on the performance data, the bandwidth monitoring tool is used to analyze the network traffic in real time to obtain bandwidth data; S3, identify network bottlenecks based on bandwidth data; Based on bandwidth data, the bandwidth data is trained through a deep learning model to identify the characteristics of network bottlenecks and obtain bottleneck information; S4, optimize data flow scheduling based on bottleneck information; Based on the bottleneck information, the data flow is optimized and scheduled through the dynamic data flow scheduling algorithm to obtain the scheduling result; S5. Perform node selection and load balancing based on the scheduling results; According to the scheduling result, the server's download node is selected through the load balancing algorithm and load balancing is performed to obtain the load balancing result; S6. Monitor server performance based on load balancing results; Based on the load balancing results, monitor the performance of each server node to obtain monitoring data; S7, fault detection and repair based on monitoring data; Detect and repair the failed server nodes according to the monitoring data to obtain repair data; S8. Generate a fault recovery report based on the repair data; Visualize the repaired data for user reference.

[0006] To further optimize the technical solution, the performance data in step S1 includes: Latency, packet loss, traffic load, bandwidth.

[0007] To further optimize the technical solution, the bandwidth data in step S2 includes: The bandwidth usage of each download task.

[0008] To further optimize the technical solution, the deep learning model in step S3 includes: Model input and output: Input data vector, ; in, is the bandwidth data, To delay data, is the packet loss rate data, is the traffic load data, Network bottleneck prediction output: ; in, is the mixed density neural network function, It is the predicted bottleneck value output by the model, indicating whether there is a bottleneck in the network. Its value is between 0 and 1, including 0 and 1. If it is greater than 0.5, it means there is a bottleneck, otherwise it means the network is normal.

[0009] To further optimize the technical solution, the dynamic stream data scheduling algorithm in step S4 includes: Define the state space vector ; in, is the bandwidth data at the current time t, is the delayed data at the current time t, is the packet loss rate data at the current time t, is the traffic load data at the current time t, is the predicted bottleneck value output by the model at the current time t; Action Space Vector , ; in, represents the bandwidth allocated to the nth download task at time t, Reward function, ; in, is the download speed of the current time step, and are weight factors that control the importance of download speed and bottleneck avoidance, respectively. It is a penalty item when a bottleneck occurs and is not effectively avoided, controlling the impact of the bottleneck; At each moment t, based on the current network status and bottleneck prediction , the scheduling algorithm selects a bandwidth allocation strategy based on the state and action , this strategy is trained by the Q-value update formula, which is: ; in, is the discount factor that controls the weight of future rewards.

[0010] To further optimize the technical solution, the load balancing algorithm in step S5 includes: Polling algorithm: Assign requests to each server in sequence in a cycle to evenly distribute the load.

[0011] To further optimize the technical solution, in step S6, the server monitoring tools used for monitoring include Zabbix, Nagios, Grafana, and Beszel.

[0012] To further optimize the technical solution, the monitoring data in step S6 includes: CPU usage, memory usage, disk I / O rate, and network bandwidth usage.

[0013] To further optimize the technical solution, the detection and repair of the failed server node in step S7 includes: Fault detection system: Combined with the real-time monitoring data of the server, the faulty node is located and the system repairs the faulty node in time.

[0014] To further optimize the technical solution, the fault recovery report in step S8 includes: The node where the failure occurred, the type of failure, and the time when the failure occurred; The recovery time, performance changes, and load balancing adjustment status of each node.

[0015] In a second aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, the steps of a large-scale rapid downloading digital analysis method as described in the first aspect of the present invention are implemented.

[0016] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: the computer program instructions, when executed by a processor, implement the steps of a large-scale rapid downloading digital analysis method as described in the first aspect of the present invention.

[0017] Compared with the prior art, the present invention provides a large-scale fast download digital analysis method, which has the following beneficial effects: This large-scale fast download digital analysis method, by combining deep reinforcement learning (DRL) and network bottleneck prediction results, not only improves the real-time performance of the dynamic data flow scheduling algorithm, but also enhances its ability to adapt to changes in complex network environments. Specifically, the scheduling algorithm can automatically adjust the bandwidth allocation of data flows according to real-time network status and bottleneck prediction results through continuous learning and feedback. When the network environment changes, the scheduling algorithm can respond quickly and optimize the traffic allocation strategy to maximize bandwidth utilization and download speed. In addition, by combining with the deep learning model, the scheduling algorithm can be adaptively adjusted, avoiding the problem of traditional static scheduling methods not responding in a timely manner to rapidly changing network environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic diagram of a large-scale rapid download digital analysis method proposed by the present invention; Figure 2 A data flow optimization scheduling flow chart of a large-scale rapid download digital analysis method proposed by the present invention; Figure 3 A schematic diagram of a server monitoring tool for a large-scale rapid download digital analysis method proposed by the present invention. DETAILED DESCRIPTION

[0020] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0022] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments. Embodiment 1

[0023] Reference Figure 1 to Figure 3 , which is the first embodiment of the present invention, and which provides a large-scale rapid download digital analysis method, comprising the following steps: S1,Data collection and download performance benchmark; Collect network performance data and benchmark the network to obtain performance data.

[0024] Performance data includes: Latency, packet loss, traffic load, bandwidth.

[0025] In this embodiment, real-time monitoring and data collection are mainly performed through network performance analysis tools (such as Wireshark). By benchmarking the download performance, the maximum throughput of the current network and its bottlenecks are understood, providing data support for subsequent analysis.

[0026] S2. Analyze bandwidth utilization based on performance data; Based on the performance data, the network traffic is analyzed in real time through the bandwidth monitoring tool to obtain bandwidth data.

[0027] Bandwidth data includes: The bandwidth usage of each download task.

[0028] In this embodiment, a bandwidth monitoring tool (such as NetFlow) is used to analyze network traffic in real time to evaluate the bandwidth usage of different download tasks. If the bandwidth usage is lower than expected, there may be a network bottleneck or download scheduling problem. The purpose of this step is to identify the waste or excessive usage of bandwidth resources and provide a basis for subsequent adjustment of download strategies.

[0029] S3, identify network bottlenecks based on bandwidth data; Based on bandwidth data, the bandwidth data is trained through a deep learning model to identify the characteristics of network bottlenecks and obtain bottleneck information.

[0030] Deep learning models include: Model input and output: Input data vector, ; in, is the bandwidth data at the current time t, is the delayed data at the current time t, is the packet loss rate data at the current time t, is the traffic load data at the current time t, Network bottleneck prediction output: ; in, is the mixed density neural network function, It is the predicted bottleneck value output by the model, indicating whether there is a bottleneck in the network. Its value is between 0 and 1, including 0 and 1. If it is greater than 0.5, it means there is a bottleneck, otherwise it means the network is normal.

[0031] In this embodiment, the model is used as follows: Data collection and preprocessing: Collect real-time bandwidth, latency, packet loss rate and other multi-dimensional network data and use them as input features. The data needs to be standardized so that it is within a uniform range (such as between 0 and 1) to facilitate model training and reasoning.

[0032] Training deep learning models: Use historical data for model training, adjust the structure of CNN and LSTM networks, convolution kernel size, time step and other hyperparameters, and optimize the loss function so that the model can accurately identify network bottlenecks.

[0033] CNN network model, which is used to extract spatial features of input data, and the convolution operation extracts local features through multiple convolution kernels. Each convolution kernel learns local patterns in network traffic (such as burst traffic or burst delay).

[0034] ; in, is the convolution kernel, are different time steps of the input data, is the bias term, is the output (feature map) of the convolution operation.

[0035] This output is used to extract local characteristics of the signal such as bandwidth and delay.

[0036] Based on the features of CNN output, LSTM network captures the time dependency of data. LSTM has a memory function and can maintain the historical state of the network, thereby capturing the long-term change trend of network load.

[0037] ; ; ; ; ; ; in, They are input gate, forget gate and output gate. is the output of the convolution operation (feature map), is the output state, All weight matrices W and bias terms b are obtained through training optimization.

[0038] Online reasoning and bottleneck prediction: In actual operation, real-time network data is input into the trained model to obtain the network bottleneck prediction value. If the predicted value exceeds a certain threshold (such as 0.5), it is determined that there is a bottleneck in the network and the corresponding adjustment mechanism is triggered (such as rescheduling bandwidth, adjusting traffic distribution, etc.).

[0039] Dynamic feedback adjustment: Based on the bottleneck prediction results, the model can provide real-time feedback to adjust system parameters (such as bandwidth allocation strategy) to ensure that when a network bottleneck occurs, it can automatically and effectively adjust to reduce latency and packet loss and improve overall download efficiency.

[0040] S4, optimize data flow scheduling based on bottleneck information; Based on the bottleneck information, the data flow optimization scheduling is performed through the dynamic data flow scheduling algorithm to obtain the scheduling result. The dynamic flow data scheduling algorithm includes: Define the state space vector ; in, is the bandwidth data, To delay data, is the packet loss rate data, is the traffic load data, is the predicted bottleneck value output by the model; Action Space Vector , ; in, represents the bandwidth allocated to the nth download task at time t; Reward function, ; in, is the download speed of the current time step, and are weight factors that control the importance of download speed and bottleneck avoidance, respectively. This is a penalty term when a bottleneck occurs and is not effectively avoided, which controls the impact of the bottleneck.

[0041] At each moment t, based on the current network status and bottleneck prediction , the scheduling algorithm selects a bandwidth allocation strategy based on the state and action , this strategy is trained by the Q-value update formula, which is: ; in, is the discount factor that controls the weight of future rewards.

[0042] In this embodiment, the formula is used as follows: Initialize the environment: Collect historical data and initialize the deep reinforcement learning model. The network bandwidth, delay, packet loss rate and other data are used as environment inputs, and the bottleneck prediction results obtained by the deep learning model in step S3 are used as external inputs.

[0043] State update and decision-making: Every time the network state changes, the model updates according to the current state. And bottleneck prediction results , select the optimal bandwidth allocation strategy through the trained Q value update formula .

[0044] Dynamic adjustment and real-time optimization: In the actual operation process, the model continuously adjusts the reward function ) Self-learning, adjusting strategies to optimize download speed and bandwidth utilization.

[0045] Perform bandwidth allocation: Perform data flow scheduling according to the selected bandwidth allocation strategy, and avoid network bottlenecks and improve overall download efficiency by adjusting the bandwidth allocation of each download task.

[0046] S5. Perform node selection and load balancing based on the scheduling results; According to the scheduling result, the download node of the server is selected through the load balancing algorithm and load balancing is performed to obtain the load balancing result.

[0047] Load balancing algorithms include: Polling algorithm: Assign requests to each server in sequence in a cycle to evenly distribute the load.

[0048] In this embodiment, it is ensured that the download tasks can be evenly distributed among multiple servers, avoiding a node from overloading and causing a drop in download speed. Load balancing not only ensures the stability of the download speed, but also improves the overall fault tolerance of the system.

[0049] S6. Monitor server performance based on load balancing results; Based on the load balancing results, the performance of each server node is monitored to obtain monitoring data.

[0050] Monitoring includes: Server monitoring tools, which include Zabbix, Nagios, Grafana, Beszel.

[0051] Monitoring data includes: CPU usage, memory usage, disk I / O rate, and network bandwidth usage.

[0052] In this embodiment, the server monitoring tool is specifically introduced: Zabbix: Zabbix is ​​an open source monitoring software for monitoring the performance and availability of networks and applications.

[0053] It can monitor servers, virtual machines, cloud services, network devices, etc.

[0054] Zabbix provides a flexible alert mechanism that can send emails, text messages, or call scripts to notify administrators.

[0055] It has a powerful data collection feature that can collect various performance indicators and generate detailed reports and charts.

[0056] Nagios: Nagios is an open source monitoring system that can monitor IT infrastructure, networks, applications, and services.

[0057] It provides real-time status monitoring and alerts, and can customize monitoring items and alarm rules.

[0058] Nagios has a robust plugin ecosystem that can extend its functionality to monitor almost anything.

[0059] It is typically used in larger IT environments because it can be complex to configure and maintain.

[0060] Grafana: Grafana is an open source data visualization and monitoring platform that can be integrated with multiple data sources (such as Prometheus, Elasticsearch, InfluxDB, etc.).

[0061] It provides rich charts and dashboards that display monitoring data in real time.

[0062] Grafana supports alerts and notifications, and can send alert information to various channels.

[0063] It is usually used to display and analyze monitoring data rather than directly collect data.

[0064] Beszel: Beszel is a lightweight server monitoring platform that includes Docker statistics, historical data, and alerting capabilities. It has a friendly web interface, simple configuration, and works out of the box. Beszel supports automatic backups, multiple users, OAuth authentication, and API access.

[0065] It supports real-time monitoring of key server resources and records historical data. It displays key indicators such as CPU, memory, disk I / O, etc. through an intuitive interface, and also supports monitoring the running status of Docker containers.

[0066] After collecting monitoring data, we can compare the resource usage of each server to find resource bottlenecks in time and make adjustments. The key to this step is to ensure that after the download tasks are assigned to each node, the server can process requests stably and efficiently through real-time feedback of monitoring data, avoiding download bottlenecks caused by insufficient hardware resources.

[0067] S7, fault detection and repair based on monitoring data; Detect and repair the failed server node based on the monitoring data to obtain repair data. Detect and repair the failed server node includes: Fault detection system: Combined with the real-time monitoring data of the server, the faulty node is located and the system repairs the faulty node in time.

[0068] In this embodiment, when the CPU load of a node is too high or the network bandwidth is abnormal, the system can immediately detect the failure of the node and take measures (such as transferring tasks or restarting nodes). This step ensures that in large-scale download scenarios, the system can promptly detect and repair failures, minimize download interruptions or delays, and improve system reliability.

[0069] S8. Generate a fault recovery report based on the repair data.

[0070] Visualize the repaired data for user reference.

[0071] The crash recovery report includes: The node where the failure occurred, the type of failure, and the time when the failure occurred; The recovery time, performance changes, and load balancing adjustment status of each node.

[0072] In this embodiment, the data visualization tool is built based on the visualization model, which includes: Time Series Cluster Analysis (TCA) Model: Time series clustering analysis is used to cluster multi-dimensional network data so that similar network behaviors are classified into the same class for clearer presentation through graphical means. Bandwidth (BW), delay (D), packet loss rate (PL), traffic load (FL) and other data are divided into several time periods according to time, and dynamic time warping (DTW) technology is used to measure the similarity of time series data.

[0073] Time series data clustering model: .

[0074] in, The clustering service result at time t includes the network behavior classes in the current time period. Each class is clustered based on multiple dimensions such as bandwidth, delay, packet loss rate, and traffic load through DTW technology to determine the similarity.

[0075] Adaptive Graph Generation (AGG) Model: The adaptive graph generation model generates and adjusts the graph layout in real time based on the time series clustering results, the bottleneck prediction of the deep learning model (from step S3), and the current network status. Specifically, the core of the graph generation process is to dynamically adjust the visualization of network indicators through the predicted bottleneck information, especially when bottlenecks occur, to highlight the network data with greater impact.

[0076] Graphics generation and adjustment formula: .

[0077] in, Indicates at time The generated graphical layout, It's time The clustering result of is the bottleneck prediction value obtained in step S3.

[0078] Graph layout and structure based on bottleneck predictions ( ) for adaptive adjustment: if , it means there is a network bottleneck, then the graphical layout focuses on the data flow and network part where the bottleneck is located, using highlight colors or zooming in to display relevant data.

[0079] if , it means there is no bottleneck, then all data flows are displayed and various network indicators are evenly displayed.

[0080] Visual display and feedback mechanism: When real-time data enters the visualization tool, the network data is first divided into different categories (i.e., the working status of the network) through time series cluster analysis (TCA), and then the corresponding visualization graphics are dynamically generated based on the bottleneck prediction results (from the deep learning model). Users can see the health of the network through the graphics. When a bottleneck occurs, the graphics will automatically update and prompt the user to take measures.

[0081] Graphical display formula: .

[0082] is the final visualization graph, It is a graph layout generated based on temporal clustering. is the bottleneck prediction result obtained from the deep learning model, Graphical display based on Dynamic updates.

[0083] Use this model: Data collection and preprocessing: Collect real-time network indicator data, including bandwidth, latency, packet loss rate, traffic load, etc., and perform necessary preprocessing (such as data standardization).

[0084] Time series clustering: The collected data is divided into multiple time periods by time, and the data is grouped through time series clustering analysis (TCA) to identify similar time periods and network behavior patterns.

[0085] Bottleneck prediction and graph generation: The bottleneck prediction results obtained from step S3 are combined with the results of temporal clustering analysis to generate a graph using the adaptive graph generation (AGG) method, and the graph layout is adjusted in real time to reflect the network status.

[0086] Visual display and feedback: Based on the bottleneck prediction results, the graph will dynamically adjust the displayed content. When a bottleneck occurs in the network, the graph will automatically update to highlight the bottleneck area and provide feedback to guide users to take necessary adjustment measures. Embodiment 2

[0087] This embodiment also provides a computer device, which is suitable for a large-scale rapid downloading digital analysis method, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement a large-scale rapid downloading digital analysis method as proposed in the above embodiment.

[0088] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, a large-scale rapid download digital analysis method as proposed in the above embodiment is implemented.

[0089] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0090] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0091] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0092] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk case (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0093] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit with a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit with a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0094] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A large-scale rapid download digital analysis method, characterized in that: The following steps are involved: S1,Data collection and download performance benchmark; Collect network performance data and benchmark the network to obtain performance data; S2. Analyze bandwidth utilization based on performance data; Based on the performance data, the bandwidth monitoring tool is used to analyze the network traffic in real time to obtain bandwidth data; S3, identify network bottlenecks based on bandwidth data; Based on bandwidth data, the bandwidth data is trained through a deep learning model to identify the characteristics of network bottlenecks and obtain bottleneck information; S4, optimize data flow scheduling based on bottleneck information; Based on the bottleneck information, the data flow is optimized and scheduled through the dynamic data flow scheduling algorithm to obtain the scheduling result; S5. Perform node selection and load balancing based on the scheduling results; According to the scheduling result, the server's download node is selected through the load balancing algorithm and load balancing is performed to obtain the load balancing result; S6. Monitor server performance based on load balancing results; Based on the load balancing results, monitor the performance of each server node to obtain monitoring data; S7, fault detection and repair based on monitoring data; Detect and repair the failed server nodes according to the monitoring data to obtain repair data; S8. Generate a fault recovery report based on the repair data; Visualize the repaired data for user reference.

2. A large-scale fast download digital analysis method according to claim 1, characterized in that: The performance data in step S1 includes: Latency, packet loss, traffic load, bandwidth.

3. A large-scale fast download digital analysis method according to claim 1, characterized in that: The bandwidth data in step S2 includes: The bandwidth usage of each download task.

4. A large-scale fast download digital analysis method according to claim 1, characterized in that: The deep learning model in step S3 includes: Model input and output: Input data vector, ； in, is the bandwidth data, To delay data, is the packet loss rate data, is the traffic load data; Network bottleneck prediction output: ； in, is the mixed density neural network function, It is the predicted bottleneck value output by the model, indicating whether there is a bottleneck in the network. Its value is between 0 and 1, including 0 and 1. If it is greater than 0.5, it means there is a bottleneck, otherwise it means the network is normal.

5. A large-scale fast download digital analysis method according to claim 1, characterized in that: The dynamic flow data scheduling algorithm in step S4 includes: Define the state space vector ; in, is the bandwidth data at the current time t, is the delayed data at the current time t, is the packet loss rate data at the current time t, is the traffic load data at the current time t, is the predicted bottleneck value output by the model at the current time t; Action Space Vector , ； in, represents the bandwidth allocated to the nth download task at time t; Reward function, ； in, is the download speed of the current time step, and are weight factors that control the importance of download speed and bottleneck avoidance, respectively. It is a penalty item when a bottleneck occurs and is not effectively avoided, controlling the impact of the bottleneck; At each moment t, based on the current network status and bottleneck prediction , the scheduling algorithm selects a bandwidth allocation strategy based on the state and action , this strategy is trained by the Q-value update formula, which is: ； in, is the discount factor that controls the weight of future rewards.

6. A large-scale fast download digital analysis method according to claim 1, characterized in that: The load balancing algorithm in step S5 includes: Polling algorithm: Assign requests to each server in sequence in a cycle to evenly distribute the load.

7. A large-scale fast download digital analysis method according to claim 1, characterized in that: In step S6, the server monitoring tools used for monitoring include Zabbix, Nagios, Grafana, and Beszel.

8. A large-scale fast download digital analysis method according to claim 1, characterized in that: The monitoring data in step S6 includes: CPU usage, memory usage, disk I / O rate, and network bandwidth usage.

9. A large-scale fast download digital analysis method according to claim 1, characterized in that: The detection and repair of the failed server node in step S7 includes: Fault detection system: Combined with the real-time monitoring data of the server, the faulty node is located and the system repairs the faulty node in time.

10. A large-scale fast download digital analysis method according to claim 1, characterized in that: The fault recovery report in step S8 includes: The node where the failure occurred, the type of failure, and the time when the failure occurred; The recovery time, performance changes, and load balancing adjustment status of each node.

Images