A method and system for resource scheduling of a streaming media server supporting dynamic load balancing

Abstract

The application belongs to the technical field of information technology, and particularly discloses a streaming media server resource scheduling method and system supporting dynamic load balancing, wherein the method comprises the following steps: obtaining the playing content identifiers and the request source geographical positions of all user requests in a current time window, obtaining the current request frequency distribution of each content through content identifier statistics, and determining the content heat ranking list of the current time window according to the frequency distribution; for the top several contents in the content heat ranking list, extracting the corresponding edge access area from the user request source geographical position, and obtaining the transmission path set that each popular content currently mainly depends on through a pre-established area to backbone link mapping table; the application aims to solve the problem of low hot content distribution efficiency and uneven resource utilization caused by the mutual coupling of server node load and network link congestion in the existing streaming media scheduling method.

Description

Technical Field

[0001] This invention relates to the technical field of information technology, specifically to a method and system for scheduling streaming media server resources that supports dynamic load balancing. Background Technology

[0002] As a core pillar of internet entertainment and information dissemination, streaming media services are experiencing explosive growth in demand for high-definition video, live streaming, and on-demand content. This makes server resource scheduling a crucial factor in ensuring a smooth viewing experience for users. In scenarios with large-scale concurrent access, the efficient allocation of computing, network, and storage resources directly determines the stability and quality of service of the entire system.

[0003] While many current scheduling methods can allocate tasks based on the server's current load, they often overlook real-time changes in network transmission paths and the interplay between multiple user requests. This leads to seemingly reasonable allocation decisions frequently failing in practice. Particularly during peak hours, when a large number of users simultaneously request similar content, traffic can easily accumulate on some critical network links or edge nodes. The scheduling system struggles to anticipate these potential pressure points, resulting in recurring issues such as video stuttering, prolonged buffering times, and even service interruptions.

[0004] A core technical challenge in streaming media services lies in the strong interplay between the computational load of server nodes and the transmission congestion of network links. On the one hand, simply increasing the processing load on a single server will rapidly increase its outbound bandwidth requirements; on the other hand, when a network link approaches saturation, even if backend servers still have ample idle capacity, frontend users will not be able to access data in a timely manner. This tight coupling between load and congestion necessitates that scheduling decisions must simultaneously consider the dynamic balance of both types of resources, and any mismatch in either will quickly amplify into a global decline in service quality. For example, during a live broadcast of a popular sporting event, if the scheduling system pushes a large number of high-bitrate video streams to several servers in the same region, although the CPU and memory of these servers can temporarily handle the load, the backbone link they all rely on will be congested within a short period, causing buffering for users in the entire region, while servers in other regions remain idle.

[0005] Therefore, accurately predicting whether a scheduling scheme will cause congestion on a specific server or network link before making resource allocation decisions, and promptly identifying and rejecting the scheme when the congestion risk exceeds an acceptable range, has become a key issue supporting dynamic load balancing streaming media server resource scheduling methods and systems. Summary of the Invention

[0006] This invention provides a streaming media server resource scheduling method and system that supports dynamic load balancing. The purpose is to solve the problems of low distribution efficiency and uneven resource utilization of hot content caused by the coupling of server node load and network link congestion in existing streaming media scheduling methods.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] A streaming media server resource scheduling method supporting dynamic load balancing includes: obtaining the playback content identifiers and request source geographical locations of all user requests within the current time window; statistically analyzing the content identifiers to obtain the current request frequency distribution of each content; determining a content popularity ranking list for the current time window based on the frequency distribution; for the top-ranked content in the popularity ranking list, extracting the corresponding edge access regions from the user request source geographical locations; obtaining the set of transmission paths currently mainly relied upon by each popular content through a pre-established region-to-backbone link mapping table; collecting real-time bandwidth utilization and queue latency data for each transmission path from a network monitoring system within the most recent collection period; calculating the short-term load trend value and rise rate value of each path using an exponentially weighted moving average; matching the set of main transmission paths for popular content with the corresponding short-term load trend value and rise rate value; and determining whether at least one path is currently experiencing high congestion by setting bandwidth utilization grading thresholds and rise rate thresholds. If a high-congestion-risk path is identified, content popularity clustering is performed on the current set of candidate servers associated with the content. Based on the similarity of user request geolocation and the time relevance of content request, candidate servers are divided into multiple load-related groups. For each load-related group, the difference between the predicted outbound bandwidth demand of servers within the group and the remaining capacity of the shared links within the group is calculated. The difference is compared to determine if the current scheduling scheme has a link saturation risk. If the difference exceeds a preset tolerance threshold, the subset of servers with the highest bandwidth pressure is removed from the load-related groups, resulting in an adjusted set of candidate server allocations. A new request distribution decision table for the content is generated based on the adjusted set of candidate server allocations, and the decision table is distributed to edge scheduling nodes for distribution based on load coupling relationships. The actual link bandwidth utilization and server outbound traffic data after this scheduling execution are transmitted back to the real-time status database for subsequent time window content popularity statistics and path load trend calculations, forming a closed-loop update.

[0009] In one aspect of the present invention, the step of obtaining the playback content identifiers and request source geographical locations of all user requests within the current time window, statistically obtaining the current request frequency distribution of each content through content identifiers, and determining the content popularity ranking list for the current time window based on the frequency distribution includes:

[0010] Obtain the playback content identifier and request source geolocation data of all user requests within the current time window from the system log. Remove invalid records through data cleaning to obtain a preliminary user request dataset.

[0011] Based on the initially compiled user request dataset, the playback content is grouped and statistically analyzed according to the content identifier, and the request frequency corresponding to each content identifier is calculated to obtain the request frequency distribution results for each content.

[0012] Based on the request frequency distribution results and combined with geographic location data, the request sources of each content identifier are divided into regions. The contribution of different geographic locations to the content request frequency is analyzed, and the popularity distribution of each content in different regions is determined.

[0013] Based on the distribution of popularity, each content identifier is sorted from high to low according to the request frequency to generate a content popularity ranking list for the current time window;

[0014] If the popularity of a certain content identifier is higher than a preset threshold in multiple geographical locations, it will be marked as high-popularity content, and a sublist of high-popularity content will be generated.

[0015] Based on the high-popularity content sublist and the overall popularity ranking list, and combined with the dynamic changes of the time window, the priority of content identifiers is adjusted to obtain the final optimized content recommendation ranking result.

[0016] In one aspect of the invention, for the top-ranked content in the content popularity ranking list, extracting the corresponding edge access region from the geographical location of the user request source, and obtaining the set of transmission paths currently mainly relied upon by each popular content through a pre-established region-to-backbone link mapping table, includes:

[0017] Extract the top-ranked popular content from the content popularity ranking list, obtain the corresponding user request data, and combine it with geographical location information to determine the main request source region for each popular content.

[0018] For the main request source region, the corresponding edge access region is obtained through pre-established regional division rules, and the access distribution of each popular content in different regions is obtained;

[0019] Based on the data from the edge access areas, a pre-established mapping table is used to obtain the backbone link information corresponding to each access area, and to determine the set of transmission paths that popular content depends on in different areas.

[0020] By analyzing the traffic distribution of each popular content on different backbone links through the transmission path set, the load distribution data of each path is obtained.

[0021] If the load distribution data of a certain transmission path exceeds the preset threshold, traffic diversion processing is performed on that path to obtain the adjusted path allocation scheme and determine the optimized transmission path set.

[0022] Based on the optimized set of transmission paths, the regional access and link mapping data of popular content are updated to obtain the final path allocation result.

[0023] In one aspect of the invention, the step of collecting real-time bandwidth utilization and queue latency data for each transmission path within the most recent collection period from the network monitoring system, and calculating the short-term load trend value and rate of increase value for each path using an exponentially weighted moving average, includes:

[0024] Obtain bandwidth utilization and queue latency data for each transmission path within the most recent period from the network monitoring system;

[0025] The short-term load trend value for each path is obtained by calculating the bandwidth utilization sequence using an exponentially weighted moving average.

[0026] The load rise rate of each path is obtained by calculating the queue delay sequence using an exponentially weighted moving average.

[0027] Based on the short-term load trend value and the load rise rate value, determine whether the path is currently in a low load stable state, a medium load rising state, or a high load rapid rise state.

[0028] For paths experiencing moderate load increases, obtain the percentage of popular content traffic currently carried by that path;

[0029] For paths that are experiencing a rapid increase in high load, obtain detailed data on all content traffic currently carried by that path;

[0030] If a path is in a state of high load and rapid increase, select the top three content items with the highest traffic share from the content traffic details data currently carried by that path and determine them as priority traffic diversion targets;

[0031] For priority traffic diversion targets, select available backup paths from the set of other paths with lower load rise rates;

[0032] Determine the offloading reception capacity of each backup path based on the short-term load trend value of the backup path;

[0033] The distribution ratio of priority traffic to each backup path is obtained by calculating the traffic share and the traffic receiving capacity.

[0034] The traffic allocation scheme for each transmission path is updated using the traffic splitting ratio allocation result to obtain the adjusted path traffic distribution data;

[0035] Based on the adjusted path traffic distribution data, update the predicted load reference value for the corresponding path in the network monitoring system.

[0036] In one aspect of the invention, the step of matching the set of main transmission paths for popular content with the short-term load trend value and rise rate value of the corresponding paths, and determining whether there is at least one path in a high-congestion risk state by setting bandwidth utilization grading thresholds and rise rate thresholds, includes:

[0037] Obtain the main transmission paths of popular content from the network monitoring system;

[0038] For each path in the main transmission path set, obtain the corresponding short-term load trend value and load rise rate value;

[0039] The bandwidth utilization sequence of each path is divided into states by a preset bandwidth utilization grading threshold to obtain the current load level of each path.

[0040] The load increase rate of each path is compared using a preset increase rate threshold to determine whether the load increase exceeds the threshold.

[0041] If the load level of a certain path reaches high load and the load increase rate exceeds the threshold, then the path is determined to be in a high congestion risk state.

[0042] For paths at high congestion risk, obtain the percentage of traffic from all popular content currently carried by that path;

[0043] Popular content is sorted based on traffic share data, and the top three most popular content items are selected as priority traffic diversion targets.

[0044] Paths with load increase rates lower than a preset threshold are selected from other path sets to obtain a set of available backup paths;

[0045] The current load status of each backup path is assessed by using short-term load trend values ​​to determine the offloading and receiving capacity of each backup path.

[0046] The proportion of traffic diverted to each backup path is calculated based on the traffic share of the priority diversion target and the diversion receiving capacity of each backup path.

[0047] The traffic allocation scheme for each transmission path in the network is updated by using the traffic splitting ratio to obtain the adjusted path traffic distribution data;

[0048] The predicted load reference value for the corresponding path in the network monitoring system is updated by adjusting the path traffic distribution data.

[0049] In one aspect of the invention, if a high-congestion-risk path is determined to exist, content popularity clustering is performed on the candidate set of servers currently associated with the content. Based on the similarity of user request geographic location and the relevance of content request time, the candidate servers are divided into multiple load-related groups, including:

[0050] Obtain target content information carried on high-congestion-risk paths from the network monitoring system;

[0051] Obtain a set of candidate servers associated with the target content information;

[0052] Clustering is performed on the candidate server set based on the similarity of user requests to obtain geographically related groups;

[0053] Adjustments are made to the geographic load-related groups based on the content request time relevance to obtain a time-geographic fusion load-related group set;

[0054] Obtain the load status data of each group in the load-related group set;

[0055] The remaining processing capacity of servers in each group is sorted based on load status data to determine the priority server sequence for load sharing.

[0056] Select at least three servers with the highest remaining processing capacity from the priority sharing server sequence to form the target server set for traffic diversion;

[0057] The content distribution path set is updated by using the target server set for traffic splitting to obtain the adjusted content distribution path set;

[0058] Modify the request scheduling strategy by adjusting the content distribution path set, and update the traffic guidance rules for the target content in the network.

[0059] In one aspect of the invention, the step of calculating the difference between the predicted server outbound bandwidth demand and the remaining capacity of the shared links within each load-related group, and determining whether the current scheduling scheme has a link saturation risk by comparing the magnitude of the difference, includes:

[0060] Extract link segments with high congestion risk from the network monitoring module;

[0061] Obtain the target content identifier carried by the high-congestion-risk link segment;

[0062] Retrieve a list of candidate content distribution servers associated with the target content identifier;

[0063] The predicted outbound bandwidth demand is calculated for each server in the candidate list of content delivery servers based on historical traffic records.

[0064] Obtain the remaining capacity value of each shared link on the content distribution path;

[0065] The difference sequence is obtained by subtracting the predicted outbound bandwidth demand of each server from the remaining capacity of the corresponding shared link.

[0066] By comparing the magnitudes of the differences in the difference sequence, it can be determined whether the current scheduling scheme has the risk of link saturation.

[0067] If it is determined that there is a risk of link saturation, the k-means clustering method is used to perform outbound load grouping on the predicted outbound bandwidth demand of the content delivery server candidate list to obtain multiple outbound load groups.

[0068] For each outbound load group, obtain the minimum remaining capacity of the shared link for the servers within the group;

[0069] The load groups at each egress point are sorted according to the minimum remaining capacity of the shared links within the group, resulting in a sequence of load groups with the largest remaining capacity in descending order.

[0070] Servers from the first few load groups in descending order of remaining capacity are selected to form a set of servers with low saturation risk.

[0071] The high-risk servers in the original content distribution path are replaced with a set of low-saturation risk servers to obtain the updated set of content distribution paths.

[0072] In one aspect of the invention, if the difference exceeds a preset allowable threshold, the subset of servers with the highest bandwidth pressure is removed from the load-related group to obtain an adjusted server allocation candidate set, including:

[0073] By obtaining data on a subset of servers with high bandwidth usage from load-related groups, a list of servers to be removed can be determined.

[0074] Based on the list of servers to be removed, perform server removal operations for load-related groups to obtain a preliminary adjusted candidate set;

[0075] Using the initially adjusted candidate set, the bandwidth pressure data of each server is obtained to determine whether there are any servers that still exceed the allowable threshold.

[0076] If there are still servers that exceed the allowable threshold, a second screening is performed on the bandwidth pressure data to obtain a further streamlined candidate set.

[0077] By further refining the candidate set, the allocation status of each server after load adjustment is obtained, and the final server allocation candidate set is determined.

[0078] For the final set of server allocation candidates, a preset allocation strategy is used to match resources and obtain an optimized load balancing scheme.

[0079] Based on the optimized load balancing scheme, bandwidth resources are redistributed for each server to determine the final load balancing state.

[0080] In one aspect of the invention, the step of regenerating a request distribution decision table for the content based on the adjusted server allocation candidate set, and distributing the decision table to edge scheduling nodes for distribution through load coupling, includes:

[0081] Obtain server request distribution related data based on the adjusted server allocation candidate set;

[0082] Construct a request distribution decision table using a load coupling matrix;

[0083] Perform a distribution target server matching operation on the request distribution decision table to obtain a preliminary distribution mapping relationship;

[0084] The initial distribution mapping relationship is sent to the edge scheduling node to execute the distribution instruction;

[0085] Obtain the execution status data returned by the edge scheduling node to determine if there are any execution status anomalies;

[0086] If there is an execution status anomaly, extract the anomaly association chain from the load coupling relationship;

[0087] Generate local decision table update items based on abnormal association chains;

[0088] The corresponding entries in the request distribution decision table are overwritten by updating the local decision table to obtain the corrected request distribution decision table.

[0089] The modified request distribution decision table drives the edge scheduling nodes to complete the request routing operation and obtain the distribution execution result.

[0090] In one aspect of the invention, the step of transmitting the actual link bandwidth utilization and server egress traffic data after this scheduling execution back to the real-time status database for subsequent time window content popularity statistics and path load trend calculation, forming a closed-loop update, includes:

[0091] Data is fed back to the real-time status database based on link bandwidth utilization and server outbound traffic.

[0092] Obtain link bandwidth utilization and server egress traffic data for the most recent multiple time windows from the real-time status database;

[0093] Calculate content popularity statistics using data from multiple recent time windows;

[0094] The path load trend value is calculated using content popularity statistics and historical path load data.

[0095] Adjust the server allocation candidate set for the current time window based on path load trend values;

[0096] Extract the target server matching information for each request from the adjusted server allocation candidate set;

[0097] A request distribution decision table for the current time window is generated by matching information from the target server.

[0098] The request distribution decision table is sent to the edge scheduling node to perform the distribution operation;

[0099] Obtain the link bandwidth utilization and server egress traffic data after the completion of this distribution from the edge scheduling node;

[0100] After the distribution is completed, the link bandwidth utilization and server outbound traffic data will be sent back to the real-time status database to update the status records.

[0101] Determine whether the path load trend value exceeds the preset threshold based on the link bandwidth utilization and server outbound traffic data in the updated status record;

[0102] If the path load trend value exceeds the preset threshold, the server with the highest current load is removed from the server allocation candidate set to obtain a new candidate set;

[0103] The request distribution decision table for the next time window is regenerated using the new candidate set.

[0104] In another aspect, the present invention also relates to a streaming media server resource scheduling system supporting dynamic load balancing, the system comprising:

[0105] The request data collection and popularity analysis module is used to obtain the playback content identifiers and request source geographical locations of all user requests within the current time window. It obtains the current request frequency distribution of each content through content identifier statistics and determines the content popularity ranking list for the current time window based on the frequency distribution.

[0106] The popular content path mapping module is used to extract the corresponding edge access area from the geographical location of the user request source for the top-ranked content in the content popularity ranking list, and obtain the set of transmission paths that each popular content currently mainly depends on through a pre-established region-to-backbone link mapping table.

[0107] The path load trend calculation module is used to collect real-time bandwidth utilization and queue latency data of each transmission path in the most recent collection period from the network monitoring system, and calculate the short-term load trend value and rise rate value of each path through exponential weighted moving average.

[0108] The congestion risk assessment module is used to match the main transmission path set of popular content with the short-term load trend value and rise rate value of the corresponding path. By setting bandwidth utilization grading threshold and rise rate threshold, it can determine whether there is at least one path in a high congestion risk state.

[0109] The server clustering and grouping module is used to perform content popularity clustering on the current set of candidate servers associated with the content if a high congestion risk path is determined. The candidate servers are divided into multiple load-related groups based on the similarity of the geographical location of the user request and the relevance of the content request time.

[0110] The link saturation risk assessment module is used to calculate the difference between the predicted value of the server outbound bandwidth demand in each load-related group and the remaining capacity of the shared link in the group. By comparing the size of the difference, it is determined whether there is a link saturation risk in the current scheduling scheme.

[0111] The server set adjustment module is used to remove the subset of servers with the greatest bandwidth pressure from the load-related group if the difference exceeds the preset allowable threshold, and obtain the adjusted server allocation candidate set.

[0112] The distribution decision generation and distribution module is used to regenerate the request distribution decision table for the content based on the adjusted server allocation candidate set, and distribute the decision table to the edge scheduling node for distribution through load coupling relationship;

[0113] The scheduling feedback and closed-loop update module is used to send the actual link bandwidth utilization and server outbound traffic data after the current scheduling execution back to the real-time status database for subsequent time window content popularity statistics and path load trend calculation, forming a closed-loop update.

[0114] Compared with the prior art, the present invention has the following beneficial effects:

[0115] This invention collects the content identifiers and source geographical locations of user requests within the current time window in real time, statistically analyzes the frequency distribution of content requests, and generates a popularity ranking. For top-ranked popular content, it determines the main transmission paths it relies on by combining the mapping between user access areas and backbone links. Subsequently, based on bandwidth utilization and queue latency data collected by network monitoring, it uses an exponentially weighted moving average to calculate the short-term load trend and rate of increase of the path, and uses a graded threshold to determine whether there are high-congestion risk paths. When a risk is detected, the candidate servers associated with the content are clustered into load-related groups based on the popularity of user request geographical similarity and time relevance. The difference between the outbound bandwidth demand and the remaining capacity of the shared link is predicted for each group. If it exceeds the allowable threshold, the subset of servers with the highest bandwidth pressure is removed, the candidate set is adjusted, and an optimized request distribution decision table is generated and executed. Finally, the actual link utilization and server traffic data are fed back to form a closed-loop continuous optimization. This invention effectively solves the problem of multi-level link congestion from edge to backbone and server load imbalance caused by concentrated requests for popular streaming media content. It realizes dynamic and accurate scheduling based on real-time popularity perception, path risk prediction and load coupling relationship, which significantly improves the stability of streaming media distribution and resource utilization efficiency. Attached Figure Description

[0116] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0117] Figure 1 This is one of the flowcharts for a streaming media server resource scheduling method that supports dynamic load balancing according to the present invention.

[0118] Figure 2 This is the second flowchart of a streaming media server resource scheduling method that supports dynamic load balancing according to the present invention.

[0119] Figure 3 This is the third flowchart of a streaming media server resource scheduling method that supports dynamic load balancing according to the present invention. Detailed Implementation

[0120] The present invention will be further described below with reference to embodiments. These embodiments are merely some, not all, of the embodiments of the present invention. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the protection scope of the present invention.

[0121] Please see Figures 1-3As shown in the figure, this embodiment discloses a streaming media server resource scheduling method and system that supports dynamic load balancing, wherein the method may specifically include:

[0122] Step 101: Obtain the playback content identifier and request source geolocation of all user requests within the current time window. Calculate the current request frequency distribution of each content based on the content identifier, and determine the content popularity ranking list for the current time window based on the frequency distribution.

[0123] The system logs are used to retrieve the playback content identifiers and request source geolocation data for all user requests within the current time window. Data cleaning removes invalid records, resulting in a preliminary user request dataset. Based on this dataset, playback content is grouped and statistically analyzed according to content identifiers, calculating the request frequency for each identifier to obtain the request frequency distribution. Based on this frequency distribution and geolocation data, the request sources for each content identifier are regionally segmented, analyzing the contribution of different geographical locations to the content request frequency and determining the popularity distribution of each content across different regions. Using this popularity distribution, content identifiers are sorted from highest to lowest request frequency, generating a content popularity ranking list for the current time window. If a content identifier's popularity exceeds a preset threshold across multiple geographical locations, it is marked as high-popularity content, generating a high-popularity content sublist. Based on the high-popularity content sublist and the overall popularity ranking list, and considering the dynamic changes within the time window, the priority of content identifiers is adjusted to obtain the final optimized content recommendation ranking result.

[0124] Specifically, the system collects all user playback request logs every 5 minutes as a sliding time window. First, it extracts the content_id (e.g., video ID v123456) and the geographical location information of the request source from the logs (obtained through IP address resolution to the provincial administrative region or city code, e.g., Beijing Chaoyang District code 110105). For example, if 4287 valid requests are captured in the current window, requests with content_id v987654 occur 812 times, v456789 531 times, and v321654 243 times, with the remaining requests being more dispersed. All content_ids are grouped and counted according to their frequency of occurrence. The frequency distribution is calculated using Python's Counter or pandasvalue_counts methods, resulting in a dictionary format like {v987654:812,v456789:531,v321654:243,v112233:189,v445566:156,}. To determine the popularity ranking, a weighted frequency algorithm is used to calculate the popularity value: Popularity value = Number of requests × (1 + Location diversity factor), where the location diversity factor = Number of unique cities where the content was requested ÷ Total number of cities requesting the content. For example, v987654 appears in 41 cities out of a total of 87 cities, so its diversity factor is 0.4715, and its popularity value = 812 × (1 + 0.4715) = 1194.58. On the other hand, v456789 only appears in 12 cities, so its diversity factor is 0.1379, and its popularity value = 531 × 1.1379 = 604.22. This method considers both the absolute number of requests and the breadth of content coverage, avoiding the influence of single-region click fraud on the ranking. All content popularity values ​​are sorted in descending order, and the top 50 are used to form a popularity ranking for the current time window. For example, the 1st place is v987654 (popularity 1194.58), the 2nd place is v789123 (popularity 892.41), and the 3rd place is v456789 (popularity 604.22). This ranking is output in JSON format for downstream recommendation systems or monitoring dashboards to use in real time. The entire process is automatically triggered at the end of each 5-minute window using streaming computing frameworks such as Flink or Spark Streaming, ensuring low latency and high accuracy.

[0125] Step 102: For the top-ranked content in the content popularity ranking list, extract the corresponding edge access area from the geographical location of the user request source, and obtain the set of transmission paths that each popular content currently mainly relies on through a pre-established region-to-backbone link mapping table.

[0126] The system extracts top-ranked popular content from a content popularity ranking list, obtains corresponding user request data, and combines this with geographic location information to determine the primary request source region for each popular content. For each primary request source region, edge access regions are obtained using pre-established regional division rules, revealing the access distribution of each popular content across different regions. Based on the edge access region data, a pre-established mapping table is used to obtain backbone link information corresponding to each access region, determining the set of transmission paths that popular content depends on in different regions. The traffic distribution of each popular content across different backbone links is analyzed using these transmission path sets, yielding load distribution data for each path. If the load distribution data for a transmission path exceeds a preset threshold, traffic is diverted for that path, resulting in an adjusted path allocation scheme and an optimized transmission path set. Based on the optimized transmission path set, the regional access and link mapping data for popular content are updated, yielding the final path allocation result.

[0127] Specifically, for the top-ranked content in the popularity ranking list, the system first extracts the corresponding edge access areas from the geographic location data of the user request source. Assuming that the top three content IDs on the popularity list are v111222, v333444, and v555666, and their requests come from multiple cities, the system maps these cities to edge access areas through IP address resolution. For example, 65% of the requests for v111222 come from edge nodes A1 and A2 in East China, 25% come from node B1 in South China, and the remaining 10% come from node C1 in North China, covering a total of three main access areas. Next, the system uses a pre-established regional-to-backbone link mapping table to associate these edge access regions with specific transmission paths. For example, East China region A1 and A2 correspond to backbone link L1, South China region B1 corresponds to link L2, and North China region C1 corresponds to link L3. The system calculates the main transmission path set for v111222 as {L1:65%, L2:25%, L3:10%}. Similarly, the path set for v333444 is {L1:50%, L2:30%, L3:20%}, and for v555666 it is {L1:40%, L2:40%, L3:20%}. Subsequently, based on a path load balancing algorithm, the system calculates the deviation between the current traffic share and the historical average load for each link. For example, if the current load share of L1 is 55%, the historical average is 45%, and the deviation is 10%, if the deviation exceeds the threshold of 5%, the path is marked as high-load and requires optimization scheduling. Combining content popularity and path load, the system further analyzes the transmission efficiency of popular content. For example, v111222 has the highest traffic share on L1, but the L1 load is already high. The system generates optimization suggestions to divert some traffic to the less loaded L2. The optimized path distribution is adjusted to {L1: 50%, L2: 35%, L3: 15%}, thereby improving transmission stability. The entire process uses automated scripts to periodically update the mapping table and load data to ensure the accuracy of the path set. The analysis results are stored in a structured format for subsequent use by the network scheduling module.

[0128] Step 103: Collect real-time bandwidth utilization and queue latency data for each transmission path within the most recent collection period from the network monitoring system, and calculate the short-term load trend value and rate of increase value for each path using an exponentially weighted moving average.

[0129] The network monitoring system obtains bandwidth utilization and queue latency data for each transmission path within the most recent period. A short-term load trend value for each path is calculated using an exponentially weighted moving average (AMA) on the bandwidth utilization sequence. Similarly, a load rise rate value for each path is calculated using an AMA on the queue latency sequence. Based on the short-term load trend and load rise rate values, the path is determined to be in a low-load stable state, a medium-load rising state, or a high-load rapidly rising state. For paths in a medium-load rising state, the percentage of popular content traffic currently carried by that path is obtained. For paths in a high-load rapidly rising state, detailed data on all content traffic currently carried by that path is obtained. If a path is in a high-load rapidly rising state, the top three content items with the highest traffic percentages from the detailed content traffic data are selected as priority traffic offloading targets. For priority traffic offloading targets, available backup paths are selected from the set of other paths with lower load rise rates. Based on the short-term load trend values ​​of the backup paths, the offloading reception capacity of each backup path is determined. The traffic offloading ratio allocation result for the priority traffic offloading targets to each backup path is obtained by calculating the traffic percentage and offloading reception capacity. The traffic allocation scheme for each transmission path is updated using the traffic splitting ratio results, resulting in adjusted path traffic distribution data. Based on the adjusted path traffic distribution data, the predicted load reference value for the corresponding path in the network monitoring system is updated.

[0130] Specifically, the network monitoring system collects and analyzes performance data for core transmission paths in real time during the most recent collection period. This is implemented through an automated process. First, the system obtains real-time bandwidth utilization and queue latency data for each transmission path from distributed monitoring probes. For example, path P1 has a bandwidth utilization of 72.5% and a queue latency of 35 milliseconds; path P2 has a bandwidth utilization of 48.3% and a queue latency of 20 milliseconds; and path P3 has a bandwidth utilization of 65.7% and a queue latency of 28 milliseconds. This data is updated every 5 minutes to ensure real-time performance. Next, the system uses an Exponentially Weighted Moving Average (EWMA) algorithm to calculate the short-term load trend value for each path. The algorithm uses a decay factor of 0.3 and weights the calculation with historical data from the past 6 periods, resulting in a short-term load trend value of 70.8% for P1, 50.2% for P2, and 63.9% for P3, reflecting the recent smooth change trend of load. Simultaneously, the system further calculates the load escalation rate by comparing the trend value difference between the current period and the previous period. For example, the trend value of P1 rises from 69.5% in the previous period to 70.8%, with an escalation rate of 1.3%, while P2's escalation rate is 0.5% and P3's is 1.1%, thus quantifying the rate of load change. Subsequently, the system compares the escalation rate value with a preset threshold of 0.8%, marking P1 and P3 as potentially high-risk paths. This data is then linked to the network quality assessment module, combining the path's physical distance and historical failure rate (e.g., P1's historical failure rate is 2.1%) for comprehensive analysis, generating a complete risk prediction report, which is automatically stored as structured data for later retrieval. The entire process is executed periodically by system scripts to ensure the continuity and logic of data processing. The high-risk marking of path P1 will trigger a further increase in monitoring frequency, shortening it from 5 minutes to 2 minutes, forming a closed-loop feedback.

[0131] The short-term load trend value for each path is calculated using the exponentially weighted moving average (EWMA). and the value of the rate of ascent v t :

[0132]

[0133] in, This is the observed value of bandwidth utilization (or queue delay) for the current cycle path;

[0134] The attenuation factor is α (0 < α < 1, in this embodiment α = 0.3).

[0135] This is the trend value from the previous period. The initial trend value can be the first observation.

[0136] Load rise rate value It reflects the rate of change in load.

[0137] Step 104: Match the main transmission path set of popular content with the short-term load trend value and rise rate value of the corresponding path, and determine whether there is at least one path in a high congestion risk state by setting bandwidth utilization grading threshold and rise rate threshold.

[0138] The system obtains a set of primary transmission paths for popular content from the network monitoring system. For each path in this set, it acquires the corresponding short-term load trend value and load rise rate value. The bandwidth utilization sequence of each path is categorized into states using a preset bandwidth utilization grading threshold to determine the current load level of each path. A preset rise rate threshold is used to compare the load rise rate values ​​of each path to determine if the load rise exceeds the threshold. If a path reaches a high load level and its load rise rate exceeds the threshold, the path is identified as being in a high-congestion-risk state. For paths in a high-congestion-risk state, the system acquires the percentage of all popular content traffic currently carried by that path. Popular content is sorted based on its traffic percentage, and the top three most popular content items are selected as priority traffic offloading targets. Paths with load rise rates below a preset threshold are selected from the other path sets to obtain a set of available backup paths. The current load of each backup path is evaluated using short-term load trend values ​​to determine the traffic offloading and receiving capacity of each backup path. Based on the traffic percentage of the priority traffic offloading targets and the traffic offloading and receiving capacity of each backup path, the offloading ratio to each backup path is calculated. The traffic allocation scheme for each transmission path in the network is updated using a traffic splitting ratio to obtain adjusted path traffic distribution data. The predicted load reference value for the corresponding path in the network monitoring system is then updated using this adjusted path traffic distribution data.

[0139] Specifically, the network management system automatically extracts matching data for the current period from the set of main transmission paths corresponding to popular service traffic. For example, the short-term load trend values ​​of the five core paths Q1 to Q5 associated with popular live video services are 81.4% for Q1, 59.6% for Q2, 77.2% for Q3, 42.8% for Q4, and 68.9% for Q5, with rising rates of 1.7% for Q1, 0.4% for Q2, 1.4% for Q3, -0.2% for Q4, and 0.9% for Q5. The system first applies a bandwidth utilization grading threshold rule to divide the trend values ​​into three intervals: below 60.0% is the low load interval, 60.0% to 75.0% is the medium load interval, and above 75.0% is the high load interval. Based on this, Q1 and Q3 are determined to be in the high load interval, Q5 is in the medium load interval, and the rest are in the low load interval. Next, a threshold of 1.2% was set for the rate of increase. Paths with a rate of increase greater than or equal to 1.2% were marked as rapidly growing. Q1 and Q3 met this condition, while Q5's 0.9% did not. The system then executed a comprehensive risk assessment logic: if a path simultaneously met the high load range and had a rate of increase greater than or equal to 1.2%, it was judged as a high congestion risk state; if it only met the high load range but had a rate of increase less than 1.2%, it was marked as a moderate concern state; other cases were considered normal. Calculations showed that Q1 was confirmed as a high congestion risk state because its trend value of 81.4% was higher than 75.0% and its rate of increase of 1.7% exceeded 1.2%. Q3 also entered the high congestion risk state because it met both the 77.2% and 1.4% criteria. Although Q5's trend value was close to the high range, its rate of increase was insufficient, so it was only listed as a moderate concern. The system associates the list of high-congestion-risk paths with business priorities, automatically generates alarm events and pushes them to the traffic scheduling module, triggering the preparation of path traffic redirection plans. The entire judgment and marking process is executed every 3 minutes through a background scheduled task to ensure continuous tracking of the transmission status of popular services.

[0140] Step 105: If a high-congestion-risk path is identified, content popularity clustering is performed on the candidate server set currently associated with the content. The candidate servers are divided into multiple load-related groups based on the similarity of the geographical location of the user request and the relevance of the content request time.

[0141] Information on target content carried on high-congestion-risk paths is obtained from the network monitoring system. A candidate set of associated servers is obtained based on the target content information. The candidate server set is clustered based on the similarity of user request geographic locations to obtain geographic load-related groups. These geographic load-related groups are then adjusted based on the temporal relevance of content requests to obtain a time-geographically integrated load-related group set. Load status data for each group within the load-related group set is obtained. The remaining processing capacity of servers within each group is ranked according to the load status data to determine a priority server sequence for load sharing. At least three servers with the highest remaining processing capacity from the priority server sequence are selected to form the target server set for traffic splitting. The content distribution path set is updated using the target server set to obtain the adjusted content distribution path set. The request scheduling strategy is modified based on the adjusted content distribution path set to update the traffic routing rules for the target content in the network.

[0142] Specifically, after detecting a high-congestion-risk path, the network management system immediately performs content popularity clustering processing on the candidate set of edge servers associated with the currently popular live video streaming services. This candidate set includes eight servers, S1 to S8, with their current cached live content popularity scores (calculated based on the number of requests in the most recent hour) being 0.92 for S1, 0.65 for S2, 0.88 for S3, 0.41 for S4, 0.79 for S5, 0.57 for S6, 0.31 for S7, and 0.84 for S8. The system first uses the K-means clustering algorithm (K is dynamically taken as the square root of the number of candidate servers, approximately 2.8, and then rounded to 3), using the popularity score as the main feature vector, and combining the geographic location distribution entropy value of the most recent content requests on each server and the standard deviation of the request arrival time interval as auxiliary dimensions to perform multi-dimensional clustering. After clustering iteration, three load-related groups were formed: the high popularity group includes S1 (0.92), S3 (0.88), and S8 (0.84), whose geographic distribution entropy values ​​are all below 0.45, indicating that users are concentrated in eastern coastal cities, and the standard deviation of the request interval is less than 1.2 seconds, showing obvious peak characteristics; the medium popularity group includes S5 (0.79), S2 (0.65), and S6 (0.57), whose server users are mainly distributed in central provinces, and the standard deviation of the time interval fluctuates relatively steadily between 2.1 seconds and 3.8 seconds; the low popularity group is S4 (0.41) and S7 (0.31), with geographic entropy values ​​above 1.6 and a standard deviation of the request interval of 7.4 seconds, indicating that the content is not popular and the users are relatively dispersed. The system then calculates the average load pressure index within each group (popularity score multiplied by current CPU utilization and weighted request rate). The average index for the high-popularity group is 0.87, for the medium-popularity group it is 0.62, and for the low-popularity group it is only 0.38. Based on this, it is determined that servers in the high-popularity group face greater scheduling pressure. Ultimately, based on the grouping results, the system automatically prioritizes allocating subsequent new user requests to servers in the medium and low-popularity groups to achieve load balancing. Simultaneously, a dynamic scaling strategy is reserved for the high-popularity group. The entire clustering and grouping determination process is refreshed every 5 minutes using a streaming computing framework to ensure coordination with the congestion risk assessment cycle.

[0143] Among them, content popularity clustering is performed on the server candidate set, and the feature vector includes:

[0144] Popularity score pi: The normalized value of the number of requests for cached content on server i in the most recent hour.

[0145] Geographical distribution entropy : Reflects the degree of dispersion of request sources, calculated using the following formula:

[0146]

[0147] in, The proportion of content requests on server i originating from region j;

[0148] Standard deviation of request time interval This reflects the volatility of the arrival of requests, and the calculation formula is:

[0149]

[0150] in, The requested arrival time;

[0151] The average interval is denoted as .

[0152] The K-means algorithm (where K is the square root of the number of candidate servers and then rounded down) is used for clustering to obtain multiple load-related groups.

[0153] Step 106: For each load-related group, calculate the difference between the predicted value of the server outbound bandwidth demand in the group and the remaining capacity of the shared link in the group. By comparing the size of the difference, determine whether there is a risk of link saturation in the current scheduling scheme.

[0154] The network monitoring module extracts link segments with high congestion risk. It then obtains the target content identifiers carried by these high-congestion-risk link segments. For each target content identifier, it retrieves a candidate list of associated content delivery servers. Based on historical traffic records, it calculates the predicted outbound bandwidth demand for each server in the candidate list. It obtains the remaining capacity of each shared link on the content distribution path. It subtracts the predicted outbound bandwidth demand from the corresponding remaining capacity of the shared link for each server to obtain a difference sequence. By comparing the differences in the difference sequence, it determines whether the current scheduling scheme has a link saturation risk. If a link saturation risk is identified, it uses k-means clustering to perform outbound load grouping based on the predicted outbound bandwidth demand of the candidate content delivery server list, resulting in multiple outbound load groups. For each outbound load group, it obtains the minimum remaining capacity of the shared links for the servers within that group. It sorts the outbound load groups according to the minimum remaining capacity of the shared links within each group, resulting in a sequence of load groups with decreasing remaining capacity. Finally, it selects the servers from the first few load groups in the sequence of load groups with decreasing remaining capacity to form a set of servers with low saturation risk. The high-risk servers in the original content distribution path are replaced with a set of low-saturation risk servers to obtain the updated set of content distribution paths.

[0155] Specifically, the load-related calculations are first performed for the video live streaming service group. Assuming this group contains 4 servers, the predicted outbound bandwidth requirements for each server in the next 5 minutes are 1200Mbps, 950Mbps, 1400Mbps, and 800Mbps, respectively. Using a weighted average algorithm (weights determined based on historical peak percentages, at 0.32, 0.25, 0.28, and 0.15), the predicted total outbound bandwidth requirement for the group is calculated as 1200 × 0.32 + 950 × 0.25 + ... 1400×0.28+800×0.15=384+237.5+392+120=1133.5Mbps; Next, obtain the real-time remaining capacity of the shared uplink of this group. The current total bandwidth of the link is 2000Mbps, and the used traffic is 760Mbps. Therefore, the remaining capacity is 2000-760=1240Mbps; Then calculate the difference 1133.5-1240=-106.5Mbps. This negative value indicates that the predicted demand is lower than the remaining capacity, and the link is not at risk of saturation. The same process is then applied to the game-related load group. The predicted outbound bandwidth requirements for the six servers in this group are 480Mbps, 620Mbps, 550Mbps, 710Mbps, 390Mbps, and 450Mbps, respectively. Using an exponential smoothing algorithm (β=0.3, based on the prediction correction of the previous three periods), the total predicted requirement for the group is calculated to be 480×0.18+620×0.22+550×0.19+710×0.16+390×0. 13 + 450 × 0.12 ≈ 86.4 + 136.4 + 104.5 + 113.6 + 50.7 + 54 = 545.6 Mbps; the total bandwidth of the shared link is 1500 Mbps, currently 980 Mbps is used, and the remaining capacity is 520 Mbps. The difference, 545.6 - 520 = 25.6 Mbps, is positive, indicating that the predicted demand exceeds the remaining capacity by 25.6 Mbps, posing a risk of link saturation. The system will trigger an alarm and initiate traffic migration or rate limiting strategies. By comparing the above differences, the live streaming group has a negative difference, indicating a safe state, while the game group has a positive difference, requiring immediate intervention. This achieves dynamic risk identification and scheduling optimization based on quantified differences.

[0156] The export bandwidth demand forecast uses exponential smoothing.

[0157]

[0158] in, This represents the actual export bandwidth for the current period.

[0159] This is the predicted value for the current period;

[0160] The smoothing coefficient is β (0 < β < 1, in this embodiment β = 0.3), and the initial predicted value can be the first actual value. For the total demand within the group, a weighted summation or direct prediction of the total amount can be used.

[0161] Step 107: If the difference exceeds the preset allowable threshold, the subset of servers with the greatest bandwidth pressure is removed from the load-related group to obtain the adjusted server allocation candidate set.

[0162] By retrieving data from a subset of servers with high bandwidth usage within load-related groups, a list of servers to be removed is determined. Based on this list, server removal operations are performed for each load-related group, resulting in a preliminary adjusted candidate set. Using this preliminary candidate set, bandwidth pressure data for each server is obtained to determine if any servers still exceed the allowable threshold. If so, a second round of filtering based on bandwidth pressure data yields a further refined candidate set. From this further refined candidate set, the load-adjusted allocation status of each server is obtained, determining the final server allocation candidate set. For this final server allocation candidate set, a pre-defined allocation strategy is used for resource matching, resulting in an optimized load balancing scheme. Based on the optimized load balancing scheme, bandwidth resources are reallocated for each server, determining the final load balancing state.

[0163] Specifically, a risk assessment was conducted for the online education live streaming load-related group. First, the predicted outbound bandwidth values ​​for the next 10 minutes of the 5 servers in the group were collected, which were 1050Mbps, 880Mbps, 1360Mbps, 720Mbps and 940Mbps respectively. The demand of each server was obtained by using a time-series forecasting method based on the ARIMA model (p=2, d=1, q=1) combined with real-time session growth rate correction. The weights were then dynamically allocated according to the load ratio within the group (the weights were 0 in sequence). Using the values ​​26, 0.21, 0.29, 0.14, and 0.10, the total predicted outbound bandwidth requirement for the group is calculated to be 1050×0.26+880×0.21+1360×0.29+720×0.14+940×0.10=273+184.8+394.4+100.8+94=1047Mbps. Subsequently, the system automatically retrieves the real-time status of the corresponding shared uplink for this group from the monitoring platform. The total bandwidth is set to 1800Mbps, and the current actual traffic usage is... The predicted demand is 1320Mbps, so the remaining available capacity is 1800-1320=480Mbps. The difference between the predicted demand and the remaining capacity is calculated as 1047-480=567Mbps. This positive difference far exceeds the system's preset tolerance threshold of 150Mbps, indicating that the link is about to experience severe saturation. At this point, the scheduling engine immediately initiates server removal logic, identifying the two servers with the highest bandwidth pressure in the group (i.e., servers with predicted demands of 1360Mbps and 1050Mbps respectively). These servers are dynamically removed from the current load-related group, forming an adjusted server allocation candidate set (the total predicted demand of the remaining three servers is 880+720+940=2540Mbps, but after reweighting, the total demand within the group drops to approximately 610Mbps). This results in the adjusted predicted demand of 610Mbps being less than the remaining capacity of 480Mbps plus the buffer released after migration, turning the difference into a negative value of approximately -30Mbps. This ensures the link regains its safety margin and avoids large-scale lag.

[0164] Step 108: Regenerate a request distribution decision table for the content based on the adjusted server allocation candidate set, and distribute the decision table to the edge scheduling node for execution through load coupling relationship.

[0165] Based on the adjusted server allocation candidate set, obtain server request distribution association data. Construct a request distribution decision table using the load coupling matrix. Perform a distribution target server matching operation on the request distribution decision table to obtain a preliminary distribution mapping relationship. Send the preliminary distribution mapping relationship to the edge scheduling node to execute the distribution instruction. Obtain the execution status data returned by the edge scheduling node and determine if there are any execution status anomalies. If there are execution status anomalies, extract the anomaly association chain from the load coupling relationship. Generate a local decision table update item based on the anomaly association chain. Overwrite the corresponding entry in the request distribution decision table with the local decision table update item to obtain the corrected request distribution decision table. Drive the edge scheduling node to complete the request routing operation using the corrected request distribution decision table to obtain the distribution execution result.

[0166] Specifically, the system generates and distributes a request distribution decision table for the adjusted server allocation candidate set. First, it extracts real-time metrics from the load coupling graph of the remaining three servers, including CPU utilization of 42%, 58%, and 31%, memory availability of 68GB, 51GB, and 79GB, and predicted new connection requests per second for the next 10 minutes of 128, 196, and 89, respectively. A weighted allocation algorithm based on load coupling is then used, where coupling is calculated using the affinity matrix of historical similar requests (hybrid push and pull requests), with coupling weights of 0.38, 0.45, and 0.17. Next, the system calculates the comprehensive capacity score for each server using the formula: Score = (1 - CPU utilization) × 0.4 + Memory availability / Total memory × 0.3 + (1 - Predicted new connection pressure / Historical peak connections) × 0. The scores for the three servers were calculated to be 0.71, 0.52, and 0.83 respectively. Then, the request distribution decision table was re-normalized based on the score ratios, with the distribution ratios adjusted to 41%, 30%, and 29%. Simultaneously, the regional latency vectors reported by the edge nodes (average latency from server A to users in North China: 18ms; server B to East China: 23ms; server C to Southwest China: 15ms) were used to adjust the decision table for regional preferences, ultimately forming a three-column decision table: server identifier, allocation ratio, and preferred regional label. The system then distributed this decision table in JSON format via a Kafka topic to all associated edge scheduling nodes. Upon receiving the table, each node immediately updated its local distribution rule configuration, ensuring that requests were accurately distributed to the adjusted server set according to a 41:30:29 ratio based on the principle of regional proximity. This ensured stable overall service quality and kept the link pressure within a safe range.

[0167] Among them, the overall capacity score The formula used to measure a server's processing power is as follows:

[0168]

[0169] in, CPU utilization;

[0170] and These are the remaining memory and total memory, respectively. To predict the pressure of new connections (the number of new connection requests expected in the next 10 minutes);

[0171] This represents the historical peak number of connections.

[0172] Weight Configured according to system requirements (taken in this embodiment) .

[0173] The load coupling matrix C records the dependency strength between servers. ij This represents the mutual influence coefficient between server i and server j due to shared users or links, which can be calculated based on the historical request co-occurrence frequency. When an edge scheduling node returns an execution status anomaly (such as server timeout or link interruption), several servers and paths with the highest correlation to the anomaly server are extracted from the matrix to form an anomaly association chain, which is used for local decision table updates.

[0174] Step 109: The actual link bandwidth utilization and server outbound traffic data after this scheduling execution are sent back to the real-time status database for subsequent time window content popularity statistics and path load trend calculation, forming a closed-loop update.

[0175] The link bandwidth utilization and server outbound traffic are fed back to the real-time status database. Link bandwidth utilization and server outbound traffic data for the most recent multiple time windows are retrieved from the real-time status database. Content popularity statistics are calculated using the data from the most recent multiple time windows. Path load trend values ​​are calculated using the content popularity statistics and historical path load data. The server allocation candidate set for the current time window is adjusted based on the path load trend values. Target server matching information for each request is extracted from the adjusted server allocation candidate set. A request distribution decision table for the current time window is generated using the target server matching information. The request distribution decision table is sent to the edge scheduling node for distribution operations. Link bandwidth utilization and server outbound traffic data after this distribution are retrieved from the edge scheduling node. The link bandwidth utilization and server outbound traffic data after this distribution are fed back to the real-time status database to update the status records. The link bandwidth utilization and server outbound traffic data in the updated status records are used to determine whether the path load trend value exceeds a preset threshold. If the path load trend value exceeds the preset threshold, the server with the highest current load is removed from the server allocation candidate set to obtain a new candidate set. The request distribution decision table for the next time window is regenerated using the new candidate set.

[0176] Specifically, after this scheduling execution is completed, the system automatically transmits the actual link bandwidth utilization and server egress traffic data back to the real-time status database to support subsequent content popularity statistics and path load trend calculations within the time window, forming a closed-loop update mechanism. In the specific implementation process, the system first collects current link bandwidth utilization data from edge nodes. For example, the utilization rates of the three main links are 65.3%, 72.8%, and 54.6%, respectively. It also records the egress traffic data of each server, such as 12.5Gbps for server X, 15.2Gbps for server Y, and 9.8Gbps for server Z. This data is uploaded to the database in batches every 5 minutes via an internal API interface. Subsequently, the system uses a time series analysis algorithm to process the transmitted data, calculates the fluctuation trend of bandwidth utilization over the past hour, and obtains an average fluctuation rate of 8.2%. Combined with historical data, it predicts that the peak traffic in the next 30 minutes may reach 18.6Gbps, providing a basis for path load trend calculations. Simultaneously, based on content popularity statistics requirements, the system correlates traffic data with content access logs, analyzing and determining that a certain popular video content accounted for 35.7% of requests. Using a weighted average algorithm (current data with a weight of 0.6 and historical data with a weight of 0.4), its popularity index is calculated to be 82.4, thus updating the priority queue of the content distribution strategy. Furthermore, to ensure the integrity of the closed-loop update logic, the system integrates the above analysis results with the business monitoring module, automatically generating load anomaly threshold alarm rules. For example, when the link utilization exceeds 80%, an alarm signal is triggered, and relevant data is synchronized to the decision engine for adjusting input parameters in the next round of scheduling, forming a data-driven optimization cycle.

[0177] In some embodiments, the streaming media server resource scheduling system supporting dynamic load balancing mainly includes:

[0178] The request data collection and popularity analysis module is used to obtain the playback content identifiers and request source geographical locations of all user requests within the current time window. It obtains the current request frequency distribution of each content through content identifier statistics and determines the content popularity ranking list for the current time window based on the frequency distribution.

[0179] The popular content path mapping module is used to extract the corresponding edge access area from the geographical location of the user request source for the top-ranked content in the content popularity ranking list, and obtain the set of transmission paths that each popular content currently mainly depends on through a pre-established region-to-backbone link mapping table.

[0180] The path load trend calculation module is used to collect real-time bandwidth utilization and queue latency data of each transmission path in the most recent collection period from the network monitoring system, and calculate the short-term load trend value and rise rate value of each path through exponential weighted moving average.

[0181] The congestion risk assessment module is used to match the main transmission path set of popular content with the short-term load trend value and rise rate value of the corresponding path. By setting bandwidth utilization grading threshold and rise rate threshold, it can determine whether there is at least one path in a high congestion risk state.

[0182] The server clustering and grouping module is used to determine whether there are high congestion risk paths. If so, content popularity clustering is performed on the current set of candidate servers associated with the content. Based on the similarity of the geographical location of the user request and the relevance of the content request time, the candidate servers are divided into multiple load-related groups.

[0183] The link saturation risk assessment module is used to calculate the difference between the predicted value of the server outbound bandwidth demand in each load-related group and the remaining capacity of the shared link in the group. By comparing the size of the difference, it is determined whether there is a link saturation risk in the current scheduling scheme.

[0184] The server set adjustment module is used to determine whether the difference exceeds the preset allowable threshold. If it does, the subset of servers with the greatest bandwidth pressure is removed from the load-related group to obtain the adjusted server allocation candidate set.

[0185] The distribution decision generation and distribution module is used to regenerate the request distribution decision table for the content based on the adjusted server allocation candidate set, and distribute the decision table to the edge scheduling node for distribution through load coupling relationship;

[0186] The scheduling feedback and closed-loop update module is used to send the actual link bandwidth utilization and server outbound traffic data after the current scheduling execution back to the real-time status database for subsequent time window content popularity statistics and path load trend calculation, forming a closed-loop update.

[0187] If the technical solution of this application involves the processing of personal information, the relevant products have established a sound user authorization mechanism: before collecting, using, or sharing personal information, the obligation to inform is fulfilled in accordance with the law, and the individual's voluntary and explicit consent is obtained; if sensitive personal information is involved, the user's separate and explicit consent is further obtained. Specific measures include, but are not limited to: setting up prominent prompts in the information collection area, or clearly displaying the processing rules (including the processor, purpose, method, information type, etc.) through electronic interfaces such as pop-ups, checkboxes, and active submissions, to ensure that users voluntarily authorize based on their knowledge. All personal information processing activities strictly comply with national laws and regulations, and effectively protect the legitimate rights and interests of personal information subjects.

[0188] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A streaming media server resource scheduling method supporting dynamic load balancing, characterized in that, include: Get the playback content identifiers and request source geolocations of all user requests within the current time window. Calculate the current request frequency distribution of each content based on the content identifiers, and determine the content popularity ranking list for the current time window based on the frequency distribution. For the top-ranked content in the popularity ranking list, the corresponding edge access area is extracted from the geographical location of the user request source. The set of transmission paths that each popular content currently relies on is obtained through a pre-established region-to-backbone link mapping table. The network monitoring system collects real-time bandwidth utilization and queue latency data for each transmission path within the most recent collection period, and calculates the short-term load trend value and rate of increase value for each path using an exponentially weighted moving average. The main transmission path set of popular content is matched with the short-term load trend value and rise rate value of the corresponding path. By setting bandwidth utilization level threshold and rise rate threshold, it is determined whether there is at least one path in a high congestion risk state. If a high congestion risk path is determined, content popularity clustering is performed on the candidate set of servers currently associated with the content. The candidate servers are divided into multiple load-related groups according to the similarity of user request geographical location and the correlation of content request time. For each load-related group, calculate the difference between the predicted value of the server outbound bandwidth demand in the group and the remaining capacity of the shared link in the group. By comparing the size of the difference, determine whether there is a risk of link saturation in the current scheduling scheme. If the difference exceeds the preset allowable threshold, determine whether the difference exceeds the preset allowable threshold. If it does, remove the subset of servers with the largest bandwidth pressure from the load-related group to obtain the adjusted server allocation candidate set. Based on the adjusted server allocation candidate set, a request distribution decision table for this content is regenerated, and the decision table is sent to the edge scheduling node for distribution through load coupling relationship; The actual link bandwidth utilization and server outbound traffic data after this scheduling execution are sent back to the real-time status database for subsequent time window content popularity statistics and path load trend calculation, forming a closed-loop update.

2. The streaming media server resource scheduling method supporting dynamic load balancing according to claim 1, characterized in that, The process of obtaining the playback content identifiers and request source geolocations of all user requests within the current time window, statistically analyzing the content identifiers to obtain the current request frequency distribution for each content, and determining the content popularity ranking list for the current time window based on the frequency distribution includes: Obtain the playback content identifier and request source geolocation data of all user requests within the current time window from the system log. Remove invalid records through data cleaning to obtain a preliminary user request dataset. Based on the initially compiled user request dataset, the playback content is grouped and statistically analyzed according to the content identifier, and the request frequency corresponding to each content identifier is calculated to obtain the request frequency distribution results for each content. Based on the request frequency distribution results and combined with geographic location data, the request sources of each content identifier are divided into regions. The contribution of different geographic locations to the content request frequency is analyzed, and the popularity distribution of each content in different regions is determined. Based on the distribution of popularity, each content identifier is sorted from high to low according to the request frequency to generate a content popularity ranking list for the current time window; If the popularity of a certain content identifier is higher than a preset threshold in multiple geographical locations, it will be marked as high-popularity content, and a sublist of high-popularity content will be generated. Based on the high-popularity content sublist and the overall popularity ranking list, and combined with the dynamic changes of the time window, the priority of content identifiers is adjusted to obtain the final optimized content recommendation ranking result.

3. A streaming media server resource scheduling method supporting dynamic load balancing according to claim 1, characterized in that, For the top-ranked content in the popularity ranking list, the corresponding edge access regions are extracted from the geographical location of the user request source. A pre-established region-to-backbone link mapping table is used to obtain the set of primary transmission paths currently relied upon by each popular piece of content, including: Extract the top-ranked popular content from the content popularity ranking list, obtain the corresponding user request data, and combine it with geographical location information to determine the main request source region for each popular content. For the main request source region, the corresponding edge access region is obtained through pre-established regional division rules, and the access distribution of each popular content in different regions is obtained; Based on the data from the edge access areas, a pre-established mapping table is used to obtain the backbone link information corresponding to each access area, and to determine the set of transmission paths that popular content depends on in different areas. By analyzing the traffic distribution of each popular content on different backbone links through the transmission path set, the load distribution data of each path is obtained. If the load distribution data of a certain transmission path exceeds the preset threshold, traffic diversion processing is performed on that path to obtain the adjusted path allocation scheme and determine the optimized transmission path set. Based on the optimized set of transmission paths, the regional access and link mapping data of popular content are updated to obtain the final path allocation result.

4. A streaming media server resource scheduling method supporting dynamic load balancing according to claim 1, characterized in that, The process involves collecting real-time bandwidth utilization and queue latency data for each transmission path from the network monitoring system within the most recent collection period, and calculating the short-term load trend and rate of increase for each path using an exponentially weighted moving average, including: Obtain bandwidth utilization and queue latency data for each transmission path within the most recent period from the network monitoring system; The short-term load trend value for each path is obtained by calculating the bandwidth utilization sequence using an exponentially weighted moving average. The load rise rate of each path is obtained by calculating the queue delay sequence using an exponentially weighted moving average. Based on the short-term load trend value and the load rise rate value, determine whether the path is currently in a low load stable state, a medium load rising state, or a high load rapid rise state. For paths experiencing moderate load increases, obtain the percentage of popular content traffic currently carried by that path; For paths that are experiencing a rapid increase in high load, obtain detailed data on all content traffic currently carried by that path; If a path is in a state of high load and rapid increase, select the top three content items with the highest traffic share from the content traffic details data currently carried by that path and determine them as priority traffic diversion targets; For priority traffic diversion targets, select available backup paths from the set of other paths with lower load rise rates; Determine the offloading reception capacity of each backup path based on the short-term load trend value of the backup path; The distribution ratio of priority traffic to each backup path is obtained by calculating the traffic share and the traffic receiving capacity. The traffic allocation scheme for each transmission path is updated using the traffic splitting ratio allocation result to obtain the adjusted path traffic distribution data; Based on the adjusted path traffic distribution data, update the predicted load reference value for the corresponding path in the network monitoring system.

5. A streaming media server resource scheduling method supporting dynamic load balancing according to claim 1, characterized in that, The process involves matching the main transmission paths of popular content with the short-term load trend and rise rate values ​​of the corresponding paths. By setting bandwidth utilization grading thresholds and rise rate thresholds, it is determined whether at least one path is in a high-congestion risk state. If a high-congestion risk path is identified, content popularity clustering is performed on the candidate server set currently associated with the content. Based on the similarity of user request geographical location and the relevance of content request time, the candidate servers are divided into multiple load-related groups, including: Obtain the main transmission paths of popular content from the network monitoring system; For each path in the main transmission path set, obtain the corresponding short-term load trend value and load rise rate value; The bandwidth utilization sequence of each path is divided into states by a preset bandwidth utilization grading threshold to obtain the current load level of each path. The load increase rate of each path is compared using a preset increase rate threshold to determine whether the load increase exceeds the threshold. If the load level of a certain path reaches high load and the load increase rate exceeds the threshold, then the path is determined to be in a high congestion risk state. For paths at high congestion risk, obtain the percentage of traffic from all popular content currently carried by that path; Popular content is sorted based on traffic share data, and the top three most popular content items are selected as priority traffic diversion targets. Paths with load increase rates lower than a preset threshold are selected from other path sets to obtain a set of available backup paths; The current load status of each backup path is assessed by using short-term load trend values ​​to determine the offloading and receiving capacity of each backup path. The proportion of traffic diverted to each backup path is calculated based on the traffic share of the priority diversion target and the diversion receiving capacity of each backup path. The traffic allocation scheme for each transmission path in the network is updated by using the traffic splitting ratio to obtain the adjusted path traffic distribution data; The predicted load reference value for the corresponding path in the network monitoring system is updated by adjusting the path traffic distribution data. If a high-congestion-risk path is identified, content popularity clustering is performed on the candidate server set currently associated with the content. Based on the similarity of user request geographic location and the relevance of content request time, the candidate servers are divided into multiple load-related groups, including: Obtain target content information carried on high-congestion-risk paths from the network monitoring system; Obtain a set of candidate servers associated with the target content information; Clustering is performed on the candidate server set based on the similarity of user requests to obtain geographically related groups; Adjustments are made to the geographic load-related groups based on the content request time relevance to obtain a time-geographic fusion load-related group set; Obtain the load status data of each group in the load-related group set; The remaining processing capacity of servers in each group is sorted based on load status data to determine the priority server sequence for load sharing. Select at least three servers with the highest remaining processing capacity from the priority sharing server sequence to form the target server set for traffic diversion; The content distribution path set is updated by using the target server set for traffic splitting to obtain the adjusted content distribution path set; Modify the request scheduling strategy by adjusting the content distribution path set, and update the traffic guidance rules for the target content in the network.

6. A streaming media server resource scheduling method supporting dynamic load balancing according to claim 1, characterized in that, The method involves calculating the difference between the predicted server outbound bandwidth demand and the remaining capacity of the shared links within each load-related group. By comparing the magnitude of this difference, it is determined whether the current scheduling scheme carries a risk of link saturation. If the difference exceeds a preset tolerance threshold, it is further determined whether the difference exceeds the preset tolerance threshold. If it does, the subset of servers with the highest bandwidth pressure is removed from the load-related groups, resulting in an adjusted set of server allocation candidates, including: Extract link segments with high congestion risk from the network monitoring module; Obtain the target content identifier carried by the high-congestion-risk link segment; Retrieve a list of candidate content distribution servers associated with the target content identifier; The predicted outbound bandwidth demand is calculated for each server in the candidate list of content delivery servers based on historical traffic records. Obtain the remaining capacity value of each shared link on the content distribution path; The difference sequence is obtained by subtracting the predicted outbound bandwidth demand of each server from the remaining capacity of the corresponding shared link. By comparing the magnitudes of the differences in the difference sequence, it can be determined whether the current scheduling scheme has the risk of link saturation. If it is determined that there is a risk of link saturation, the k-means clustering method is used to perform outbound load grouping on the predicted outbound bandwidth demand of the content delivery server candidate list to obtain multiple outbound load groups. For each outbound load group, obtain the minimum remaining capacity of the shared link for the servers within the group; The load groups at each egress point are sorted according to the minimum remaining capacity of the shared links within the group, resulting in a sequence of load groups with the largest remaining capacity in descending order. Servers from the first few load groups in descending order of remaining capacity are selected to form a set of servers with low saturation risk. The high-risk servers in the original content distribution path are replaced with a set of low-saturation risk servers to obtain an updated set of content distribution paths. If the difference exceeds a preset allowable threshold, the subset of servers with the highest bandwidth pressure is removed from the load-related group to obtain an adjusted server allocation candidate set, including: By obtaining data on a subset of servers with high bandwidth usage from load-related groups, a list of servers to be removed can be determined. Based on the list of servers to be removed, perform server removal operations for load-related groups to obtain a preliminary adjusted candidate set; Using the initially adjusted candidate set, the bandwidth pressure data of each server is obtained to determine whether there are any servers that still exceed the allowable threshold. If there are still servers that exceed the allowable threshold, a second screening is performed on the bandwidth pressure data to obtain a further streamlined candidate set. By further refining the candidate set, the allocation status of each server after load adjustment is obtained, and the final server allocation candidate set is determined. For the final set of server allocation candidates, a preset allocation strategy is used to match resources and obtain an optimized load balancing scheme. Based on the optimized load balancing scheme, bandwidth resources are redistributed for each server to determine the final load balancing state.

7. A streaming media server resource scheduling method supporting dynamic load balancing according to claim 1, characterized in that, The step of regenerating a request distribution decision table for the content based on the adjusted server allocation candidate set, and then distributing the decision table to the edge scheduling node for execution through load coupling, includes: Obtain server request distribution related data based on the adjusted server allocation candidate set; Construct a request distribution decision table using a load coupling matrix; Perform a distribution target server matching operation on the request distribution decision table to obtain a preliminary distribution mapping relationship; The initial distribution mapping relationship is sent to the edge scheduling node to execute the distribution instruction; Obtain the execution status data returned by the edge scheduling node to determine if there are any execution status anomalies; If there is an execution status anomaly, extract the anomaly association chain from the load coupling relationship; Generate local decision table update items based on abnormal association chains; The corresponding entries in the request distribution decision table are overwritten by updating the local decision table to obtain the corrected request distribution decision table. The modified request distribution decision table drives the edge scheduling nodes to complete the request routing operation and obtain the distribution execution result.

8. A streaming media server resource scheduling method supporting dynamic load balancing according to claim 1, characterized in that, The step of transmitting the actual link bandwidth utilization and server outbound traffic data after this scheduling execution back to the real-time status database for subsequent time window content popularity statistics and path load trend calculations, forming a closed-loop update, includes: Data is fed back to the real-time status database based on link bandwidth utilization and server outbound traffic. Obtain link bandwidth utilization and server egress traffic data for the most recent multiple time windows from the real-time status database; Calculate content popularity statistics using data from multiple recent time windows; The path load trend value is calculated using content popularity statistics and historical path load data. Adjust the server allocation candidate set for the current time window based on path load trend values; Extract the target server matching information for each request from the adjusted server allocation candidate set; A request distribution decision table for the current time window is generated by matching information from the target server. The request distribution decision table is sent to the edge scheduling node to perform the distribution operation; Obtain the link bandwidth utilization and server egress traffic data after the completion of this distribution from the edge scheduling node; After the distribution is completed, the link bandwidth utilization and server outbound traffic data will be sent back to the real-time status database to update the status records. Determine whether the path load trend value exceeds the preset threshold based on the link bandwidth utilization and server outbound traffic data in the updated status record; If the path load trend value exceeds the preset threshold, the server with the highest current load is removed from the server allocation candidate set to obtain a new candidate set; The request distribution decision table for the next time window is regenerated using the new candidate set.

9. A streaming media server resource scheduling system supporting dynamic load balancing, characterized in that, The system includes: The request data collection and popularity analysis module is used to obtain the playback content identifiers and request source geographical locations of all user requests within the current time window. It obtains the current request frequency distribution of each content through content identifier statistics and determines the content popularity ranking list for the current time window based on the frequency distribution. The popular content path mapping module is used to extract the corresponding edge access area from the geographical location of the user request source for the top-ranked content in the content popularity ranking list, and obtain the set of transmission paths that each popular content currently mainly depends on through a pre-established region-to-backbone link mapping table. The path load trend calculation module is used to collect real-time bandwidth utilization and queue latency data of each transmission path in the most recent collection period from the network monitoring system, and calculate the short-term load trend value and rise rate value of each path through exponential weighted moving average. The congestion risk assessment module is used to match the main transmission path set of popular content with the short-term load trend value and rise rate value of the corresponding path. By setting bandwidth utilization grading threshold and rise rate threshold, it can determine whether there is at least one path in a high congestion risk state. The server clustering and grouping module is used to determine whether there are high congestion risk paths. If so, content popularity clustering is performed on the current set of candidate servers associated with the content. Based on the similarity of the geographical location of the user request and the relevance of the content request time, the candidate servers are divided into multiple load-related groups. The link saturation risk assessment module is used to calculate the difference between the predicted value of the server outbound bandwidth demand in each load-related group and the remaining capacity of the shared link in the group. By comparing the size of the difference, it is determined whether there is a link saturation risk in the current scheduling scheme. The server set adjustment module is used to determine whether the difference exceeds the preset allowable threshold. If it does, the subset of servers with the greatest bandwidth pressure is removed from the load-related group to obtain the adjusted server allocation candidate set. The distribution decision generation and distribution module is used to regenerate the request distribution decision table for the content based on the adjusted server allocation candidate set, and distribute the decision table to the edge scheduling node for distribution through load coupling relationship; The scheduling feedback and closed-loop update module is used to send the actual link bandwidth utilization and server outbound traffic data after the current scheduling execution back to the real-time status database for subsequent time window content popularity statistics and path load trend calculation, forming a closed-loop update.

Images