A distributed information publishing dynamic scheduling system, method and storage medium

Abstract

The application relates to the technical field of distributed information publishing, and discloses a distributed information publishing dynamic scheduling system and method and a storage medium, wherein the method comprises the following steps: acquiring a set of information to be published and extracting associated attributes; generating a publishing sequence dependency graph and calculating the cascade influence weight of each node through reverse topological traversal; collecting the capacity state vector of each downstream distribution channel and predicting the available receiving rate; estimating the end-to-end completion delay of the cascade link; identifying the bottleneck node and generating a channel rerouting mark; performing priority scheduling with cascade perception; performing whole-chain delay propagation on the node that cannot be distributed on time; and performing distribution and feeding back updated scheduling parameters.

Description

Technical Field

[0001] This invention relates to the field of distributed information publishing technology, specifically to a distributed information publishing dynamic scheduling system, method, and storage medium. Background Technology

[0002] In scenarios such as natural disaster warnings, public safety announcements, and financial market anomaly notices, the same emergency may trigger multiple related messages with a release order dependency. These messages need to be processed by the publishing node before being finally distributed through multiple downstream distribution channels.

[0003] In existing schemes, the cascading scheduling scheme and the downstream traffic shaping scheme make independent decisions. When allocating low-latency nodes for preceding information, the cascading scheduling scheme is unaware of the capacity status of downstream channels; when performing flow limiting, the downstream traffic shaping scheme is unaware of the cascading dependencies between information.

[0004] The aforementioned isolated operation mode leads to the following technical problems: when the downstream channel corresponding to the preceding information experiences capacity contraction, the entire cascaded link fails due to the obstruction of the distribution of the preceding information; at the same time, downstream traffic shaping may delay the preceding information while allowing the subsequent information to pass during traffic throttling, making the subsequent information ineffective for the audience because the preceding information has not arrived, resulting in a combined failure of wasted channel bandwidth and cascaded propagation failure. Summary of the Invention

[0005] This invention provides a distributed dynamic scheduling system, method, and storage medium for publishing information, which solves the technical problem in related technologies that causes end-to-end completion delay exceeding limits and scheduling benefits to be lost in multi-channel distribution scenarios of information related to sudden events due to channel congestion and failure to perceive cascading dependencies.

[0006] This invention discloses a distributed dynamic scheduling method for publishing information, comprising: acquiring a set of information to be published, extracting content category identifier, generating timestamp, preset distribution deadline, event association identifier, and target distribution area or platform attribute identifier for each piece of information, forming a structured information attribute record; Based on the event association identifier, the information to be published is grouped to generate a directed acyclic publishing order dependency graph. A reverse topological traversal is performed along the publishing order dependency graph, and the cascading influence weight is calculated for each information node. The real-time throughput, current queuing depth, response delay, and rate limiting status code of each downstream distribution channel in the most recent sampling windows are collected, aggregated into a capacity status vector, and extrapolated to obtain the predicted available receiving rate of each channel in the next few scheduling cycles. The available receiving rate after adjustment is obtained by downgrading the congested channel according to the attenuation coefficient. Based on the target distribution region or platform attribute identifier, the information to be published is mapped to the corresponding downstream distribution channel. The larger of the queuing delay of the publishing node and the queuing delay of the downstream channel is accumulated along the topological order of the publishing order dependency graph to obtain the end-to-end completion delay of each level of the link. Information nodes whose end-to-end completion delay exceeds the preset distribution deadline are identified as bottleneck information nodes. The source of the bottleneck is determined based on the queuing delay of the publishing node and the queuing delay of the downstream channel. Alternative channels covering the same target are searched for the bottleneck nodes of the downstream channels. When the alternative end-to-end delay is lower than the original end-to-end completion delay, a channel rerouting mark is generated. With the optimization objective of maximizing the total scheduling benefit weighted by the global cascading impact weight, the combination of publishing nodes and channels is allocated according to the cascading impact weight from high to low. For information nodes that cannot be distributed on time, the delay mark is propagated downward along the entire publishing order dependency graph and the corresponding downstream channel capacity quota is released. The distribution is performed, and the attenuation coefficient and the propagation attenuation coupling coefficient in the calculation of the cascade influence weight are updated based on the deviation between the actual receiving rate and the predicted value, and the deviation between the actual completion rate and the predicted completion rate of the cascade link.

[0007] Furthermore, the weight of the cascading effect is calculated using the following formula: in, Indicates information node Cascade effect weights, Indicates information node Its own initial value weight, Indicates information node The set of direct successor nodes, Represents a set One of the successor node elements, Represents the propagation attenuation coupling coefficient. Indicates the successor node Cascade effect weights; The initial value weight of itself The content category identifier in the information attribute record and the preset distribution deadline are jointly determined. The higher the category priority corresponding to the content category identifier and the more urgent the preset distribution deadline, the better. The larger the value, the better.

[0008] Furthermore, the step of extrapolating to obtain the predicted value of the available receiving rate includes: performing Z-score normalization on the real-time throughput, queuing depth, response delay and rate limiting status code in the capacity state vector respectively; encoding the rate limiting status code as a binary value of 0 or 1 to participate in the normalization; inputting the normalized real-time throughput sequence into a sliding window linear regression algorithm; and extrapolating the predicted value of the available receiving rate for several future scheduling cycles using the real-time throughput of the most recent sampling windows as the fitting input.

[0009] Furthermore, the step of adjusting the available reception rate by reducing the congested channel according to the attenuation coefficient includes: when the queuing depth of a channel exceeds a preset proportional threshold of its buffer capacity, or when a rate-limiting status code appears consecutively within the sampling window, the adjusted available reception rate is generated according to the following formula: in, Indicates the first The available receiving rate after adjustment over one scheduling cycle, Indicates the scheduling cycle number. This represents the rate attenuation factor for a congested channel. Indicates the first The original predicted rate for each scheduling cycle.

[0010] Furthermore, after generating the adjusted available receive rate sequence, the absolute difference between the adjusted available receive rates of two adjacent scheduling periods is calculated. When the absolute difference exceeds the stability threshold, the predicted value of the current scheduling period is replaced with the arithmetic mean of the predicted values ​​of the most recent scheduling periods, and then used for the estimation of end-to-end completion delay.

[0011] Furthermore, the end-to-end completion delay is calculated node-by-node using the following formula: in, Indicates the first on the link End-to-end cumulative latency of each node, This represents the end-to-end cumulative delay of its preceding nodes, with the link starting point being... , This represents the initial value of the end-to-end cumulative delay at the starting point of the link. This represents the queuing latency of the node on the publishing node, and is obtained by the ratio of the current task queue length of the publishing node to the processing rate of the node's information. This represents the queuing delay of the downstream channel corresponding to the node, which is obtained by the ratio of the current queuing depth of the downstream channel corresponding to the node to the adjusted available receiving rate.

[0012] Furthermore, the step of searching for alternative channels covering the same target includes: firstly, sorting the alternative channels in descending order according to the adjusted available receiving rate based on the target distribution area or platform attribute identifier; calculating the alternative end-to-end delay only for the top-ranked alternative channels; the calculation method of the alternative end-to-end delay is consistent with the end-to-end completion delay; and replacing the downstream channel queuing delay of the original channel with the queuing delay corresponding to the alternative channel and then substituting it back in.

[0013] Furthermore, the step of allocating publishing nodes and channel combinations according to the cascading influence weight from high to low includes: prioritizing the allocation of publishing nodes and channel combinations with low processing latency and sufficient downstream channel capacity to the preceding information with the highest cascading influence weight; When the capacity of the preferred channel for the preceding information is insufficient, the preceding information is switched to the alternative channel according to the channel rerouting mark. Publishing tasks that exceed the total adjusted available receive rate of all reachable channels are marked as delayed scheduling tasks, and delayed scheduling queues are generated by sorting them according to the cascading impact weights.

[0014] This invention provides a distributed dynamic scheduling system for publishing information, including: an information attribute extraction module, used to obtain a set of information to be published and extract content category identifiers, generated timestamps, preset distribution deadlines, event association identifiers, and target distribution area or platform attribute identifiers; The cascade weighting module is used to generate a publishing order dependency graph and calculate the cascade impact weights by traversing the reverse topology; the channel capacity prediction module is used to collect the capacity state vector and extrapolate it to obtain the adjusted available receiving rate. The delay estimation module is used to accumulate queuing delays along the topology order to obtain the end-to-end completion delay of each level of the interconnection link; the bottleneck identification and rerouting module is used to identify bottleneck information nodes and generate channel rerouting tags. The cascading awareness scheduling module is used to allocate publishing node and channel combinations according to cascading impact weights, and to mark the overall propagation delay of nodes that cannot be distributed on time along the publishing order dependency graph. The distribution and feedback module is used to perform distribution and update the attenuation coefficient and propagation attenuation coupling coefficient based on the deviation.

[0015] This invention incorporates the topological constraints of the publishing order dependency graph and the capacity prediction of the downstream distribution channel into a unified end-to-end completion delay estimation, enabling bottleneck identification to simultaneously reflect both the processing delay of the publishing node and the queuing delay of the downstream channel. This allows it to identify the composite bottleneck of the failure of the entire cascaded link caused by the congestion of the downstream channel corresponding to the preceding information.

[0016] The cascading influence weight accumulates the propagation value of successor nodes in a reverse topology manner. The scheduling process can prioritize the processing and distribution of preceding information resources, avoiding the failure of the entire link caused by the erroneous delay of preceding information due to traffic shaping.

[0017] The channel rerouting tag is generated based on the comparison of alternative channels and alternative end-to-end delays covering the same audience, so that preceding information can still switch to reachable channels when the capacity of the preferred channel shrinks, ensuring that the publishable window of subsequent dependent information is not compressed.

[0018] The whole-chain delayed propagation moves the preceding information that cannot be distributed on time, along with all its subsequent information, into the delayed scheduling queue and releases the channel capacity quota, thus avoiding the waste of channel bandwidth caused by downstream traffic shaping allowing subsequent information to pass without knowing the dependency relationship.

[0019] By analyzing the attenuation coefficient Coupling coefficient with propagation attenuation The deviation feedback update allows the scheduling parameters to be continuously corrected according to the dynamic changes in channel capacity and the completion status of cascading, integrating cascading scheduling and channel traffic shaping into a two-way constrained collaborative scheduling process, thus alleviating the compound failure problem caused by the isolated operation of the two. Attached Figure Description

[0020] Figure 1 This is a flowchart of the distributed dynamic scheduling method for publishing information provided in an embodiment of the present invention; Figure 2 This is a schematic diagram comparing the cascading influence weights of each node in the cascading link provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the available receiving rate for each downstream distribution channel in three scheduling cycles provided in an embodiment of the present invention; Figure 4 This is a schematic diagram comparing the queuing delay of each node publishing node with the queuing delay of the downstream channel, provided in an embodiment of the present invention. Figure 5 This is a schematic diagram comparing the end-to-end cumulative delay of the cascaded link with the preset remaining time before the deadline, provided in an embodiment of the present invention. Figure 6 This is a schematic diagram comparing the end-to-end latency of MSG001 and MSG003 before and after rerouting, provided in an embodiment of the present invention. Figure 7 This is a schematic diagram comparing the actual completion time and the estimated cumulative time of each piece of information provided in the embodiments of the present invention; Figure 8 This is a schematic diagram comparing the actual receiving rate of each channel with the original predicted rate provided in the embodiments of the present invention; Figure 9 This is a schematic diagram showing the distribution of remaining deadline times for each node in the cascaded link provided in an embodiment of the present invention. Detailed Implementation

[0021] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0022] Example According to an embodiment of the present invention, a distributed information publishing collaborative scheduling method is as follows: Figure 1 As shown, it includes the following steps: Step 1: Obtain the set of information to be published and extract related attributes; Obtain the set of information to be published submitted by each publishing source in the distributed publishing platform, and extract the content category identifier, generation timestamp, preset distribution deadline, event association identifier, and target distribution area or platform attribute identifier for each piece of information in the set to form a structured information attribute record.

[0023] Step 2: Generate a release order dependency graph and calculate the weights of cascading effects; Based on event association identifiers, the information to be published is grouped, and the content dependency declaration fields of the information within each group are parsed to generate a directed acyclic publishing order dependency graph. Nodes in the publishing order dependency graph correspond to each associated piece of information, and directed edges represent preceding publishing constraints.

[0024] Perform a reverse topology traversal along the release order dependency graph, and calculate the cascaded impact weight for each node. : in, Indicates information node Cascade effect weights, Indicates information node Its own initial value weight, Indicates information node The set of direct successor nodes, Represents the set of direct successor nodes One of the successor node elements, Represents the propagation attenuation coupling coefficient. Indicates the successor node The weight of cascading effects. The weight of its own initial value. The content category identifier in the information attribute record and the preset distribution deadline are jointly determined. The higher the category priority corresponding to the content category identifier and the more urgent the preset distribution deadline, the better. The larger the value, the better.

[0025] It should be noted that content category identifiers are non-numerical classification data, and therefore require encoding to map category priorities to numerical values ​​before participation. The calculation involves two time-based parameters: the preset distribution deadline and the generated timestamp. The difference between the two yields the remaining deadline, which is a numerical value that can be directly used in the calculation. The calculation requires no additional dimension handling.

[0026] Reverse topological traversal begins with the leaf nodes (nodes without successors) of the published order dependency graph. Directly equal to The weighted weights of successor nodes are accumulated layer by layer towards the root node until all nodes have reached their weighted weights. Calculation complete.

[0027] Step 3: Collect channel capacity status and predict available receive rate; The collection of data from various downstream distribution channels in the most recent Real-time throughput, current queuing depth, response latency, and rate limiting status codes within each sampling window are aggregated into a capacity status vector for each channel, categorized by channel identifier. This indicates the number of sampling windows participating in capacity status acquisition.

[0028] The real-time throughput sequence in the capacity state vector is input into a sliding window linear regression algorithm, with the most recent The real-time throughput of each sampling window is used as the fitting input, and extrapolation is used to obtain the future throughput of each channel. The predicted available receive rate over a scheduling period, of which This indicates the number of scheduling cycles covered by the extrapolation prediction.

[0029] It should be noted that the real-time throughput, queuing depth, response latency, and rate limiting status code in the capacity state vector have different dimensions. Before inputting the capacity state vector into the sliding window linear regression algorithm, each component is Z-score standardized to eliminate the influence of the difference in dimensions on the regression fit. The rate limiting status code is categorical data and is encoded as a binary value of 0 or 1 before being standardized.

[0030] When the queuing depth of a channel exceeds a preset percentage threshold of its buffer capacity, or when a rate-limiting status code appears consecutively within the sampling window, the predicted available receiving rate for that channel is reduced by an attenuation coefficient, generating an adjusted available receiving rate sequence for each channel. Let the queuing depth of a certain channel be... The original prediction rate for each scheduling cycle is The attenuation coefficient is The adjusted usable receiving rate is: in, Indicates the first The available receiving rate after adjustment over one scheduling cycle, Indicates the scheduling cycle number. This represents the rate attenuation factor for a congested channel. Indicates the first The original predicted rate for each scheduling cycle.

[0031] It should be noted that the attenuation coefficient For rate prediction correction factors for congested channels, the attenuation coefficient is... The initial value is determined by statistically analyzing the deviation between the actual reception rate of the channel during historical congestion periods and the predicted value, and is updated in step 8 based on the operational deviation.

[0032] In this embodiment of the application, to avoid scheduling oscillations caused by prediction bias in scenarios with frequent fluctuations in channel capacity, after generating the adjusted available receive rate sequence, a differential stability check is performed on the predicted available receive rate values ​​for two adjacent scheduling periods. Specifically, the calculation of the... The and the first The absolute value of the difference in available receiving rate after each scheduling cycle adjustment ,in Indicates the first The adjusted available receiving rate for the [number] scheduling cycle, when the absolute value of the differential exceeds the stability threshold, will [be adjusted]. The predicted value for each scheduling cycle is replaced with the arithmetic mean of the predicted values ​​for the most recent scheduling cycles, and then used for the delay estimation in step 4.

[0033] Step 4: Estimate the end-to-end completion delay of the cascaded links; Based on the target distribution region or platform attribute identifier, each piece of information to be published is mapped to its corresponding downstream distribution channel.

[0034] For each cascaded link in the release order dependency graph, calculate the end-to-end completion delay estimate node by node along the topology. : in, Indicates the first on the link End-to-end cumulative latency of each node, This represents the end-to-end cumulative delay of its preceding nodes (taken as the link start point). , (Indicates the initial value of the end-to-end cumulative delay at the link's origin). This indicates the queuing latency of the node on the publishing node. This indicates the queuing delay of the downstream channel corresponding to this node. It is estimated by the ratio of the current task queue length of the publishing node to the processing rate of the node's information. The value is estimated by the ratio of the current queuing depth of the downstream channel corresponding to the node to the adjusted available receiving rate obtained in step 3. Both of the above are time-based and can be directly added and compared.

[0035] Step 5: Identify bottleneck nodes and generate channel rerouting tags; End-to-end completion delay Information nodes that exceed the preset distribution deadline are identified as bottleneck information nodes.

[0036] Comparison of bottleneck information nodes and Size: If If the bottleneck is significant, the bottleneck is identified as a delay in downstream channel capacity; conversely, if it is minor, the bottleneck is identified as a delay in the publishing node's processing.

[0037] For downstream bottleneck nodes, alternative channels covering the same target distribution area or platform attribute identifier are retrieved. The replacement end-to-end delay is calculated based on the adjusted usable receive rate sequence of the alternative channels. If the replacement end-to-end delay is lower than the original end-to-end completion delay, a channel rerouting marker is generated for that node. The calculation method for the replacement end-to-end delay is the same as in step 4, using the original channel's... The formula is obtained by substituting the queue delay corresponding to the alternative channel.

[0038] It should be noted that the queuing delay corresponding to the alternative channel is estimated by the ratio of the current queuing depth of the alternative channel to the adjusted available receiving rate of the alternative channel, and the calculation method is the same as in step 4. The estimation method is the same.

[0039] In this embodiment of the application, in order to reduce the search cost of rerouting decision when there are many cascaded links, when searching for alternative channels, the alternative channels are first sorted in descending order according to the adjusted available receiving rate based on the target distribution area or platform attribute identifier. Only the top few alternative channels are calculated to avoid exhaustively evaluating all alternative channels.

[0040] Step 6: Perform priority scheduling for cascaded sensing; The optimization objective is to maximize the total scheduling benefit weighted by the global cascading impact. This is achieved while simultaneously satisfying the topological order constraint of the deployment order dependency graph and the upper limit constraint of the available receiving rate after adjustments to each downstream channel. The optimization is based on the cascading impact weights. Tasks are assigned to preceding information from highest to lowest priority.

[0041] Prioritize the allocation of publishing nodes and channel combinations with low processing latency and sufficient downstream channel capacity to the preceding information with the highest cascading impact weight; when the preferred channel capacity of the preceding information is insufficient, switch the preceding information to the alternative channel according to the channel rerouting mark.

[0042] Publishing tasks that exceed the total adjusted available receive rate of all reachable channels are marked as delayed scheduling tasks, and delayed scheduling queues are generated by sorting them according to the cascading impact weights.

[0043] Step 7: Delayed propagation throughout the entire chain; For information nodes that cannot complete distribution before the preset distribution deadline due to insufficient capacity of all reachable channels, the delay mark is propagated down the publishing order dependency graph, the information node and all its successor nodes are moved into the delay scheduling queue, and the downstream channel capacity quota originally occupied by the information node is released to other non-blocking cascaded links.

[0044] Step 8: Execute the distribution and update the scheduling parameters; According to the above scheduling scheme, each piece of information to be published is pushed to the corresponding publishing node and downstream distribution channel for distribution. The actual end-to-end completion delay of each piece of information, the deviation between the actual receiving rate and the predicted value of the available receiving rate of each channel, and the overall completion rate of each level of the interconnection link are recorded.

[0045] The attenuation coefficient is updated by utilizing the deviation between the actual received rate and the predicted available received rate of the channel. The ratio of the actual receiving rate to the original predicted rate of the congested channel during the current scheduling cycle is used as the new observation value. This value is then combined with the historical attenuation coefficient using an exponentially weighted average to obtain the updated attenuation coefficient. The propagation attenuation coupling coefficient is updated by utilizing the deviation between the overall completion rate of the cascaded links and the predicted completion rate. When the actual completion rate is lower than the predicted completion rate, the propagation attenuation coupling coefficient should be appropriately increased. To increase the influence weight of successor nodes on the scheduling decisions of predecessor nodes; when the actual completion rate is higher than the predicted completion rate, the propagation attenuation coupling coefficient should be appropriately reduced. The adjustment range is positively correlated with the absolute value of the deviation.

[0046] This implementation incorporates the topological constraints of the publishing order dependency graph and the capacity prediction of the downstream distribution channel into a unified end-to-end completion delay estimation, enabling bottleneck identification to simultaneously reflect both the processing delay of the publishing node and the queuing delay of the downstream channel. Therefore, the scheduling decision has the ability to identify the composite bottleneck of "the failure of the entire cascaded link due to congestion of the downstream channel corresponding to the preceding information".

[0047] Because the cascading influence weights accumulate the propagation value of successor nodes in a reverse topology manner, the scheduling process can prioritize the processing and distribution of preceding information resources, avoiding the failure of the entire link caused by the erroneous delay of preceding information due to traffic shaping.

[0048] The channel rerouting tag is generated based on the comparison of alternative channels and alternative end-to-end delays covering the same audience, so that preceding information can still switch to reachable channels when the preferred channel shrinks, ensuring that the publishable window of subsequent dependent information is not compressed.

[0049] The whole-chain delayed propagation moves the preceding information that cannot be distributed on time, along with all its successors, into the delayed scheduling queue and releases the channel capacity quota, thereby avoiding the waste of channel bandwidth caused by downstream traffic shaping allowing successors to pass without knowing the dependencies.

[0050] The attenuation coefficient is adjusted through deviation feedback in step 8. Coupling coefficient with propagation attenuation The scheduling parameters are updated so that they can be continuously corrected according to the dynamic changes in channel capacity and the completion status of cascading. This integrates cascading scheduling and channel traffic shaping into a two-way constrained collaborative scheduling process, alleviating the compound failure problem caused by the isolated operation of the two.

[0051] The following is an example of an application of the present invention, such as Figure 2-9 As shown, the implementation process is as follows: On a trading day in 20XX, a regional financial information service platform triggered a series of related announcements with a release order dependency in response to a major asset restructuring event of a listed company. The platform deployed several release nodes, with downstream distribution channels covering regional CDN push channels and mainstream financial data platform API terminals. This event generated four related announcements in total: a restructuring plan announcement (preceding), a suspension notice (dependent on the restructuring plan announcement), a resumption of trading arrangements (dependent on the suspension notice), and a risk warning (dependent on the resumption of trading arrangements), forming a complete cascading release chain. The platform needed to complete the push before the preset distribution deadline for each announcement, while also handling capacity fluctuations in downstream channels during peak trading periods.

[0052] The platform collects four pieces of information to be published from each publishing source, extracts the content category identifier, generation timestamp, preset distribution deadline, event association identifier, and target distribution area or platform attribute identifier for each piece of information, and forms a structured information attribute record.

[0053] Table 1 Records of Information Attributes to be Published

[0054] All four messages carry the event association identifier EVT-RZ01. After parsing the content dependency declaration field, a directed acyclic publication order dependency graph is generated: MSG001→MSG002→MSG003→MSG004, where MSG001 is the root node and MSG004 is the leaf node.

[0055] The reverse topology traversal begins from leaf node MSG004. Content category identifiers are encoded and mapped to category priority values. The remaining deadline for each piece of information is obtained by the difference between the preset distribution deadline and the generation timestamp; these two factors together determine their initial value weight. The propagation attenuation coupling coefficient is taken as... The weights of the cascading effects of each node are calculated using the formula. The increments are made layer by layer towards the root node, where For nodes The set of direct successor nodes.

[0056] MSG004 is a leaf node. MSG003's direct successor is MSG004. MSG002's direct successor is MSG003. MSG001's direct successor is MSG002. .

[0057] Table 2 Calculation Results of Cascade Impact Weights

[0058] The platform collects the most recent data from three downstream distribution channels: CDN-Region A, API-Platform B, and API-Platform C. The real-time throughput, queuing depth, response latency, and rate limiting status code of each sampling window are input into a sliding window linear regression algorithm after Z-score standardization to extrapolate the future... The original predicted rate for each scheduling cycle.

[0059] If CDN-Region A is found to have consecutive rate-limiting status codes within the sampling window, and the queuing depth exceeds the preset percentage threshold of the buffer capacity, it is determined to be a congested channel, and the rate limit is applied according to the attenuation coefficient. The original predicted rate is lowered to generate the adjusted available receiving rate. A stability check is performed on the absolute difference between adjacent scheduling periods. In CDN-Region A, the absolute difference between scheduling periods 2 and 3 exceeds the stability threshold; therefore, the predicted value for scheduling period 3 is replaced with the arithmetic mean of the predicted values ​​for the last three periods (resulting in 7.80 Mbps). No congestion was detected in API-Platform B and API-Platform C, and the attenuation coefficient remains at 1.00; the original predicted rate is directly used as the adjusted available receiving rate.

[0060] Table 3. Available Receive Rate Sequences After Channel Adjustment

[0061] Based on the target channel attributes, MSG001 and MSG003 are mapped to CDN-Region A, MSG002 is mapped to API-Platform B, and MSG004 is mapped to API-Platform C. The end-to-end cumulative latency is calculated node-by-node along the topology, starting from the link origin. Queuing latency of each node's publishing node. The downstream channel queuing latency is obtained by dividing the current task queue length of the publishing node by its processing rate. It is obtained by the ratio of the current queuing depth of the corresponding channel to the adjusted available receiving rate. For each node, the larger of the two values ​​is summed. The calculation formula is as follows: .

[0062] MSG001 corresponds to CDN-Region A. Second, Second, Second, Seconds; MSG002 corresponds to API-Platform B. Second, Second, Second, Seconds; MSG003 corresponds to CDN-Region A. Second, Seconds; MSG004 corresponds to API-Platform C. Second, Second, Second, Second.

[0063] Table 4. End-to-end completion delay estimation for cascaded links

[0064] End-to-end completion delay Comparison with the preset distribution deadline. The cumulative latency of these four messages did not exceed their respective remaining deadlines, and therefore did not trigger timeout detection. However, the queuing latency of the downstream channel in CDN-Region A is... The second is significantly higher than the queuing latency of the release node. Within seconds, it was determined that the bottleneck source for MSG001 and MSG003 was downstream channel capacity delay, and alternative channels were searched.

[0065] Search for alternative channels in coverage area A, sort them in descending order of adjusted available receive rate, and evaluate only the top-ranked alternative channels (CDN-Area A-alternatives). Then, retrieve the original channels. After replacing the queuing delay with the alternative channel's corresponding delay, substitute it back into the formula in step 4 to obtain the alternative end-to-end delay. The alternative end-to-end delay for MSG001 is... The cumulative latency for MSG003 replacement is updated to 97.10 seconds, lower than the original 117.00 seconds. Channel rerouting tags are generated for MSG001 and MSG003, pointing to CDN-Region A-Alternate.

[0066] Table 5 Bottleneck Identification and Rerouting Marker Generation

[0067] Arranged from highest to lowest cascading impact weight, the scheduling order is MSG001 (4.42), MSG002 (3.09), MSG003 (2.25), and MSG004 (1.20). MSG001 and MSG003 carry channel rerouting tags and are switched to CDN-Region A-Alternate; MSG002 is assigned to API-Platform B; and MSG004 is assigned to API-Platform C. After the adjustment, the available receiving rate of each channel can cover the distribution requirements of the current scheduled task, and no task enters the delayed scheduling queue.

[0068] During this scheduling cycle, all four messages were allocated within the corresponding channel capacity range. There was no situation where all reachable channels were undercapacity. The entire chain delay propagation was not triggered, the delay scheduling queue was empty, and the channel capacity quota did not need to be released.

[0069] The four messages are pushed to the corresponding publishing nodes and downstream channels according to the scheduling plan. After the actual distribution is completed, the actual end-to-end completion delay of each message, the deviation between the actual receiving rate of each channel and the predicted value, and the overall completion rate of the cascaded link are recorded.

[0070] The actual received rate of CDN Area A is 20.10 Mbps, and the original predicted rate is 21.30 Mbps. The ratio of the two is 0.94. This ratio is combined with the historical attenuation coefficient using an exponentially weighted average to obtain the updated attenuation coefficient. The overall completion rate of this cascaded link was 100%, higher than the predicted completion rate. Therefore, the propagation attenuation coupling coefficient should be appropriately reduced. The adjustment range is positively correlated with the absolute value of the deviation.

[0071] Table 6 Distribution Results and Parameter Feedback

[0072] Throughout the process, data starts with the structured attribute records in step 1, then undergoes reverse topology traversal in step 2 to form cascaded impact weights, followed by capacity status acquisition and attenuation correction in step 3 to obtain the usable receiving rate sequence for each channel after adjustment. Step 4 merges the two types of delay sources into end-to-end cumulative delay. Step 5 identifies downstream channel bottlenecks based on delay comparison and generates rerouting markers. Step 6 prioritizes tasks based on cascaded impact weights. Step 7 confirms that no full-chain delay propagation is needed. Finally, the actual distribution results in step 8 are fed back to the attenuation coefficient. Coupling coefficient with propagation attenuation This forms a closed loop for continuous parameter correction. The input to each step comes directly from the output of the upstream step, and the data flow remains consistent under the time and capacity constraints of the entire cascaded link.

[0073] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A distributed information publishing dynamic scheduling method, characterized in that, Includes the following steps: Obtain the set of information to be published, and extract the content category identifier, generation timestamp, preset distribution deadline, event association identifier, and target distribution area or platform attribute identifier for each piece of information to form a structured information attribute record; Based on the event association identifier, the information to be published is grouped to generate a directed acyclic publishing order dependency graph. A reverse topological traversal is performed along the publishing order dependency graph, and the cascading influence weight is calculated for each information node. The real-time throughput, current queuing depth, response delay, and rate limiting status code of each downstream distribution channel in the most recent sampling windows are collected, aggregated into a capacity status vector, and extrapolated to obtain the predicted available receiving rate of each channel in the next few scheduling cycles. The available receiving rate after adjustment is obtained by downgrading the congested channel according to the attenuation coefficient. Based on the target distribution region or platform attribute identifier, the information to be published is mapped to the corresponding downstream distribution channel. The larger of the queuing delay of the publishing node and the queuing delay of the downstream channel is accumulated along the topological order of the publishing order dependency graph to obtain the end-to-end completion delay of each level of the link. Information nodes whose end-to-end completion delay exceeds the preset distribution deadline are identified as bottleneck information nodes. The source of the bottleneck is determined based on the queuing delay of the publishing node and the queuing delay of the downstream channel. Alternative channels covering the same target are searched for the bottleneck nodes of the downstream channels. When the alternative end-to-end delay is lower than the original end-to-end completion delay, a channel rerouting mark is generated. With the optimization objective of maximizing the total scheduling benefit weighted by the global cascading impact weight, the combination of publishing nodes and channels is allocated according to the cascading impact weight from high to low. For information nodes that cannot be distributed on time, the delay mark is propagated downward along the entire publishing order dependency graph and the corresponding downstream channel capacity quota is released. The distribution is performed, and the attenuation coefficient and the propagation attenuation coupling coefficient in the calculation of the cascade influence weight are updated based on the deviation between the actual receiving rate and the predicted value, and the deviation between the actual completion rate and the predicted completion rate of the cascade link.

2. The distributed information publishing dynamic scheduling method according to claim 1, characterized in that, The weight of the cascaded impact is calculated using the following formula: in, Represents information nodes Cascade effect weights, Represents information nodes Its own initial value weight, Represents information nodes The set of direct successor nodes, Represents a set One of the successor node elements, Represents the propagation attenuation coupling coefficient. Indicates the successor node Cascade effect weights; The initial value weight of itself The content category identifier in the information attribute record and the preset distribution deadline are jointly determined. The higher the category priority corresponding to the content category identifier and the more urgent the preset distribution deadline, the better. The larger the value, the better.

3. The distributed information publishing dynamic scheduling method according to claim 1, characterized in that, The step of extrapolating to obtain the predicted value of the available receiving rate includes: performing Z-score standardization on the real-time throughput, queuing depth, response delay and rate limiting status code in the capacity state vector respectively; encoding the rate limiting status code as a binary value of 0 or 1 to participate in the standardization; inputting the standardized real-time throughput sequence into a sliding window linear regression algorithm; and using the real-time throughput of the most recent sampling windows as the fitting input to extrapolate to obtain the predicted value of the available receiving rate for the next few scheduling cycles.

4. The distributed information publishing dynamic scheduling method according to claim 1, characterized in that, The step of adjusting the available reception rate by reducing the congested channel according to the attenuation coefficient includes: when the queuing depth of a channel exceeds a preset proportional threshold of its buffer capacity, or when a rate-limiting status code appears consecutively within the sampling window, the adjusted available reception rate is generated according to the following formula: in, Indicates the first The available receiving rate after adjustment over one scheduling cycle, Indicates the scheduling cycle number. This represents the rate attenuation factor for a congested channel. Indicates the first The original predicted rate for each scheduling cycle.

5. The distributed information publishing dynamic scheduling method according to claim 4, characterized in that, After generating the adjusted available receive rate sequence, the absolute difference between the adjusted available receive rates of two adjacent scheduling periods is calculated. When the absolute difference exceeds the stability threshold, the predicted value of the current scheduling period is replaced with the arithmetic mean of the predicted values ​​of the most recent scheduling periods, and then used for the estimation of end-to-end completion delay.

6. The distributed information publishing dynamic scheduling method according to claim 1, characterized in that, The end-to-end completion delay is calculated node-by-node using the following formula: in, Indicates the first on the link The end-to-end cumulative latency of each node, This represents the end-to-end cumulative delay of its preceding nodes, with the link starting point being... , This represents the initial value of the end-to-end cumulative delay at the starting point of the link. This represents the queuing latency of the node on the publishing node, and is obtained by the ratio of the current task queue length of the publishing node to the processing rate of the node's information. This represents the queuing delay of the downstream channel corresponding to the node, which is obtained by the ratio of the current queuing depth of the downstream channel corresponding to the node to the adjusted available receiving rate.

7. The distributed information publishing dynamic scheduling method according to claim 1, characterized in that, The steps for retrieving alternative channels covering the same target include: firstly, sorting the alternative channels in descending order based on the target distribution area or platform attribute identifier according to the adjusted available receiving rate; calculating the alternative end-to-end delay only for the top-ranked alternative channels; the calculation method for the alternative end-to-end delay is consistent with the end-to-end completion delay; and replacing the queuing delay of the downstream channel of the original channel with the queuing delay corresponding to the alternative channel and then substituting it back in.

8. The distributed information publishing dynamic scheduling method according to claim 1, characterized in that, The step of allocating publishing nodes and channel combinations according to the cascading influence weight from high to low includes: prioritizing the allocation of publishing nodes and channel combinations with low processing latency and sufficient downstream channel capacity to the preceding information with the highest cascading influence weight; When the capacity of the preferred channel for the preceding information is insufficient, the preceding information is switched to the alternative channel according to the channel rerouting mark. Publishing tasks that exceed the total adjusted available receive rate of all reachable channels are marked as delayed scheduling tasks, and delayed scheduling queues are generated by sorting them according to the cascading impact weights.

9. A distributed dynamic scheduling system for publishing information, used to execute the distributed dynamic scheduling method for publishing information according to any one of claims 1 to 9, characterized in that, include: The information attribute extraction module is used to obtain the set of information to be published and extract the content category identifier, the generated timestamp, the preset distribution deadline, the event association identifier, and the target distribution area or platform attribute identifier. The cascade weight module is used to generate a release order dependency graph and calculate the cascade impact weights by traversing the reverse topology. The channel capacity prediction module is used to acquire the capacity status vector and extrapolate it to obtain the adjusted available receiving rate. The delay estimation module is used to accumulate queuing delays along the topology order to obtain the end-to-end completion delay of each level of the interconnection link; The bottleneck identification and rerouting module is used to identify bottleneck information nodes and generate channel rerouting tags; The cascading awareness scheduling module is used to allocate publishing node and channel combinations according to cascading impact weights, and to mark the overall propagation delay of nodes that cannot be distributed on time along the publishing order dependency graph. The distribution and feedback module is used to perform distribution and update the attenuation coefficient and propagation attenuation coupling coefficient based on the deviation.

10. A computer-readable medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the distributed information publishing dynamic scheduling method according to any one of claims 1-8.

Images