A multi-modal heterogeneous network deterministic transmission optimization method in an industrial scene

Abstract

The application relates to the technical field of industrial network communication, and discloses a multi-modal heterogeneous network determinacy transmission optimization method in an industrial scene, which comprises the following steps: collecting TSN domain port time window resource data and inter-domain gateway buffer state data; screening time window feasible candidate paths based on hop-by-hop timing alignment constraints; extracting gateway correlation arrival time distribution corresponding to the candidate paths; inputting the gateway correlation arrival time distribution and buffer state timing feature sequence into a timing prediction model based on sliding window attention to predict gateway buffer congestion states coupled to arrival perception; calculating an inter-domain time delay compensation amount; solving an integer linear programming joint optimization problem to generate a globally optimal path allocation scheme; and correcting prediction parameters online based on prediction deviation and triggering re-optimization.

Description

Technical Field

[0001] This invention relates to the field of industrial network communication technology, specifically to a deterministic transmission optimization method for multimodal heterogeneous networks in industrial scenarios. Background Technology

[0002] In industrial manufacturing scenarios, multimodal industrial data streams, such as PLC synchronous control signals, visual inspection data streams, and safety interlock signals, need to achieve end-to-end deterministic transmission in a multimodal heterogeneous network composed of the TSN domain and the 5G domain. Within the TSN domain, multiple data streams share the gated time window resources of the switch port, and some data streams need to cross the inter-domain gateway to enter the 5G domain to complete cross-domain transmission.

[0003] Existing solutions typically optimize flow path scheduling independently within the TSN domain to reduce port resource contention, while allocating latency budgets at the inter-domain level based on resource availability information reported by each domain. These two optimization processes are independent of each other.

[0004] The aforementioned independent optimization methods have two shortcomings. First, while intra-domain path optimization reduces inter-flow queuing contention by minimizing port time window overlap, it fails to consider the correlated arrival bursts that occur when multiple data flows scheduled through the same port subsequently converge to the same inter-domain gateway. Even if port-level overlap is reduced, multiple flows within adjacent time windows still arrive at the same gateway sequentially within short time intervals, causing sudden congestion. Second, inter-domain gateway buffer state prediction assumes that the arrival of each flow is independent, but the actual arrival pattern is constrained by the time structure of TSN port gating scheduling, exhibiting periodic correlated bursts. This leads to a systematic underestimation of peak queue depth in buffer congestion prediction, resulting in insufficient feedforward compensation and failure of end-to-end deterministic transmission guarantees. Summary of the Invention

[0005] This invention provides a deterministic transmission optimization method for multimodal heterogeneous networks in industrial scenarios, which solves the technical problem in related technologies that the cross-domain path allocation of heterogeneous networks cannot simultaneously take into account the coupling conflict of TSN domain port time windows and the congestion of inter-domain gateway buffers, resulting in insufficient end-to-end delay deterministic guarantee capability.

[0006] This invention discloses a deterministic transmission optimization method for multimodal heterogeneous networks, comprising the following steps: collecting the time window resource occupancy matrix of each switch port in the TSN domain and the buffer state time-series characteristic sequence of each gateway between domains; Candidate paths are filtered based on hop-by-hop temporal alignment constraints to generate a set of feasible candidate paths within a time window. For candidate paths that need to traverse inter-domain gateways, extract the time set of arrival times of each relevant data stream to the gateway, and generate the gateway-related arrival time distribution; The gateway correlation arrival time distribution and the buffer state time series feature sequence are input into the time series prediction model based on sliding window attention, and the gateway buffer congestion prediction result is output. The inter-domain delay compensation for each candidate path is calculated based on the gateway buffer congestion prediction results. The path allocation is modeled as an integer linear programming problem, and the objective function is the weighted sum of port time window coupling cost and gateway buffer congestion cost to generate a globally optimal path allocation scheme.

[0007] Further, the step of collecting the time window resource occupancy matrix of each switch port in the TSN domain and the buffer status time-series feature sequence of each inter-domain gateway includes: traversing each switch port in the TSN domain, collecting the gating scheduling list configuration of the switch port and the bandwidth occupancy of each currently allocated data stream in each time window, and generating the time window resource occupancy matrix of each port; continuously collecting the buffer queue operation status time-series data of each inter-domain gateway node, wherein the buffer queue operation status time-series data includes the number of enqueued frames, the number of dequeued frames, the peak queue depth and the average queuing waiting time of each priority queue in each scheduling cycle, and aggregating them according to a fixed time window to generate the inter-domain gateway buffer status time-series feature sequence; and performing Z-score normalization processing on each dimension feature in the buffer status time-series feature sequence.

[0008] Furthermore, the step of filtering candidate paths based on hop-by-hop timing alignment constraints includes: for each candidate path of each data stream to be allocated, traversing each TSN switch port traversed by the candidate path, extracting the set of remaining available time windows of the port in the priority queue required by the data stream from the time window resource occupancy matrix; performing hop-by-hop timing alignment constraint checks on the available time windows of adjacent ports, adding the end time of the sending window of the upstream port to the link propagation delay to obtain the earliest time when the data frame arrives at the downstream port, and determining whether the earliest arrival time is not later than the opening time of the receiving window of the downstream port and not earlier than the difference between the closing time of the receiving window of the downstream port and the data frame transmission duration; if any pair of adjacent ports on the candidate path does not meet the above conditions, it is determined that the candidate path has a timing alignment conflict and is eliminated.

[0009] Furthermore, the process of generating the gateway-related arrival time distribution includes: for each candidate path that needs to traverse inter-domain gateways, identifying the inter-domain gateway identifiers traversed by the candidate path, and extracting the transmission time window position of the candidate path on the last TSN port on the gateway ingress side; retrieving the transmission time window positions of other data streams already allocated on the same gateway on the corresponding ports, and calculating the arrival time of each stream after adding the link propagation delay to the end time of the transmission time window; applying mean normalization based on the range to the arrival time, and applying Z-score normalization to the data frame length information.

[0010] Furthermore, the time-series prediction model based on sliding window attention consists of an input concatenation layer, a sliding window attention layer, and a prediction output layer, wherein: the input concatenation layer concatenates the buffer state time-series feature sequence with the gateway correlation arrival time distribution along the feature dimension to form a fused input sequence; the sliding window attention layer performs attention calculation within a fixed-length sliding window centered on the current time, assuming the fused input sequence within the sliding window is... ,in The length of the sliding window. The attention output is the concatenated feature dimension. Calculated by the following formula:

[0011] in , , They are respectively from The query matrix, key matrix, and value matrix obtained through linear transformation The dimension of the key vector is denoted as . The prediction output layer is a fully connected layer, which takes the output vector corresponding to the current time and outputs the peak queue depth prediction value and queuing waiting time prediction value of each gateway and each priority queue after linear mapping.

[0012] Furthermore, the objective function of the integer linear programming problem is:

[0013] in To represent data stream Should candidate paths be selected? Decision variables, For the set of data streams to be allocated, For data stream The set of feasible candidate paths within the time window. The cost of port time window coupling, This refers to the buffer congestion cost obtained based on the gateway buffer congestion prediction results. and These are the corresponding non-negative weight coefficients.

[0014] Furthermore, the port time window coupling cost The calculation method is as follows: for candidate paths For each TSN switch port, statistics are collected on the data flow within the same scheduling cycle. The number of other allocated data streams with overlapping transmission time windows is calculated by summing the overlap counts on each port; the buffer congestion cost... The calculation method is as follows: for candidate paths For each inter-domain gateway, the peak queue depth prediction value of each priority queue of the gateway is taken from the gateway buffer congestion prediction result, denormalized and restored, and then normalized and accumulated using the mean normalization method based on the range.

[0015] Furthermore, the constraints of the integer linear programming problem include: each data stream has one and only one path to choose from. The end-to-end delay estimates of the inter-domain delay compensation for each flow do not exceed the deterministic delay boundary. ,in This is the estimated end-to-end delay. For data stream The deterministic latency boundary; the total bandwidth usage of each shared port's time window does not exceed the capacity limit; the depth of each gateway's buffer queue does not exceed the queue depth limit.

[0016] Furthermore, it also includes: after the global optimal path allocation scheme is issued and executed, continuously collecting the actual queuing delay data of gateways between domains; comparing the actual queuing delay with the corresponding predicted value in the gateway buffer congestion prediction result after standardization, and calculating the deviation; feeding the deviation as an online correction signal back to the time-series prediction model based on sliding window attention to update the model parameters, and updating the time window resource occupancy matrix of each port based on the actual traffic allocation situation; and accumulating the absolute value of the deviation in each cycle within multiple consecutive scheduling cycles, and triggering the re-execution of the optimization process from path filtering to path allocation solution when the accumulated value exceeds a preset threshold.

[0017] This invention discloses a deterministic transmission optimization system for multimodal heterogeneous networks, used to execute the aforementioned deterministic transmission optimization method for multimodal heterogeneous networks, including: a data acquisition module, used to acquire the time window resource occupancy matrix of each switch port in the TSN domain and the buffer state time-series characteristic sequence of each domain gateway; The path filtering module is used to filter candidate paths based on hop-by-hop temporal alignment constraints and generate a set of feasible candidate paths for a time window. The arrival distribution extraction module is used to extract the arrival time set of each relevant data stream from the candidate path that needs to cross the inter-domain gateway, and generate the gateway-related arrival time distribution. The buffer congestion prediction module is used to input the gateway correlation arrival time distribution and the buffer state temporal feature sequence into the time-series prediction model based on sliding window attention, and output the gateway buffer congestion prediction result. The delay compensation calculation module is used to calculate the inter-domain delay compensation amount for each candidate path based on the gateway buffer congestion prediction results. The path allocation optimization module is used to model path allocation as an integer linear programming problem, and solves it with the weighted sum of port time window coupling cost and gateway buffer congestion cost as the objective function to generate the globally optimal path allocation scheme. The online correction module is used to calculate the deviation between the actual queuing delay and the gateway buffer congestion prediction result, feed it back to the time-series prediction model to update the parameters, and trigger the re-execution of path allocation optimization when the accumulated deviation exceeds a preset threshold.

[0018] The present invention has the following beneficial effects: This invention extracts the gateway-related arrival time distribution from TSN port gating scheduling information and explicitly injects it into the gateway buffer congestion prediction process. Compared with traditional methods that implicitly assume independent arrival, this method enables peak queue depth prediction to cover periodically related burst arrival scenarios, thus making the calculation results of inter-domain delay compensation closer to the actual queuing situation.

[0019] This invention integrates arrival-aware gateway buffer congestion prediction results with inter-domain delay compensation into the joint optimization objective and constraints of integer linear programming. This allows minimizing inter-flow port contention within the TSN domain and suppressing inter-domain gateway buffer congestion to be performed collaboratively within the same optimization framework. Compared to separate optimization methods, this approach avoids the contradiction of low-overlapping paths at the port level generating high burst congestion at the gateway level, ensuring that the path allocation scheme is reasonable at both the port and gateway levels.

[0020] This invention employs an online parameter correction method driven by prediction bias to continuously calibrate inter-domain queuing delay prediction results and triggers re-optimization when the accumulated bias exceeds a preset threshold. This method can distinguish between single random disturbances and systematic prediction biases, ensuring that the end-to-end deterministic transmission constraints of multimodal industrial data streams remain effective over the long term during network operation. Attached Figure Description

[0021] Figure 1 This is a flowchart of the deterministic transmission optimization method for multimodal heterogeneous networks in industrial scenarios provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of the bandwidth occupancy and remaining distribution of each port in the TSN domain provided in an embodiment of the present invention; Figure 3 This is a schematic diagram showing the number of flows allocated to each port in the TSN domain according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the timing change trend of the buffer state of Gateway A queue 1 provided in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the time-series variation of the average queuing time of gateway queue 1 provided in an embodiment of the present invention; Figure 6This is a schematic diagram comparing the end-to-end delay estimates and delay boundaries of each feasible candidate path provided in the embodiments of the present invention; Figure 7 This is a schematic diagram of the decomposition of the end-to-end delay components of each feasible candidate path provided in the embodiments of the present invention; Figure 8 This is a schematic diagram of the congestion prediction results of each priority queue buffer of the inter-domain gateway provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the distribution of preprocessing features of gateway A ingress flow arrival time and frame length provided in an embodiment of the present invention. Detailed Implementation

[0022] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0023] At least one embodiment of the present invention discloses a deterministic transmission optimization method for multimodal heterogeneous networks in industrial scenarios, see [link to relevant documentation]. Figure 1 This includes the following steps: Step 1: Collect TSN domain port time window resource data and inter-domain gateway buffer status data.

[0024] Iterate through each switch port in the TSN domain, collect the gating scheduling list configuration of that switch port and the bandwidth occupancy of each currently allocated multimodal data stream within each time window, and generate a time window resource occupancy matrix for each port. At the same time, continuously collect the time-series data of the cross-domain buffer queue operation status of the inter-domain gateway nodes, aggregate them according to fixed time windows, and generate a time-series feature sequence of inter-domain gateway buffer status.

[0025] It should be noted that the aforementioned buffer queue operation status time-series data includes the number of enqueue frames, the number of dequeue frames, the peak queue depth, and the average queuing wait time for each priority queue within each scheduling cycle. Aggregation by fixed time window refers to statistically summarizing the above indicators within multiple consecutive scheduling cycles according to a preset time window length to form a feature vector corresponding to that time window. The feature vectors of multiple time windows are arranged in chronological order to constitute the inter-domain gateway buffer status time-series feature sequence.

[0026] Before inputting the temporal feature sequence of the inter-domain gateway buffer state into the prediction model, Z-score standardization is performed on each dimension of the feature sequence to eliminate the influence of the dimensional differences between different physical quantities such as the number of enqueued frames, the number of dequeued frames, the peak queue depth, and the average queuing waiting time on the model training.

[0027] Step 2: Filter feasible candidate paths within the time window based on hop-by-hop temporal alignment constraints.

[0028] For each candidate path in the candidate path set for each multimodal data stream to be allocated, traverse each TSN switch port traversed by that candidate path. Extract the set of remaining available time windows for that TSN switch port in the priority queue required by the data stream from the time window resource occupancy matrix. Perform hop-by-hop timing alignment constraint checks on the available time windows of adjacent ports to verify whether the end time of the upstream port's sending window plus the link propagation delay falls within the opening time range of the downstream port's receiving window. Eliminate candidate paths with timing alignment conflicts to generate a set of feasible candidate paths for each time window.

[0029] It should be noted that the hop-by-hop timing alignment constraint check process is as follows: For two consecutive TSN switch ports on a candidate path, the end time of the upstream port's transmission window is added to the propagation delay of that link segment to obtain the earliest time when the data frame arrives at the downstream port. It is then determined whether this earliest arrival time is no later than the opening time of the downstream port's receive window and no earlier than the difference between the closing time of the downstream port's receive window and the data frame transmission duration. If any pair of adjacent ports on the candidate path does not meet the above conditions, then the candidate path is determined to have a timing alignment conflict.

[0030] Step 3: Extract the gateway relevance arrival time distribution corresponding to the candidate path.

[0031] For each candidate path in the feasible candidate path set that needs to traverse inter-domain gateways, the following processing is performed: identify the inter-domain gateway identifiers traversed by the candidate path; extract the transmission time window position of the candidate path on the last TSN port on the gateway ingress side; retrieve the transmission time window positions of other data flows already allocated on the same gateway on the corresponding ports; calculate the arrival time at the gateway after adding the link propagation delay to the end time of each flow's transmission time window. After the above processing is completed, the gateway-related arrival time distribution corresponding to each candidate path is generated.

[0032] It should be noted that the gateway-related arrival time distribution describes the set of times when all related data flows arrive at the inter-domain gateway via the same inter-domain gateway within the same scheduling period, along with the corresponding data frame length information for each time. The gateway-related arrival time distribution reflects the periodic clustering arrival pattern of multiple data flows at the gateway inlet due to TSN port gating scheduling constraints, which differs from the traditional modeling approach that assumes that the arrival times of each flow are independent.

[0033] Before inputting the gateway correlation arrival time distribution into the prediction model, the arrival time is normalized by mean based on the range, and the data frame length information is normalized by Z-score, so that the arrival time and the inter-domain gateway buffer state time series feature sequence are in a similar numerical range, so that they can be jointly input into the prediction model after feature dimension splicing.

[0034] Step 4: Predict the congestion status of the gateway buffer that is coupled to the sense.

[0035] The preprocessed gateway relevance arrival time distribution is used as an additional input feature, and together with the standardized inter-domain gateway buffer state temporal feature sequence, it is input into the sliding window attention-based temporal prediction model. The input of the sliding window attention-based temporal prediction model is the concatenated sequence of the gateway relevance arrival time distribution and the inter-domain gateway buffer state temporal feature sequence. The output is the peak queue depth prediction value and queuing waiting time prediction value of each priority queue of each gateway under each candidate path allocation scheme within a future preset number of scheduling periods. The above output constitutes the gateway buffer congestion prediction result coupled with arrival awareness.

[0036] It should be noted that the time series prediction model based on sliding window attention is an unconventional time series prediction model, and its constituent units and data transmission relationships are as follows.

[0037] The time-series prediction model based on sliding window attention consists of three parts: an input concatenation layer, a sliding window attention layer, and a prediction output layer. The input concatenation layer receives the temporal feature sequence of inter-domain gateway buffer states and the arrival time distribution of gateway correlations, concatenating them along the feature dimension to form a fused input sequence. The sliding window attention layer performs attention calculations on the fused input sequence, where the attention calculation scope is limited to a fixed-length sliding window centered on the current time, rather than the entire sequence range. This allows the time-series prediction model based on sliding window attention to focus on capturing recent temporal dependencies. Let the fused input sequence within the sliding window be... ,in The length of the sliding window. For the concatenated feature dimensions, the attention output within the window... Calculated by the following formula:

[0038] in, , , They are respectively from The query matrix, key matrix, and value matrix obtained through linear transformation Let be the dimension of the key vector. The dimension of the value vector. Indicates transpose. This is a normalized exponential function. The prediction output layer is a fully connected layer that receives attention outputs. After linearly mapping the output vector corresponding to the current moment, the predicted peak queue depth and queuing time of each priority queue of each gateway are output; let the dimension of the output vector be... ,but It equals the product of the number of gateways, the number of priority queues, and the number of prediction steps. Each prediction step includes two predicted quantities: peak queue depth and queuing wait time. The peak queue depth and queuing wait time predicted values ​​output by the prediction output layer are dimensionless values ​​after normalization. Before use in subsequent steps, they need to be denormalized to restore them to actual physical quantities in units of frames and time.

[0039] The time-series prediction model based on sliding window attention uses the true peak queue depth and true queuing wait time from historical time-series data as supervision labels. Supervised training is performed using the mean squared error loss function, and the optimization strategy is the Adam optimization algorithm. During training, the gateway relevance arrival time distribution and the time-series feature sequences of inter-domain gateway buffer states are both derived from historically collected data. The output layer directly generates predicted values ​​without using an additional decoder.

[0040] In this embodiment, to enable the sliding window attention-based time-series prediction model to perceive the impact of periodically correlated burst arrival patterns on the gateway buffer queue, the gateway-related arrival time distribution is used as an additional input feature, replacing the original independent arrival assumption. In traditional methods, gateway buffer prediction is extrapolated solely based on historical buffer state time-series data, implicitly assuming that the arrival times of each flow are independent. This step explicitly introduces the inter-flow arrival time correlation generated by TSN domain gating scheduling into the prediction process, enabling the sliding window attention-based time-series prediction model to capture the impact of periodically correlated burst arrival patterns on the gateway buffer queue depth.

[0041] Step 5: Calculate the inter-domain delay compensation.

[0042] Based on the gateway buffer congestion prediction results of coupled arrival awareness, for each candidate path that needs to traverse inter-domain gateways, the predicted queuing wait time in the coupled arrival awareness gateway buffer congestion prediction results is denormalized to restore it to the actual time. Then, the predicted queuing wait times of all gateways traversed by the candidate path are added to obtain the queuing-added delay of the candidate path. The queuing-added delay is then added to the sum of the link propagation delay and the intra-domain forwarding delay of the candidate path to obtain the end-to-end delay estimate including inter-domain delay compensation.

[0043] Step 6: Solve the joint optimization to obtain the globally optimal path allocation scheme.

[0044] The path allocation problem for multiple multimodal data streams is modeled as an integer linear programming problem.

[0045] Define decision variables Represents data stream Should candidate paths be selected? ,in , For the data stream index in the set of data streams to be allocated, For the set of data streams to be allocated, For data stream The path index of the feasible candidate path set within the time window. For data stream The set of feasible candidate paths within the time window.

[0046] The objective function is to minimize the weighted sum of the inter-stream port time window coupling cost and the inter-domain gateway buffer congestion cost for all data streams:

[0047] in, For data stream Select candidate path The time window coupling cost with other allocated flows on the shared port. For data stream Select candidate path The buffer congestion cost is obtained from the gateway buffer congestion prediction results based on the coupling arrival perception at each gateway. The non-negative weighting coefficients corresponding to the port time window coupling cost are: This is the non-negative weighting coefficient corresponding to the cost of buffering congestion.

[0048] It should be noted that, The calculation method is as follows: for candidate paths Each TSN switch port Statistics on data flow within the same scheduling cycle The number of other allocated data streams whose transmission time windows overlap is calculated by summing the overlap counts on each port to obtain the port time window coupling cost of the candidate path. . The calculation method is as follows: for candidate paths Each inter-domain gateway passed through The peak queue depth prediction values ​​(already denormalized to the actual frame number) of each priority queue of the inter-domain gateway are taken from the coupled arrival perception gateway buffer congestion prediction results. The peak queue depth prediction values ​​of each queue of each inter-domain gateway are normalized and then summed to obtain the buffer congestion cost of the candidate path. The normalization method uses the mean normalization method based on the range, with the upper limit of the queue depth of each queue as the normalization benchmark.

[0049] The constraints include: Each data stream selects one and only one path:

[0050] The inter-domain compensation for end-to-end delay of each stream does not exceed the deterministic delay boundary:

[0051] in This is the end-to-end delay estimate that includes inter-domain delay compensation. For data stream The deterministic time delay boundary.

[0052] The total bandwidth usage of each shared port within its time window shall not exceed the capacity limit:

[0053] in For data stream On the TSN switch port Bandwidth usage on For TSN switch ports The maximum bandwidth capacity of the time window. This is the set of all TSN switch ports.

[0054] The depth of each gateway's buffer queue shall not exceed the upper limit of queue depth:

[0055] in For data stream Select candidate path Inter-domain gateway priority queue The predicted queue depth contribution value, Inter-domain gateway priority queue The upper limit of queue depth, For a set of inter-domain gateways, This is a set of priority queues.

[0056] The above integer linear programming problem is solved using a branch and bound solver, generating a globally optimal path allocation scheme for a multimodal heterogeneous network.

[0057] In this embodiment of the application, in order to ensure the long-term effectiveness of the globally optimal path allocation scheme for multimodal heterogeneous networks during operation, the following steps are also included in step 6: Step 7: Correct the prediction parameters online based on the prediction bias and trigger re-optimization.

[0058] The globally optimal path allocation scheme for the multimodal heterogeneous network is distributed to each domain controller for scheduling. Actual queuing delay data from gateways across domains is continuously collected. This actual queuing delay is standardized using Z-score and compared with the corresponding predicted value (both within the standardized space) in the gateway buffer congestion prediction results coupled with arrival awareness. The deviation between the two is calculated. This deviation is used as an online correction signal to feed back to the sliding window attention-based time-series prediction model, updating its parameters. Simultaneously, the time window resource occupancy matrix for each port is updated based on the actual traffic allocation. When the accumulated deviation exceeds a preset threshold, steps 2 through 6 of the path allocation optimization process are re-executed.

[0059] It should be noted that the method for determining the cumulative deviation is as follows: within multiple consecutive scheduling cycles, the absolute value of the difference between the actual queuing delay and the predicted queuing waiting time in each cycle is taken and accumulated; when the accumulated value exceeds a preset threshold, it is determined that the current prediction deviation has deviated from the acceptable range, triggering re-optimization. This method can distinguish between single random disturbances and systematic prediction deviations, avoiding unnecessary re-optimization caused by instantaneous fluctuations.

[0060] This invention extracts the gateway-dependent arrival time distribution of flows arriving at the gateway from TSN port gating scheduling information and explicitly injects this distribution into the gateway buffer congestion prediction process. This eliminates the independent arrival assumption bias that traditional methods rely on in gateway buffer prediction, enabling peak queue depth prediction to cover periodically correlated burst arrival scenarios, thus making the calculation of inter-domain delay compensation more accurate. Furthermore, the invention integrates the arrival-aware gateway buffer congestion prediction results with the inter-domain delay compensation into the joint optimization objective and constraints of integer linear programming. This allows minimizing inter-flow port contention within the TSN domain and suppressing inter-domain gateway buffer congestion to be performed synergistically within the same optimization framework, avoiding the contradiction that low-overlapping paths at the port level may lead to high burst congestion at the gateway level when optimized separately. Through an online parameter correction method driven by prediction bias, the inter-domain queuing delay prediction results are continuously calibrated, ensuring the long-term effectiveness of end-to-end deterministic transmission constraints for multimodal industrial data flows.

[0061] On the other hand, this invention also proposes a deterministic transmission optimization system for multimodal heterogeneous networks in industrial scenarios, comprising: The data acquisition module is used to collect time window resource data of TSN domain ports and inter-domain gateway buffer status data. It iterates through each switch port in the TSN domain to collect the gating scheduling list configuration and bandwidth usage of allocated data flows, generating a time window resource usage matrix. It also continuously collects the time-series data of the buffer queue operation status of each inter-domain gateway, aggregates it according to fixed time windows to generate an inter-domain gateway buffer status time-series feature sequence, and performs Z-score standardization on each dimension of the inter-domain gateway buffer status time-series feature sequence.

[0062] The path filtering module is used to filter feasible candidate paths within a time window based on hop-by-hop timing alignment constraints. For each candidate path, hop-by-hop timing alignment constraint checks are performed on all TSN ports traversed, eliminating candidate paths with timing alignment conflicts, and generating a set of feasible candidate paths for the time window.

[0063] The arrival distribution extraction module is used to extract the gateway-related arrival time distribution corresponding to candidate paths. For candidate paths that need to traverse inter-domain gateways, it calculates the set of arrival times of each related data stream to the inter-domain gateway, generates the gateway-related arrival time distribution, and performs normalization and standardization preprocessing on the arrival time and data frame length information.

[0064] The buffer congestion prediction module is used to predict the congestion state of the gateway buffer in coupled arrival sensing. It inputs the preprocessed gateway correlation arrival time distribution and the standardized inter-domain gateway buffer state temporal feature sequence into a sliding window attention-based temporal prediction model. The model outputs the peak queue depth prediction and queuing wait time prediction values ​​for each priority queue of each gateway, and then denormalizes the output results to restore them to actual physical quantities.

[0065] The delay compensation calculation module is used to calculate the inter-domain delay compensation. It accumulates the predicted queuing wait times of all inter-domain gateways traversed by each candidate path to obtain an end-to-end delay estimate that includes the inter-domain delay compensation.

[0066] The path allocation optimization module is used to solve joint optimization problems to obtain the globally optimal path allocation scheme. It models the path allocation problem as an integer linear programming problem and uses a branch and bound solver to generate the globally optimal path allocation scheme for multimodal heterogeneous networks.

[0067] The online correction module is used to correct prediction parameters online based on prediction deviation and trigger re-optimization. It calculates the deviation between the actual queuing delay and the corresponding predicted value in the gateway buffer congestion prediction result coupled with arrival awareness, feeds it back to the time-series prediction model based on sliding window attention to update parameters, and triggers the re-execution of path allocation optimization when the accumulated deviation exceeds a preset threshold.

[0068] By way of example, the system proposed in this invention may also include various features and combinations thereof in the method embodiments, which will not be elaborated here.

[0069] See Figures 2-9 The following application examples are proposed: In an automated production line of an automotive parts manufacturing plant, PLC synchronization control signals, visual inspection data streams, and safety interlock signals require end-to-end deterministic transmission through a multimodal heterogeneous network comprised of a TSN domain and a 5G domain. The TSN domain includes three industrial switches (Switch A, Switch B, and Switch C) with five critical ports (Ports 1 to 5). There are two inter-domain gateway nodes (Gateway A and Gateway B), responsible for forwarding TSN domain data streams to the 5G domain. The current scheduling cycle is 125 microseconds, and there are three data streams to be allocated: Stream A is the PLC synchronization control signal (with strict latency constraints), Stream B is the visual inspection data stream (with high bandwidth requirements), and Stream C is the safety interlock signal (with the strictest latency constraints). Initially, some bandwidth on the TSN domain ports is occupied by historical streams, and historical loads accumulate in the gateway buffer queues. The optimization objective is to minimize the port time window coupling cost and gateway buffer congestion cost while ensuring deterministic latency constraints for each stream.

[0070] The data acquisition module iterates through ports 1 to 5, reading the gating scheduling list configuration and bandwidth usage of allocated historical flows for each port, generating a time-window resource usage matrix. Simultaneously, it continuously collects the inbound and outbound frame counts, peak queue depth, and average queuing wait time for each priority queue of Gateway A and Gateway B within a continuous scheduling cycle. This data is aggregated within a fixed time window of eight consecutive scheduling cycles to form a temporal characteristic sequence of inter-domain gateway buffer states. Z-score standardization is then applied to each dimension of this sequence to eliminate dimensional differences between different physical quantities.

[0071] Table 1. TSN Domain Port Time Window Resource Usage Matrix (Collection Results)

[0072] Table 2. Temporal characteristics of inter-domain gateway buffer states (before standardization, example of a single time window)

[0073] The path filtering module performs hop-by-hop timing alignment constraint checks on the candidate path sets of flows A, B, and C. Taking flow C (security interlocking signal) as an example, its candidate path set includes path C-1 (via port 1 → port 3 → gateway A) and path C-2 (via port 2 → port 4 → gateway B). For path C-1, the end time of the transmission window of port 1 in priority queue 2 is extracted, and the propagation delay of the link from port 1 to port 3 is added to verify whether this time falls within the valid range of the receiving window of port 3. Path C-1 passes the timing alignment check. Path C-2 is eliminated because the end time of the transmission window of port 2 plus the propagation delay exceeds the closing time of the receiving window of port 4, resulting in a timing alignment conflict. The filtering results for each flow are shown in Table 3.

[0074] Table 3. Results of Filtering for Jump-by-Jump Timing Alignment Constraints

[0075] The arrival distribution extraction module processes all paths that need to traverse inter-domain gateways in the feasible candidate path set within the time window. Taking gateway A as an example, paths A-1, B-1, and C-1 all pass through gateway A. The module extracts the end time of the transmission time window for path A-1 on port 5 (the last TSN port on the gateway A's ingress side), adds the link propagation delay from port 5 to gateway A, and obtains the arrival time of flow A via path A-1. Similarly, the arrival times of flow B via path B-1 and flow C via path C-1 are calculated. The arrival times of the three flows are concentrated within the same scheduling period, forming the gateway-dependent arrival time distribution for gateway A. The arrival times are normalized using mean normalization based on the range, and the data frame length information is normalized using Z-score.

[0076] Table 4. Distribution of arrival times of gateway A correlations (comparison before and after preprocessing)

[0077] The buffer congestion prediction module concatenates the preprocessed gateway-related arrival time distribution (normalized arrival time and frame length) from Table 4 with the Z-score-normalized inter-domain gateway buffer state temporal feature sequence from Table 2 along the feature dimension to form a fused input sequence, which is then input into the sliding window attention-based temporal prediction model. In the fused input sequence, the inter-domain gateway buffer state temporal feature sequence provides the temporal context of historical buffer load, while the gateway-related arrival time distribution provides structural information on the periodic aggregation of multiple flows within the current scheduling period. The sliding window attention layer performs attention calculations within a fixed-length window, capturing recent temporal dependencies and focusing on the burst effect caused by the dense arrival of flows A, B, and C at gateway A between 83.60 microseconds and 90.20 microseconds. The prediction output layer, after linear mapping, outputs the peak queue depth prediction and queuing wait time prediction values ​​for each queue, which are then denormalized to restore them to actual physical quantities.

[0078] Table 5. Congestion prediction results for the gateway buffer coupled to sensing (after denormalization)

[0079] The delay compensation calculation module accumulates the predicted queuing wait times of all gateways traversed by each feasible candidate path. Path A-1, via gateway A queue 1, has a predicted queuing wait time of 96.40 microseconds. Adding the link propagation delay and the TSN domain forwarding delay, the end-to-end delay estimate for path A-1 is obtained. Path A-2, via gateway B queue 1, has a predicted queuing wait time of 28.70 microseconds, and its end-to-end delay estimate is significantly lower than that of path A-1. The end-to-end delay estimates for each feasible path are shown in Table 6.

[0080] Table 6. End-to-end delay estimates for each feasible candidate path

[0081] The path allocation optimization module models the path allocation problem for flows A, B, and C as an integer linear programming problem. The objective function is to minimize the coupling cost of the port time window. With buffer congestion cost The weighted sum, weight coefficients and The settings are pre-defined based on the production line's sensitivity to port contention and gateway congestion. Based on the normalized and summed peak queue depth prediction values ​​of each queue in Table 5, the following results were obtained: Path A-1, Path B-1, and Path C-1 all pass through Gateway A. The peak queue depth prediction value of queue 1 in Gateway A is 31 frames, the normalization benchmark is the upper limit of queue depth of 64 frames, and the normalization value is 0.48; Path A-2 passes through Gateway B, and the normalization value of queue 1 in Gateway B is 0.14, indicating a significantly lower buffer congestion cost. In one scenario, path A-1 passes through port 1, while paths B-1 and C-1 also pass through port 1, resulting in high coupling costs due to the three flows sharing port 1. Path A-2 passes through port 2 and has no shared port with the paths of flows B and C, thus its port coupling cost is zero. Under the premise of satisfying various constraints (path uniqueness constraint, delay constraint, port bandwidth constraint, queue depth constraint), the branch and bound solver outputs a globally optimal path allocation scheme: flow A chooses path A-2 (via port 2 → port 4 → gateway B), flow B chooses path B-1 (via port 1 → port 3 → gateway A), and flow C chooses path C-1 (via port 1 → port 3 → gateway A). This scheme diverts flow A from gateway A to gateway B, effectively reducing the sudden congestion pressure on gateway A, while simultaneously reducing the port coupling cost of flow A to zero.

[0082] After the online correction module distributes the aforementioned globally optimal path allocation scheme to each domain controller, it continuously collects the actual queuing delays of Gateway A and Gateway B. Taking Gateway A's queue 1 as an example, the actual queuing delay is standardized by Z-score and compared with the predicted value (standardized space). The absolute value of the deviation for each cycle is calculated and accumulated. If the accumulated deviation exceeds a preset threshold within a continuous scheduling cycle, it is determined that there is a systematic deviation in the prediction. The deviation signal is fed back to the time-series prediction model based on sliding window attention to update the parameters and trigger the re-execution of the path allocation optimization process from steps 2 to 6.

[0083] The data flow throughout the implementation process embodies a complete logical chain. Step 1, which collects the time window resource occupancy matrix, provides the basis for the available port window in Step 2's hop-by-hop timing alignment constraint check; Step 2, which filters out the feasible candidate path set for the time window, limits the extraction range of the arrival distribution in Step 3; Step 4, the gateway-related arrival time distribution generated in Step 3 (Table 4) and the inter-domain gateway buffer state timing feature sequence generated in Step 1 (Table 2) are fused together to drive the prediction model to output the congestion prediction results shown in Table 5; The queuing wait time prediction values ​​in Table 5 are superimposed in Step 5 to form the end-to-end delay estimate shown in Table 6; The peak queue depth prediction value in Table 5 and the path port information in Table 3 are used together in Step 6 to construct the objective function and constraints of integer linear programming, ultimately outputting the globally optimal path allocation scheme by the branch and bound solver; Step 7, with its online correction mechanism, continuously calibrates the prediction parameters, ensuring the accuracy and effectiveness of the above data flow chain during long-term production line operation.

[0084] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. A deterministic transmission optimization method for multimodal heterogeneous networks, characterized in that, Includes the following steps: Collect the time window resource occupancy matrix of each switch port in the TSN domain and the time-series characteristic sequence of the buffer status of gateways between domains; Candidate paths are filtered based on hop-by-hop temporal alignment constraints to generate a set of feasible candidate paths within a time window. For candidate paths that need to traverse inter-domain gateways, extract the time set of arrival times of each relevant data stream to the gateway, and generate the gateway-related arrival time distribution; The gateway correlation arrival time distribution and the buffer state time series feature sequence are input into the time series prediction model based on sliding window attention, and the gateway buffer congestion prediction result is output. The inter-domain delay compensation for each candidate path is calculated based on the gateway buffer congestion prediction results. The path allocation is modeled as an integer linear programming problem, and the objective function is the weighted sum of port time window coupling cost and gateway buffer congestion cost to generate a globally optimal path allocation scheme.

2. The deterministic transmission optimization method for multimodal heterogeneous networks according to claim 1, characterized in that, The time window resource occupancy matrix of each switch port in the TSN domain and the time-series characteristic sequence of the buffer status of each gateway between domains are collected, including: Traverse each switch port in the TSN domain, collect the gating and scheduling list configuration of that switch port and the bandwidth usage of each currently allocated data stream in each time window, and generate a time window resource usage matrix for each port. The system continuously collects time-series data on the buffer queue operation status of each inter-domain gateway node. The time-series data on the buffer queue operation status includes the number of enqueue frames, the number of dequeue frames, the peak queue depth, and the average queuing waiting time for each priority queue in each scheduling cycle. The data is then aggregated according to a fixed time window to generate a time-series feature sequence of inter-domain gateway buffer status. Z-score normalization is performed on each dimension of the buffer state temporal feature sequence.

3. The deterministic transmission optimization method for multimodal heterogeneous networks according to claim 1, characterized in that, The process of filtering candidate paths based on hop-by-hop temporal alignment constraints includes: For each candidate path of each data stream to be allocated, traverse each TSN switch port that the candidate path passes through, and extract the set of remaining available time windows of that port in the priority queue required by the data stream from the time window resource occupancy matrix; Perform hop-by-hop timing alignment constraint checks on the available time windows of adjacent ports. Add the end time of the sending window of the upstream port to the link propagation delay to obtain the earliest time when the data frame arrives at the downstream port. Determine whether the earliest arrival time is not later than the opening time of the receiving window of the downstream port and not earlier than the difference between the closing time of the receiving window of the downstream port and the data frame transmission duration. If any pair of adjacent ports on a candidate path does not meet the above conditions, the candidate path is determined to have a timing alignment conflict and is eliminated.

4. The deterministic transmission optimization method for multimodal heterogeneous networks according to claim 1, characterized in that, The process of generating the gateway correlation arrival time distribution includes: For each candidate path that needs to traverse an inter-domain gateway, identify the identifier of the inter-domain gateway that the candidate path passes through, and extract the transmission time window position of the candidate path on the last TSN port on the gateway ingress side. Retrieve the sending time window positions of other data streams already allocated on the same gateway on the corresponding port, and calculate the arrival time at the gateway after adding the link propagation delay to the end time of each stream's sending time window. The arrival time is normalized using mean normalization based on the range, and the data frame length information is normalized using Z-score.

5. The deterministic transmission optimization method for multimodal heterogeneous networks according to claim 1, characterized in that, The time-series prediction model based on sliding window attention consists of an input concatenation layer, a sliding window attention layer, and a prediction output layer, wherein: The input splicing layer splices the buffer state temporal feature sequence with the gateway correlation arrival time distribution on the feature dimension to form a fused input sequence; The sliding window attention layer performs attention calculations within a fixed-length sliding window centered on the current time. Let the fused input sequence within the sliding window be... ,in The length of the sliding window. Given the concatenated feature dimensions, the attention output is... Calculated by the following formula: ； in , , They are respectively from The query matrix, key matrix, and value matrix obtained through linear transformation The dimension of the key vector; The prediction output layer is a fully connected layer. It takes the output vector corresponding to the current moment, performs linear mapping, and outputs the peak queue depth prediction value and queuing waiting time prediction value of each priority queue of each gateway.

6. The deterministic transmission optimization method for multimodal heterogeneous networks according to claim 1, characterized in that, The objective function of the integer linear programming problem is: ； in To represent data stream Should candidate paths be selected? Decision variables, For the set of data streams to be allocated, For data stream The set of feasible candidate paths within the time window. The cost of port time window coupling, This refers to the buffer congestion cost obtained based on the gateway buffer congestion prediction results. and These are the corresponding non-negative weight coefficients.

7. The deterministic transmission optimization method for multimodal heterogeneous networks according to claim 6, characterized in that, The port time window coupling cost The calculation method is as follows: for candidate paths For each TSN switch port, statistics are collected on the data flow within the same scheduling cycle. The number of other allocated data streams with overlapping transmission time windows is calculated, and the number of overlaps on each port is summed. The buffer congestion cost The calculation method is as follows: for candidate paths For each inter-domain gateway, the peak queue depth prediction value of each priority queue of the gateway is taken from the gateway buffer congestion prediction result, denormalized and restored, and then normalized and accumulated using the mean normalization method based on the range.

8. The deterministic transmission optimization method for multimodal heterogeneous networks according to claim 6, characterized in that, The constraints of the integer linear programming problem include: Each data stream selects one and only one path: ; The end-to-end delay estimates of the inter-domain delay compensation for each flow do not exceed the deterministic delay boundary: ,in This is the estimated end-to-end delay. For data stream Deterministic time delay boundary; The total bandwidth usage of each shared port within the time window shall not exceed the capacity limit; The depth of each gateway's buffer queue does not exceed the upper limit of the queue depth.

9. The deterministic transmission optimization method for multimodal heterogeneous networks according to claim 1, characterized in that, Also includes: After the global optimal path allocation scheme is issued and executed, the actual queuing delay data of the gateways between each domain is continuously collected. The actual queuing delay is standardized and then compared with the corresponding predicted value in the gateway buffer congestion prediction result to calculate the deviation. The deviation is fed back as an online correction signal to the time series prediction model based on sliding window attention to update the model parameters, and the time window resource occupancy matrix of each port is updated based on the actual traffic allocation. The absolute value of the deviation in each consecutive scheduling cycle is accumulated. When the accumulated value exceeds a preset threshold, the optimization process from path filtering to path allocation solution is triggered to be re-executed.

10. A deterministic transmission optimization system for multimodal heterogeneous networks, used to execute the deterministic transmission optimization method for multimodal heterogeneous networks according to any one of claims 1 to 9, characterized in that, include: The data acquisition module is used to collect the time window resource occupancy matrix of each switch port in the TSN domain and the buffer status time-series characteristic sequence of each domain gateway; The path filtering module is used to filter candidate paths based on hop-by-hop temporal alignment constraints and generate a set of feasible candidate paths for a time window. The arrival distribution extraction module is used to extract the arrival time set of each relevant data stream from the candidate path that needs to cross the inter-domain gateway, and generate the gateway-related arrival time distribution. The buffer congestion prediction module is used to input the gateway correlation arrival time distribution and the buffer state temporal feature sequence into the time-series prediction model based on sliding window attention, and output the gateway buffer congestion prediction result. The delay compensation calculation module is used to calculate the inter-domain delay compensation amount for each candidate path based on the gateway buffer congestion prediction results. The path allocation optimization module is used to model path allocation as an integer linear programming problem, and solves it with the weighted sum of port time window coupling cost and gateway buffer congestion cost as the objective function to generate the globally optimal path allocation scheme. The online correction module is used to calculate the deviation between the actual queuing delay and the gateway buffer congestion prediction result, feed it back to the time-series prediction model to update the parameters, and trigger the re-execution of path allocation optimization when the accumulated deviation exceeds a preset threshold.

Images