A Traffic Scheduling Method for TSN-5G High-Speed ​​Rail Onboard Network Based on Reinforcement Immune Algorithm

Abstract

This invention provides a traffic scheduling method for TSN-5G high-speed rail onboard networks based on a reinforcement immune algorithm. The method includes: obtaining all possible routes for the set of traffic to be scheduled in the TSN-5G high-speed rail onboard network, and the latency of each route; setting various operators for the reinforcement immune algorithm, using a DQN network as the cloning and mutation operator for the reinforcement immune algorithm, and comprehensively considering latency and load as the optimization objective of the reinforcement immune algorithm; the reinforcement immune algorithm iteratively using various operators to obtain the transmission order and route of each traffic, as well as the transmission time of each traffic at the TSN switch; obtaining a gated scheduling list for the TSN switch based on the transmission time of each traffic; and the TSN switch performing end-to-end deterministic transmission of each traffic according to the gated scheduling list. This invention, by combining an immune algorithm and reinforcement learning, overcomes the shortcomings of existing scheduling methods, improves scheduling efficiency and flexibility, and achieves deterministic scheduling of critical traffic within the TSN-5G train onboard network.

Description

Technical Field

[0001] This invention relates to the field of vehicle-mounted network technology, and in particular to a TSN-5G high-speed rail vehicle-mounted network traffic scheduling method based on a reinforced immune algorithm. Background Technology

[0002] Currently, due to the complex deployment environment of vehicle-mounted terminals, numerous influencing factors such as electromagnetic interference, dust and oil contamination, vehicle vibration, and signal obstruction cause significant uncertainties in vehicular 5G wireless networks, including node failures and link failures. In TSN (Time-Sensitive Networking)-5G train-mounted networks, a fast-responding and dynamically adjustable traffic scheduling algorithm is urgently needed to ensure deterministic transmission of critical service flows.

[0003] Numerous scholars both domestically and internationally have conducted research on traffic scheduling in TSN-5G networks. One approach uses optimization module theory to define the joint message segmentation and scheduling problem and leverages readily available solvers to find the optimal solution. This research also proposes a heuristic algorithm that constructs low-latency scheduling through worst-case latency analysis, significantly improving scheduling performance.

[0004] To address the routing and scheduling problem of dynamic traffic, a joint optimization model for routing and scheduling in TSN-5G networks has been established, and an online heuristic algorithm has been proposed. This scheme considers both transmission latency and network load factors to determine the route and utilizes two pruning operations to quickly determine the scheduling time, ensuring the transmission performance of real-time traffic in dynamically changing network environments.

[0005] To address the wait-free packet scheduling problem in TSN-5G networks, a solution maps it to the wait-free job shop scheduling problem (NW-JSP) and proposes a heuristic optimization method based on tabu search. Furthermore, the proposed scheduling compression technique effectively reduces the number of guard bands and improves bandwidth utilization.

[0006] One approach introduces ant colony optimization (ACO) into TSN-5G networks, proposing an improved ACO as a scheduling algorithm for time-triggered flows. Simulation results show that this algorithm meets the stringent requirements of end-to-end latency in industrial applications and outperforms the traditional ACO algorithm in terms of convergence speed and optimization capability.

[0007] In traffic scheduling algorithms for TSN-5G networks, some schemes have found that TSN flows, when transmitted in 5G bridges, can compete with 5G eMBB flows for time slots and channel resources, impacting network throughput and end-to-end latency. Therefore, a transmission scheme is proposed that calculates the priorities of TSN and eMBB flows based on parameters such as latency and packet size, and allocates resources accordingly.

[0008] One approach utilizes Network Function Virtualization (NFV) technology to achieve QoS-aware mapping and scheduling in 5G-TSN networks. First, an incremental greedy algorithm is designed to map NFV to 5G and TSN resources. Then, a 5G resource scheduling scheme based on dynamic priority preemption is proposed to provide wait-free transmission for higher priority applications.

[0009] The shortcomings of the optimization theory algorithm for TSN-5G networks in the above-mentioned prior art include: although the method can provide the theoretical optimal solution, it faces a number of challenges in the TSN-5G vehicle network scenario, such as difficulty in setting constraints, long solution time, and it is not suitable for dynamic environments.

[0010] The drawbacks of the heuristic-based traffic scheduling algorithm for TSN-5G networks mentioned above include: the method has some inherent defects, such as difficulty in guaranteeing global optimality, strong parameter dependence, and slow convergence speed. Summary of the Invention

[0011] The embodiments of the present invention provide a traffic scheduling method for TSN-5G high-speed rail onboard network based on the enhanced immune algorithm, so as to achieve high-efficiency transmission of traffic in TSN-5G high-speed rail onboard network.

[0012] To achieve the above objectives, the present invention adopts the following technical solution.

[0013] A traffic scheduling method for TSN-5G high-speed rail onboard network based on reinforcement immune algorithm, comprising:

[0014] Obtain all possible routes for the set of traffic to be scheduled in the TSN-5G high-speed rail vehicle network, as well as the latency of each route;

[0015] Various operators for the reinforcement immune algorithm are set up, and the DQN network is used as the cloning mutation operator of the reinforcement immune algorithm. The optimization objective of the reinforcement immune algorithm is to minimize the weighted sum of end-to-end delay, load balancing index and routing hop count.

[0016] All possible routes of the traffic set to be scheduled, the latency of each route, and the network information of the TSN-5G high-speed rail vehicle network are input into the reinforcement immune algorithm. The reinforcement immune algorithm uses various operators to iteratively obtain the transmission order and route of each traffic, as well as the transmission time of each traffic in the time-sensitive network TSN switch.

[0017] The TSN switch obtains a gating schedule list based on the transmission time of each traffic flow, and the TSN switch performs end-to-end deterministic transmission of each traffic flow according to the gating schedule list.

[0018] Preferably, obtaining all possible routes for the set of traffic to be scheduled in the TSN-5G high-speed rail vehicle network, and the latency of each route, includes:

[0019] Let the flow rate be f i There are n possible routes in the TSN-5G high-speed rail vehicle network, denoted as R, r ij (j∈[1,2,...,n]) represents a route in n, starting from the sending end. After passing through k intermediate nodes Arrival at the receiving end Then route r ij Represented as

[0020] Configure the end-to-end latency for each route, including TSN network latency and 5G network latency, and the traffic f. i The latency of selecting the j-th route in the TSN network It consists of four parts, namely, transmission delay. Transmission delay Processing latency and queuing delay Latency in TSN networks The calculation formula is:

[0021]

[0022] Flow f i Latency in 5G networks It consists of four parts, namely, transmission delay. Propagation delay Processing latency and queuing delay Latency in 5G networks The calculation formula is:

[0023]

[0024] Flow f i The end-to-end delay d for selecting the j-th route ij for:

[0025]

[0026] Flow f i The total queuing delay for selecting the j-th route is:

[0027]

[0028] 5G network queuing latency TTI stands for Transmission Time Interval;

[0029] TSN network queuing latency k is the flow rate f i Select the number of TSN switches that the j-th route passes through.

[0030] Preferably, the various operators of the reinforcement immune algorithm are configured, using a DQN network as the cloning and mutation operator of the reinforcement immune algorithm, and the optimization objective of the reinforcement immune algorithm is to minimize the weighted sum of end-to-end delay, load balancing index, and routing hop count, including:

[0031] The operators for setting up the enhanced immunization algorithm include: affinity calculation operator, antibody concentration calculation operator, stimulus calculation operator, immune selection operator, DQN-based clonal mutation operator, and population refresh operator;

[0032] (8) Population initialization

[0033] Antibodies within population P are discretely encoded, with each antibody p... i Record one possible sending order s of the set of streams F to be scheduled. i and route number r i ;

[0034] (9) Affinity Evaluation Operator

[0035] Calculate the end-to-end delay D of the x-th antibody based on the transmission order and routing number of the antibody records. x Load balancing index V x and routing hop count H x The affinity aff(x) is the optimization objective. The algorithm's goal is to iteratively find the antibody with the lowest affinity. The affinity calculation formula is:

[0036] aff(x)=ln(D x )+ln(V x )+ln(H x )

[0037] (10) Antibody concentration evaluation operator

[0038] Defined only if the sending order s of the intra-antibody flow is... i With route number r i When all are the same, the two antibodies are considered to be identical. The antibody concentration den(x) of the x-th antibody is defined as:

[0039]

[0040] (11) Excitation degree calculation operator

[0041] The activation degree sim(x) of the x-th antibody is:

[0042] sim(x)=w·aff(x)+(1-w)·den(x)

[0043] (12) Immune selection operator

[0044] Define an immune selection operator to select the top three antibodies with the highest activation level;

[0045] (13) Population refresh operator

[0046] The population refresh operator randomly generates new antibodies to replenish the population.

[0047] (14) Cloning and mutation operators based on DQN

[0048] The DQN algorithm in reinforcement learning is used as the cloning and mutation operator in the immune algorithm;

[0049] State space s: The current population's excitation distribution and the vector representation of antibodies;

[0050] Action space a: Defines the optional number of clones, mutation strategy, and mutation probability;

[0051] The number of clones ranges from 1 to Z, where Z = S / N, S is the number of antibodies to be cloned, and N is the population size.

[0052] There are three mutation strategies:

[0053] (4) No mutations were made.

[0054] (5) Change the flow scheduling order according to the mutation probability.

[0055] (6) Change traffic routing according to the mutation probability.

[0056] Reward function r: If the overall affinity of the population increases after cloning and mutation, a positive reward is given; otherwise, a negative reward is given.

[0057] Preferably, the optimization objective of the reinforcement immune algorithm is to minimize the weighted sum of end-to-end latency, load balancing index, and routing hop count, including:

[0058] Let the flow rate be f i When the j-th route is selected, the switch load is x. ij The number of hops is |r ij | Define the end-to-end delay of the entire set of traffic to be scheduled, F, as D = min(d ij The load balancing index is Route hop count The optimization objective of the enhanced immune algorithm is set to min(ln(D)+ln(V)+ln(H)), which means minimizing the weighted sum of end-to-end latency, load balancing index, and routing hop count.

[0059] Preferably, the step of inputting all possible routes of the traffic set to be scheduled, the latency of each route, and the network information of the TSN-5G high-speed rail vehicle network into the reinforcement immune algorithm, wherein the reinforcement immune algorithm uses various operators to iteratively obtain the transmission order and route of each traffic, as well as the transmission time of each traffic at the TSN switch, including:

[0060] The current TSN-5G vehicular network switch bandwidth, network topology, and scheduled flow information are input into the enhanced immune algorithm for population initialization, and antibody p is randomly generated. i s represents a possible sending order of the set of streams F to be scheduled. i and route number r i A population of antibodies is formed, and the end-to-end delay D of the stream is calculated using the sending order and routing number of the stream stored in each antibody. x Load balancing index V x and routing hop count H x The affinity operator is used to calculate the affinity of each antibody in the population, the antibody concentration evaluation operator is used to calculate the similarity between each antibody, and the activation value calculation operator is used to jointly calculate the antibody concentration and antibody affinity to obtain the final evaluation index for each antibody.

[0061] The immune selection operator selects excellent antibodies based on evaluation indicators. The cloning and mutation operator based on DQN adaptively clones and mutates the excellent antibodies to generate new antibodies. These new antibodies combine with the excellent antibodies to initially form a new population. The population refresh operator randomly constructs new antibodies to supplement or deletes excess antibodies to obtain the final new population. The above process is repeated. When the population affinity meets the termination condition, the population iteration update stops, and the sending order and route number of the scheduled streams stored in the current population antibodies are output.

[0062] Preferably, the step of obtaining the gated scheduling list of the TSN switch based on the transmission time of each traffic flow, and the TSN switch performing end-to-end deterministic transmission of each traffic flow according to the gated scheduling list, includes:

[0063] The transmission time of each TSN switch through which the traffic reaches its destination address is calculated based on the transmission order of the flow and the routing path pointed to by the routing number. The transmission time of each flow at each TSN switch is integrated to obtain the gating schedule table for each port of each TSN switch. When the traffic is sent by the sender, each TSN switch through which the traffic passes controls the transmission time of the traffic according to the gating schedule table, so as to achieve end-to-end deterministic transmission of traffic.

[0064] As can be seen from the technical solutions provided by the embodiments of the present invention above, the present invention proposes a traffic scheduling method based on reinforcement immune algorithm. By combining immune algorithm and reinforcement learning, it overcomes the shortcomings of existing scheduling methods, improves scheduling efficiency and flexibility, and realizes deterministic scheduling of critical traffic in TSN-5G train onboard network.

[0065] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0066] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is an architecture diagram of a 5G-TSN train onboard network provided in an embodiment of the present invention.

[0068] Figure 2 This is a flowchart illustrating a TSN-5G high-speed rail onboard network traffic scheduling method based on a reinforced immune algorithm, as proposed in an embodiment of the present invention.

[0069] Figure 3 A flowchart of a reinforcement immune algorithm provided for an embodiment of this method;

[0070] Figure 4 A flowchart of a DQN algorithm is provided for an embodiment of this method;

[0071] Figure 5 This is a population diagram provided for an embodiment of the present invention. Detailed Implementation

[0072] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0073] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0074] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0075] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0076] Immune algorithms, by simulating the process of antibody recognition, adaptation, and elimination of antigens in the biological immune system, possess good adaptability and diversity preservation capabilities. This invention improves the immune algorithm by introducing a reinforcement learning DQN (Deep Q-Network), resulting in a reinforced immune algorithm. During the iterative process of the reinforced immune algorithm, the reinforcement learning DQN network is introduced for auxiliary decision-making. Through continuous interaction with the immune algorithm population, the DQN network adaptively optimizes the decision strategy, significantly improving the search capability and convergence speed of the immune algorithm. The improvements include:

[0077] 1. Enhance global search capability: Reinforcement learning continuously interacts with the immune algorithm population and dynamically adjusts the search strategy to overcome the shortcomings of the immune algorithm in getting stuck in local optima in complex problems, thereby enhancing its global search capability.

[0078] 2: Improve convergence speed: Reinforcement learning can make decisions based on historical experience and continuously optimize the solution space exploration route of the immune algorithm, which greatly accelerates the convergence speed of the algorithm and shortens the solution time.

[0079] 3. Reduced Parameter Dependency: Using the DQN network as the cloning mutation operator in the immune algorithm significantly reduces the number of parameters that need to be adjusted, lowering the algorithm's sensitivity to parameter tuning. This not only simplifies the algorithm design but also improves its stability and adaptability.

[0080] An embodiment of the present invention provides an architecture for a 5G-TSN train onboard network as follows: Figure 1 As shown. Assume the set of traffic to be scheduled is F, and use a quintuple to represent one traffic item in F. in and prd represents the sender and receiver of the data stream. i Indicates the data stream sending period, size i The frame size of the data stream, ddl i This indicates the maximum allowable transmission delay for the data stream.

[0081] Based on the above Figure 1 The processing flow of the TSN-5G high-speed rail onboard network based on the reinforcement immune algorithm proposed in this embodiment of the invention is as follows: Figure 2 As shown, the processing steps include the following:

[0082] Step S10: Obtain all possible routes of the traffic set F to be scheduled in the TSN-5G high-speed rail vehicle network, as well as the latency of each route.

[0083] Step S20: Set up various operators for the reinforcement immune algorithm, using the DQN network as the cloning mutation operator for the reinforcement immune algorithm, and minimize the weighted sum of end-to-end delay, load balancing index, and routing hop count.

[0084] min(ln(D)+ln(V)+ln(H)) is used as the optimization objective of the reinforcement immune algorithm;

[0085] Step S30: Input all possible routes of the traffic set to be scheduled, the latency of each route and the network information (topology information, switch bandwidth, data transmission rate, etc.) of the TSN-5G high-speed rail vehicle network into the reinforcement immune algorithm. The reinforcement immune algorithm uses various operators to iteratively obtain the sending order and route of each traffic, as well as the sending time of each traffic in the TSN switch.

[0086] Step S40: Obtain the GCL (Gate Control List) of the TSN switch according to the transmission time of each traffic, and the TSN switch performs end-to-end deterministic transmission of each traffic according to the gate control list.

[0087] Step S10 above includes assuming the flow rate fi There are n possible routes in the TSN-5G high-speed rail vehicle network, denoted as R, r ij (j∈[1,2,...,n]) represents a route in n, starting from the sending end. After passing through k intermediate nodes Arrival at the receiving end Then route r ij It can be represented as

[0088] The end-to-end latency for each route includes TSN network latency and 5G network latency.

[0089] Flow f i The latency of selecting the j-th route in the TSN network It consists of four parts, namely, transmission delay. Transmission delay Processing latency and queuing delay Latency in TSN networks The calculation formula is:

[0090]

[0091] In a TSN-5G network, 5G only acts as a logical bridge, so the traffic f i Latency in 5G networks Only the queuing delay is affected by the selected route j, and it consists of four parts: transmission delay, transmission delay, and so on. Propagation delay Processing latency and queuing delay Latency in 5G networks The calculation formula is:

[0092]

[0093] Therefore, the flow f i The end-to-end delay d for selecting the j-th route ij for:

[0094]

[0095] Sending delay, transmission delay, and processing delay are mainly determined by link length or hardware performance; therefore, these delays are relatively fixed and not easily reduced through algorithmic optimization. Thus, this invention focuses on optimizing queuing delay. Queuing delay refers to the time a data packet waits for processing in the network node's buffer queue. It is affected by network traffic, task allocation, and scheduling strategies, and has significant room for optimization. (The flow f is mentioned in the original text, but its relevance to the preceding sentence is unclear.) i The total queuing delay for selecting the j-th route is:

[0096]

[0097] 5G data frames can only acquire resources to transmit data during intervals that are integer multiples of the Transmission Time Interval (TTI).

[0098] The TSN queuing delay is the sum of the queuing delays of each switch along the path, therefore k is the flow rate f i Select the number of TSN switches that the j-th route passes through.

[0099] To improve network performance and resource utilization efficiency, avoid single points of failure due to overload of a switch or link, reduce latency and congestion, increase throughput and response speed, ensure the QoS (Quality of Service) of critical services, and enhance network stability and scalability, this invention also considers the load and routing hop count of TSN switches. (The flow f is then used.) i When the j-th route is selected, the switch load is x. ij The number of hops is |r ij |

[0100] Step S20 above includes: the processing flow of an enhanced immune algorithm provided in this embodiment of the method is as follows: Figure 3 As shown, the evolutionary optimization process of the immune algorithm is implemented through operators.

[0101] The enhanced immunization algorithm designed in this invention includes the following operators: affinity calculation operator, antibody concentration calculation operator, activation degree calculation operator, immune selection operator, DQN-based clonal mutation operator, and population refresh operator.

[0102] (1) Population initialization

[0103] Antibodies within population P are discretely encoded, with each antibody p... i Record one possible sending order s of the set of streams F to be scheduled. i and route number r i .

[0104] (2) Affinity Evaluation Operator

[0105] Calculate the end-to-end delay D of the x-th antibody based on the transmission order and routing number of the antibody records. x Load balancing index V x and routing hop count H x The affinity aff(x) is the optimization objective, so the goal of this algorithm is to iteratively find the antibody with the minimum affinity. The formula for calculating affinity is:

[0106] aff(x)=ln(D x )+ln(V x )+ln(H x )

[0107] (3) Antibody concentration evaluation operator

[0108] Defined only if the sending order s of the intra-antibody flow is... i With route number r i When all are the same, the two antibodies are considered to be identical. Therefore, the antibody concentration den(x) of the x-th antibody is defined as:

[0109]

[0110] (4) Excitation degree calculation operator

[0111] Antibody activation degree is the final evaluation result of antibody quality, which requires comprehensive consideration of antibody affinity and antibody concentration. Generally, antibodies with high affinity and low concentration will obtain a higher activation degree. Antibody activation degree can usually be obtained by weighted summation of antibody affinity and antibody concentration. Therefore, the activation degree sim(x) of the x-th antibody is:

[0112] sim(x)=w·aff(x)+(1-w)·den(x)

[0113] (5) Immune selection operator

[0114] Define an immune selection operator to select the top three antibodies with the highest activation levels.

[0115] (6) Population refresh operator

[0116] Adding cloned and mutated antibodies to a new population may result in an insufficient population size. A population refresh operator is needed to randomly generate new antibodies to replenish the population. This helps maintain antibody diversity, enables global search, and allows exploration of new feasible solution space regions.

[0117] (7) Cloning and mutation operators based on DQN

[0118] The cloning operator replicates the antibody individuals selected by the immune selection operator. In immune algorithms, the cloning operator enhances the diversity and efficiency of the search process. By replicating individuals with high fitness and then mutating them, the cloning operator can explore different regions of the solution space, which is key to accelerating convergence and avoiding local optima. Traditional statically determined cloning operators limit the algorithm's ability to explore the solution space and are prone to getting trapped in local optima. Furthermore, traditional immune algorithms consider the cloning operator and mutation operator separately, lacking collaborative optimization and making it difficult to find suitable operators.

[0119] Figure 4 This invention provides a flowchart of a DQN algorithm for an embodiment of the method. Therefore, this invention proposes using the DQN algorithm from reinforcement learning as the cloning and mutation operator in the immune algorithm. DQN evaluates the value of each antibody through a neural network to determine the number of clones and mutation strategy for that antibody. This allows for better exploration of different combinations of clones and mutations, helping the immune algorithm converge to the optimal solution more quickly.

[0120] State space s: The current population's excitation distribution and the vector representation of antibodies.

[0121] Action space a: Defines the optional number of clones, mutation strategy, and mutation probability.

[0122] The number of clones ranges from 1 to Z, where Z = S / N, S is the number of antibodies to be cloned, and N is the population size.

[0123] There are three mutation strategies:

[0124] (7) No variations were made.

[0125] (8) Change the flow scheduling order according to the mutation probability.

[0126] (9) Change traffic routing according to the mutation probability.

[0127] Reward function r: If the overall affinity of the population increases after cloning and mutation, a positive reward is given; otherwise, a negative reward is given.

[0128] Define the end-to-end delay of the entire set of traffic to be scheduled, F, as D = min(d ij The load balancing index is Route hop count Since the numerical differences of these three parameters are too large, the natural logarithm is taken to reduce the numerical differences, and the optimization objective is min(ln(D)+ln(V)+ln(H)).

[0129] Step S30 above includes: first, acquiring information such as the current TSN-5G vehicle network switch bandwidth and network topology, and then acquiring the flow information to be scheduled. Next, population initialization is performed, and antibody p is randomly generated. i s represents a possible sending order of the set of streams F to be scheduled. i and route number r i A population is formed by a number of antibodies. An example population diagram provided in this invention is shown below. Figure 5 As shown. Then, the end-to-end delay D of the stream is calculated using the sending order and route number of the stream stored in each antibody. x Load balancing index V x and routing hop count H xNext, the affinity operator is used to calculate the affinity of each antibody in the population. Then, the antibody concentration evaluation operator is used to calculate the similarity between each antibody. Finally, the activation value calculation operator is used to jointly calculate the antibody concentration and antibody affinity to obtain the final evaluation index for each antibody.

[0130] The immune selection operator selects excellent antibodies based on evaluation indicators. The cloning and mutation operator based on DQN adaptively clones and mutates the excellent antibodies to generate new antibodies. These new antibodies combine with the excellent antibodies to initially form a new population. At this time, the number of antibodies in the new population may be insufficient or excessive. The population refresh operator needs to randomly construct new antibodies to make up the difference or delete excess antibodies to obtain the final new population. The above process is repeated. The population iteration update stops when the population affinity meets the termination condition. The sending order and route number of the scheduled streams stored in the current population antibodies are output.

[0131] Step S40 above includes: calculating the transmission time of each TSN switch that the traffic passes through to reach the destination address based on the transmission order of the flow and the routing path pointed to by the routing number; integrating the transmission time of each flow at each TSN switch to obtain the GCL gating schedule table for each port of each TSN switch; when the traffic is sent by the sender, each TSN switch will control the transmission time of the traffic according to the GCL gating schedule table to ensure deterministic transmission of the traffic.

[0132] In summary, this invention, by constructing a latency model for the TSN-5G vehicular network, clarifies that the key factors affecting end-to-end latency are the queuing latency of the TSN switch and the 5G base station. Taking into account important factors such as switch load and routing hop count, a synchronization scheduling method based on a reinforced immune algorithm is proposed. This method effectively reduces latency and congestion in the network, ensuring network stability and reliability under high load scenarios. It is particularly capable of handling the complexity and dynamic changes of the TSN-5G vehicular network, ultimately achieving deterministic scheduling of critical train traffic and meeting the requirements for high real-time communication.

[0133] This invention integrates cloning and mutation operators into the DQN network, leveraging the adaptive capabilities of deep learning to flexibly adjust decision-making strategies based on changes in the real-time network environment. Through the integrated DQN cloning and mutation operators, the global search capability of the immune algorithm is enhanced, and population diversity is significantly improved, enabling the algorithm to explore different scheduling schemes more efficiently when facing complex and ever-changing network conditions.

[0134] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0135] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0136] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0137] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A traffic scheduling method for TSN-5G high-speed rail onboard network based on reinforcement immune algorithm, characterized in that, include: Obtain all possible routes for the set of traffic to be scheduled in the TSN-5G high-speed rail vehicle network, as well as the latency of each route; Various operators for the reinforcement immune algorithm are set up, and the DQN network is used as the cloning mutation operator of the reinforcement immune algorithm. The optimization objective of the reinforcement immune algorithm is to minimize the weighted sum of end-to-end delay, load balancing index and routing hop count. All possible routes of the traffic set to be scheduled, the latency of each route, and the network information of the TSN-5G high-speed rail vehicle network are input into the reinforcement immune algorithm. The reinforcement immune algorithm uses various operators to iteratively obtain the transmission order and route of each traffic, as well as the transmission time of each traffic in the time-sensitive network TSN switch. The TSN switch obtains a gating schedule list based on the transmission time of each traffic flow, and the TSN switch performs end-to-end deterministic transmission of each traffic flow according to the gating schedule list. The process involves inputting all possible routes of the traffic set to be scheduled, the latency of each route, and the network information of the TSN-5G high-speed rail vehicle network into the reinforcement immune algorithm. The reinforcement immune algorithm iteratively calculates the transmission order and route of each traffic item, as well as the transmission time of each traffic item at the TSN switch, using various operators. This includes: The current TSN-5G vehicular network switch bandwidth, network topology, and scheduled flow information are input into the enhanced immune algorithm for population initialization, and antibody p is randomly generated. i s represents a possible sending order of the set of streams F to be scheduled. i and route number r i A population of antibodies is formed, and the end-to-end delay D of the stream is calculated using the sending order and routing number of the stream stored in each antibody. x Load balancing index V x and routing hop count H x The affinity operator is used to calculate the affinity of each antibody in the population, the antibody concentration evaluation operator is used to calculate the similarity between each antibody, and the activation value calculation operator is used to jointly calculate the antibody concentration and antibody affinity to obtain the final evaluation index for each antibody. The immune selection operator selects excellent antibodies based on evaluation indicators. The cloning and mutation operator based on DQN adaptively clones and mutates the excellent antibodies to generate new antibodies. These new antibodies combine with the excellent antibodies to initially form a new population. The population refresh operator randomly constructs new antibodies to supplement or deletes excess antibodies to obtain the final new population. The above process is repeated. When the population affinity meets the termination condition, the population iteration update stops, and the sending order and route number of the scheduled streams stored in the current population antibodies are output.

2. The method according to claim 1, characterized in that, The acquisition of all possible routes for the set of traffic to be scheduled in the TSN-5G high-speed rail vehicle network, and the latency of each route, includes: Let the flow rate be f i There are n possible routes in the TSN-5G high-speed rail vehicle network, denoted as R, r ij (j∈[1,2,...,n]) represents a route in n, starting from the sending end. After passing through k intermediate nodes Arrival at the receiving end Then route r ij Represented as Configure the end-to-end latency for each route, including TSN network latency and 5G network latency, and the traffic f. i The latency of selecting the j-th route in the TSN network It consists of four parts, namely, transmission delay. Transmission delay Processing latency and queuing delay Latency in TSN networks The calculation formula is: Flow f i Latency in 5G networks It consists of four parts, namely, transmission delay. Propagation delay Processing latency and queuing delay Latency in 5G networks The calculation formula is: Flow f i The end-to-end delay d for selecting the j-th route ij for: Flow f i The total queuing delay for selecting the j-th route is: 5G network queuing latency X satisfies The smallest integer, where TTI is the transmission time interval; TSN network queuing latency k is the flow rate f i Select the number of TSN switches that the j-th route passes through.

3. The method according to claim 2, characterized in that, The various operators for setting up the reinforcement immune algorithm, using the DQN network as the cloning mutation operator of the reinforcement immune algorithm, and taking the minimum weighted sum of end-to-end delay, load balancing exponent, and routing hop count as the optimization objective of the reinforcement immune algorithm, include: The operators for setting up the enhanced immunization algorithm include: affinity calculation operator, antibody concentration calculation operator, stimulus calculation operator, immune selection operator, DQN-based clonal mutation operator, and population refresh operator; (1) Population initialization Antibodies within population P are discretely encoded, with each antibody p... i Record one possible sending order s of the set of streams F to be scheduled. i and route number r i ; (2) Affinity Evaluation Operator Calculate the end-to-end delay D of the x-th antibody based on the transmission order and routing number of the antibody records. x Load balancing index V x and routing hop count H x The affinity aff(x) is the optimization objective. The algorithm's goal is to iteratively find the antibody with the lowest affinity. The affinity calculation formula is: aff(x)=ln(D x )+ln(V x )+ln(H x ) (3) Antibody concentration evaluation operator Defined only if the sending order s of the intra-antibody flow is... i With route number r i When all are the same, the two antibodies are considered to be identical. The antibody concentration den(x) of the x-th antibody is defined as: (4) Excitation degree calculation operator The activation degree sim(x) of the x-th antibody is: sim(x)=w·aff(x)+(1-w)·den(x) (5) Immune selection operator Define an immune selection operator to select the top three antibodies with the highest activation level; (6) Population refresh operator The population refresh operator randomly generates new antibodies to replenish the population. (7) Cloning and mutation operators based on DQN The DQN algorithm in reinforcement learning is used as the cloning and mutation operator in the immune algorithm; State space s: The current population's excitation distribution and the vector representation of antibodies; Action space a: Defines the optional number of clones, mutation strategy, and mutation probability; The number of clones ranges from 1 to Z, where Z = S / N, S is the number of antibodies to be cloned, and N is the population size. There are three mutation strategies: (1) No variations were made; (2) Change the flow scheduling order according to the mutation probability; (3) Change the traffic routing according to the mutation probability; Reward function r: If the overall affinity of the population increases after cloning and mutation, a positive reward is given; otherwise, a negative reward is given.

4. The method according to claim 3, characterized in that, The optimization objective of the reinforcement immune algorithm, which is to minimize the weighted sum of end-to-end latency, load balancing index, and routing hop count, includes: Let the flow rate be f i When the j-th route is selected, the switch load is x. ij The number of hops is |r ij | Define the end-to-end delay of the entire set of traffic to be scheduled, F, as D = min(d ij The load balancing index is Route hop count The optimization objective of the enhanced immune algorithm is set to min(ln(D)+ln(V)+ln(H)), which means minimizing the weighted sum of end-to-end latency, load balancing index, and routing hop count.

5. The method according to claim 1, characterized in that, The process of obtaining a gated scheduling list for the TSN switch based on the transmission time of each traffic flow, and the TSN switch performing end-to-end deterministic transmission of each traffic flow according to the gated scheduling list, includes: The transmission time of each TSN switch through which the traffic reaches its destination address is calculated based on the transmission order of the flow and the routing path pointed to by the routing number. The transmission time of each flow at each TSN switch is integrated to obtain the gating schedule table for each port of each TSN switch. When the traffic is sent by the sender, each TSN switch through which the traffic passes controls the transmission time of the traffic according to the gating schedule table, so as to achieve end-to-end deterministic transmission of traffic.

Images