Digital twinning-oriented hybrid service deterministic transmission scheduling method

By building a timely network and deterministic Internet interconnection protocol of virtual and real mapping in railway operation and maintenance, combined with Markov decision-making process and D3QN algorithm, deterministic transmission scheduling of hybrid service flows is realized, solving the low transmission scheduling efficiency and network The problem of oscillation increases delay and returns.

CN119012237AActive Publication Date: 2024-11-22BEIJING JIAOTONG UNIV

Patent Information

Application Number
CN202411108144.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-11-22
Estimated Expiration
2044-08-13

AI Technical Summary

Technical Problem

The existing railway operation and maintenance transmission scheduling methods are difficult to meet the needs of low latency, high reliability and differentiated service. In a non-stationary actual network environment, pre-trained scheduling strategies cannot ensure service quality, resulting in inefficiency and network oscillation.

Method used

Using a hybrid service deterministic transmission scheduling method for digital twins, the network architecture of TSN+deterministic Internet interconnection protocol DIP with virtual and real mapping is proposed, and the deterministic scheduling model and business flow model under the digital twin architecture will be mixed. The cross-domain problem of business flow is transformed into constraint problem, and online resource allocation is used to achieve optimal end-to-end transmission scheduling of hybrid service flows.

Benefits of technology

Under the conditions of ensuring successful scheduling of service flows, low end-to-end overall delay and higher transmission returns are achieved, transmission efficiency and system performance are improved, and differentiated needs of different service flows are met.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119012237A_ABST
    Figure CN119012237A_ABST
Patent Text Reader

Abstract

The invention provides a hybrid service deterministic transmission scheduling method oriented to digital twinning. The method comprises the following steps: constructing a network scene of a TSN + DIP network architecture of virtual-real mapping, and proposing a deterministic scheduling model and a service flow model under a digital twinning architecture; a mixed service flow cross-domain problem in construction of the railway operation and maintenance scene is converted into a constraint problem; a service flow end-to-end deterministic scheduling process is modeled as a Markov decision process, and an online mixed service flow end-to-end transmission scheduling method based on a breach dual-depth Q network D3QN is proposed to obtain an optimal resource allocation deployment result. According to the method provided by the invention, in a railway operation and maintenance scene, aiming at the end-to-end transmission condition of the mixed service flow, the differentiated requirements of different service flows are considered, and lower end-to-end overall time delay and higher transmission benefit are realized under the condition of ensuring that the service flows are successfully scheduled.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of railway operation and maintenance technology, and in particular to a hybrid business deterministic transmission scheduling method for digital twins. Background Art

[0002] In recent years, technologies such as big data and artificial intelligence have flourished, gradually becoming research hotspots and being applied to railway intelligent operation and maintenance. The data in these scenarios are often relatively precise and require efficient and reliable transmission, while the traditional "best effort" network is difficult to meet the needs of various businesses for low latency, high reliability and differentiated services. In this context, deterministic scheduling technology has been proposed. The key to deterministic scheduling technology is to achieve deterministic latency and jitter.

[0003] However, existing studies have focused on transmission scheduling in physical networks. On the one hand, the algorithm lacks the perception of real-time changes. In fact, the communication environment in the actual network is non-stationary, and the pre-trained scheduling strategy cannot ensure the quality of service (QoS) requirements; on the other hand, the business flow in railway operation and maintenance has strict transmission requirements. Directly applying transmission scheduling methods in the physical world may lead to inefficiency and even cause serious network shocks. As an efficient solution, digital twin technology can reflect and predict the state of real space entities with high accuracy by mapping the physical world and the virtual world. By observing peers in the virtual space, it is widely used in the manufacturing industry.

[0004] Digital twin technology also provides a new opportunity for the intelligent development of railway operation and maintenance. IoT sensing devices are widely deployed at various levels such as railway stations to realize the digitization of railway operation and maintenance elements. Through data transmission and processing of the IoT, communication network and bearer network, a digital railway that is a "twin" of the physical railway is formed. Reliable decisions are made based on the twin and the application is synchronized to the entity after QoS is guaranteed, thus realizing intelligent structural monitoring and intelligent management decision-making. Summary of the invention

[0005] An embodiment of the present invention provides a hybrid service deterministic transmission scheduling method for digital twins, so as to effectively improve the end-to-end transmission revenue of the hybrid service.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme.

[0007] A hybrid service deterministic transmission scheduling method for digital twins, comprising:

[0008] Constructing a network scenario of the network architecture of TSN+DIP, a time-sensitive network for virtual-to-real mapping;

[0009] Based on the TSN+DIP network architecture of virtual-real mapping, a deterministic scheduling model and business flow model under the digital twin architecture are proposed;

[0010] Based on the deterministic scheduling model and business flow model under the digital twin architecture, the cross-domain problem of hybrid business flows in the railway operation and maintenance scenario is transformed into a constraint problem;

[0011] Based on the constraint problem, the end-to-end deterministic scheduling process of the business flow is modeled as a Markov decision process;

[0012] Based on the above Markov decision process, an online hybrid service flow end-to-end transmission scheduling method based on dueling dual deep Q network D3QN is proposed to obtain the optimal hybrid service resource allocation and deployment result.

[0013] Preferably, the network scenario of constructing the network architecture of the time-effective network TSN+deterministic internet interconnection protocol DIP for virtual-real mapping includes:

[0014] Set the physical network to consist of source TSN1, DIP domain and peer TSN2 domain. The switches in the TSN domain run the cyclic queuing and forwarding CQF mechanism. The DIP domain runs the cyclic specific queuing and forwarding CSQF mechanism. Time synchronization is implemented in the TSN domain and frequency synchronization is implemented in the DIP domain.

[0015] Both CQF and CSQF divide time into multiple equal-length time slots and abstract the entire network into a directed connected graph G = {V, E}, where V is the set of network nodes and E = {e|e = (v i ,v j ), i≠j} is a link set consisting of nodes, node v i ,v j are the source endpoint and end point of link e respectively, and the delay on link e is d e , delay d e Includes processing delay and queuing delay at upstream nodes;

[0016] The virtual space is mapped from the physical space. By acquiring data from various sensors deployed in the physical space, the data access side, core network and data analysis side that depict the geographic location are constructed to determine the topological structure of the TSN network and the DIP network.

[0017] Preferably, the TSN+DIP network architecture based on the virtual-to-real mapping proposes a deterministic scheduling model and a service flow model under the digital twin architecture, including:

[0018] Setting up a deterministic scheduling model under the digital twin architecture includes: the service flow is transmitted from the source TSN1 domain across the DIP domain to the peer TSN2 domain, and the transmission path of the service flow is p = {v0,v1,...,vp}, where the node set (v0,v1,...,v a ) is located in the TSN1 domain, and the set of nodes in the DIP domain in the transmission path is (v a+1 ,v a+2 ,...,v b ), the set of nodes in the TSN2 domain in the transmission path is (v b+1 ,v b+2 ,...,v p ), the transmission periods of the super-periodic HP source TSN1 and the peer TSN2 are defined as T TSN1 and T TSN2 , the transmission period of DIP is T DIP , the mapping relationship is:

[0019] T HP =T TSN1 N TSN1 =T TSN2 N TSN2 =T DIP N DIP

[0020] Among them, N TSN1 、N TSN2 、N DIP is the number of cycles forwarded by different domain nodes within a supercycle;

[0021] In the TSN domain, the CQF mechanism is run to synchronize queue entry and exit, and two queues are reserved. In the DIP domain, the CSQF mechanism is used for macro scheduling. The service flow scheduling consists of TSN1 domain transmission, TSN1-DIP cross-domain mapping, DIP domain transmission, DIP-TSN2 cross-domain mapping, and TSN2 domain transmission.

[0022] The business flow model under the digital twin architecture includes: F = {f1,f2,...,f |F|} is a set of business flows, each business flow f sent by the source station i By quintuple definition, They are the transmission starting point and end point of the business flow respectively. is the data size contained in the business flow, is the maximum end-to-end delay of the service flow, For the benefits of successful scheduling;

[0023] The service flows are roughly divided into control and execution service flows that are sensitive to latency, monitoring and data collection flows that have relatively high bandwidth requirements, and services for data analysis and service optimization, which correspond to the time-triggered TT flow, audio and video bridge AVB flow, and BE flow in TSN respectively. The importance of each service flow is distinguished by the benefit value of successful transmission:

[0024]

[0025] Preferably, the deterministic scheduling model and business flow model based on the digital twin architecture transform the cross-domain problem of hybrid business flows in the railway operation and maintenance scenario into a constraint problem, including:

[0026] Based on the deterministic scheduling model and business flow model under the digital twin architecture, the hybrid business flow cross-domain problem in the railway operation and maintenance scenario is transformed into a constraint problem. The constraint problem includes:

[0027] 1: Delay constraint:

[0028] Business Flow i The total delay of cross-domain transmission is are the transmission delays within the three domains, It is the cross-domain transmission delay;

[0030] In the TSN1 domain, service flow f i From arriving at v0 to leaving v a The delay is

[0031] When TSN1-DIP is transmitted across domains, the service flow f i From leaving v a To leave v a+1 The delay is:

[0032]

[0033] In the DIP domain, service flow f i From leaving v a+1 To leave v b The delay is:

[0034]

[0035] When DIP-TSN2 transmits across domains, the service flow f i From leaving v b To leave v b+1 The delay is:

[0036]

[0037] In the TSN2 domain, service flow f i From leaving v b To leave The delay is

[0038] For business flow i The end-to-end delay constraint is

[0039] 2: Queue capacity constraint: The cache capacity of the node queue determines the upper limit of the business flow that can be transmitted. is scheduled to node v in the same cycle j business flow, For node v j The cache capacity of a single queue, when node v j ∈TSN domain, m∈{1,2}; when node v j ∈DIP domain, m∈{1,2,3,4};

[0040] 3: Objective function: maxF = α*|P| + β / D

[0041] Where α+β=1, |P| is the number of time-sensitive flows that are successfully scheduled, and D is the total transmission delay.

[0042] Preferably, based on the constraint problem, the end-to-end deterministic scheduling process of the service flow is modeled as a Markov decision process, including:

[0043] The state space is composed of the remaining buffer capacity of each network node queue and the service flow transmission requirements. The state space includes: the remaining buffer capacity of each network node queue and business flow transmission requirements in It is expressed as:

[0044]

[0045] in, Represents node v i The remaining cache capacity of queue j.

[0046] The action space consists of paths and queues: The action space is characterized by a two-dimensional tuple (path, queue). The agent takes an action vector a t Guide the transmission of business flows. At the beginning of the transmission, extract the transmission start and end points of the business flow, and obtain all optional transmission paths for the business flow. i ∈PATH i For business flow f i All optional transmission paths, For node v i The queue number of the

[0047] The reward function considering the transmission delay ratio is set as:

[0048]

[0049] Preferably, based on the above Markov decision process, an online hybrid service flow end-to-end transmission scheduling method based on D3QN is proposed to obtain the optimal hybrid service resource allocation and deployment result, including:

[0050] Initialize D3QN algorithm parameters: Initialize controller parameters, including the experience replay pool D size N, initialize the current Q network parameters θ and target Q network parameters θ′, and the initial exploration rate ε ini , the final exploration rate ε fin , the number of samples taken at a time is m, and the number of synchronization rounds is L;

[0051] The agent conducts action exploration: selects actions based on the current strategy and exploration mechanism x∈[0,1) is the random variable generated at each step;

[0052] The agent performs action a t , get real-time rewards t and the new environment state s t+1 , d t Indicates whether this exploration is accepted;

[0053] Experience replay mechanism: convert the quintuple (s t ,a t ,r t ,s t+1 ,d t ) is stored in the playback memory and the system status is updated: t ←s t+1 , if the memory is full, discard the old tuple, randomly extract m samples from the memory, and calculate y j and Loss;

[0054]

[0055]

[0056] Neural network parameter update: Perform gradient descent on the Loss function to update the current Q network parameters, and synchronize the current Q network parameters to the target Q network every L steps.

[0057] The above process is repeated until the iteration upper limit reaches the set value, and finally the optimal hybrid service resource allocation scheme can be obtained, which includes the transmission path allocated to each service flow and the queue number allocated to each node in the path.

[0058] It can be seen from the technical solutions provided by the above-mentioned embodiments of the present invention that the present invention provides a hybrid business deterministic transmission scheduling method for digital twins, which is used for end-to-end transmission of hybrid business flows in railway operation and maintenance scenarios, taking into account the differentiated needs of different business flows, and achieving lower end-to-end overall latency and higher transmission benefits while ensuring that the business flows are successfully scheduled.

[0059] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0061] Figure 1 This is a schematic diagram of an implementation principle of a hybrid service deterministic transmission scheduling method for digital twins in an embodiment of the present invention;

[0062] Figure 2 This is a processing flow chart of a hybrid service deterministic transmission scheduling method for digital twins in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0064] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when the present invention refers to an element being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0065] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0066] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0067] The embodiment of the present invention provides a hybrid service deterministic transmission scheduling method for digital twins. Figure 1 As shown, it can be used in scenarios such as railway operation and maintenance. The processing flow of this method is as follows Figure 2 As shown, the processing steps include the following:

[0068] Step S1, constructing a network scenario of a network architecture of "TSN (Time-Sensitive Networking) + DIP (Deterministic Internet Protocol)" for virtual-real mapping.

[0069] The physical network consists of the source TSN1, DIP domain and the peer TSN2 domain. The TSN domain has fewer hops and each sensor is directly connected to the TSN switch. The TSN switch runs the cyclic queuing and forwarding (CQF) mechanism. The DIP domain has more hops and converges business flows, and runs a cyclic specified queuing and forwarding (CSQF) mechanism to achieve multi-queue caching.

[0070] Time synchronization is implemented in the TSN domain, while frequency synchronization is implemented in the DIP domain. Both CQF and CSQF mechanisms divide time into multiple equal-length time slots for formal description, abstracting the entire network into a directed connected graph G = {V, E}, where V is the set of network nodes, E = {e|e = (v i ,v j ), i≠j} is a link set consisting of nodes, node v i ,v j are the source endpoint and end point of link e respectively, and the delay on link e is d e , delay d e Includes processing delay and queuing delay at upstream nodes.

[0071] The virtual space is obtained by high-precision mapping of the physical space. By acquiring data from various sensors deployed in the physical space, the data access side, core network and data analysis side that depict the geographic location are constructed, the topology of the TSN network and the DIP network is determined, and the performance of sensors, switches, routers, etc. is extracted to obtain the model data of the virtual space; at the same time, a deterministic end-to-end transmission strategy based on transmission knowledge and evaluation knowledge is deployed to achieve all-round high-precision synchronization of virtual and real spaces. In the virtual space, E2ETSM-OMT is used to centrally schedule deterministic business flows under limited resources, while ensuring the scheduling success rate, minimizing the overall delay and maximizing the benefits. Finally, the strategy that tends to be stable in training is synchronized to the physical space to meet the actual railway operation and maintenance requirements for transmission effectiveness and reliability.

[0072] Step S2: Based on the "TSN+DIP" network architecture of virtual-real mapping, a deterministic scheduling model and business flow model under the digital twin architecture are proposed.

[0073] The deterministic scheduling model under the digital twin architecture proposed in the embodiment of the present invention includes: the service flow is transmitted from the source end TSN1 domain across the DIP domain to the opposite end TSN2 domain. The transmission path of the service flow is p = {v0,v1,…,v p}, where the node set (v0,v1,…,v a ) is located in the TSN1 domain, (v a+1 ,v a+2 ,…,v b ) is located in the DIP domain, (v b+1 ,v b+2 ,…,v p ) is located in the TSN2 domain. In order to perform time slot mapping and rectification mechanism across domain gateways, the transmission periods of the source TSN1 and the peer TSN2 are defined as T TSN1 and T TSN2 , the transmission period of DIP is T DIP , the mapping relationship is:

[0074] T HP =T TSN1 N TSN1 =T TSN2 N TSN2 =T DIP N DIP

[0075] Among them, N TSN1 、N TSN2 、N DIP It is the number of cycles forwarded by different domain nodes within a super cycle.

[0076] The present invention defines link e=(v i , v i+1 ) spans different domains D1 and D2, then the upstream node v i The traffic flow sent in cycle c is mapped to the downstream node v i+1 The period Φ e (c), that is, the periodic mapping is:

[0077]

[0078] The time slots / cycles of the two mechanisms are abstracted as node queues for unified representation. The service flow assigned to queue 0 will be forwarded immediately, and the service flow assigned to queue k will be forwarded after waiting for k cycles (0<k≤N), where N is the number of queues reserved for the node.

[0079] In the TSN domain, the CQF mechanism is run to synchronize enqueue and dequeue, and two queues are reserved.

[0080] In the DIP domain, due to the coexistence of a large number of mixed service flows in the network, multiple time slots of different flows overlap, resulting in a reduction in the number of schedulable flows. The CSQF mechanism is used for macro scheduling. The periodic offset r is introduced at each node. k ∈{0,1,...,n-1} to schedule the service flow to a specific CSQF queue, where n is the number of reserved queues. m If the node is issued within the c cycle, it will be in the DIP domain v m+1 Node (c+r k )%N DIP The packets are forwarded within the period.

[0081] From the above, it can be seen that business flow scheduling consists of TSN1 domain transmission, TSN1-DIP cross-domain mapping, DIP domain transmission, DIP-TSN2 cross-domain mapping, and TSN2 domain transmission.

[0082] A business flow model under a digital twin architecture proposed in an embodiment of the present invention includes: F = {f1, f2, ..., f F} is a set of business flows, each business flow f sent by the source station i By quintuple definition, They are the transmission starting point and end point of the business flow respectively. is the data size contained in the business flow, is the maximum end-to-end delay of the service flow, The revenue of successful scheduling. According to the characteristics of the transmission business flow in railway operation and maintenance, the business flow is roughly divided into control and execution business flow that is sensitive to delay, monitoring and data acquisition flow that has relatively high bandwidth requirements, and business for data analysis and business optimization, which correspond to the time triggered (TT) flow, audio / video bridging (AVB) flow and BE flow in TSN respectively. The present invention uses the revenue value of successful transmission to distinguish and reflect its importance.

[0083]

[0084] Step S3: Based on the deterministic scheduling model and business flow model under the digital twin architecture, the cross-domain problem of hybrid business flows in the railway operation and maintenance scenario is transformed into a constraint problem.

[0085] 1. Latency constraint:

[0086] Business Flow i The total delay of cross-domain transmission is in,

[0087] are the transmission delays within the three domains, It is the cross-domain transmission delay.

[0088] In the TSN1 domain, due to the running of the CQF mechanism, on node v m The business flow issued by the c cycle will be at node v m+1 The c cycle is received and then at node v m+1 The c+1 cycle is forwarded, so the service flow f i From arriving at v0 to leaving v a The delay is

[0089] When TSN1-DIP is transmitted across domains, the service flow f i From leaving v a To leave v a+1 The delay is

[0090]

[0091] In the DIP domain, due to the operation of the CSQF mechanism, the service flow f i From leaving v a+1 To leave v b The delay is:

[0092]

[0093] When DIP-TSN2 transmits across domains, the service flow f i From leaving vb To leave v b+1 The delay is:

[0094]

[0095] In the TSN2 domain, service flow f i From leaving v b To leave The delay is

[0096] For business flow i The end-to-end delay constraint is

[0097] 2 Queue capacity constraint: The mixed service flow is transmitted along a certain end-to-end link. On the one hand, the service flow can only be transmitted when the queue cache resources allocated to each hop on the allocated path are sufficient. When the allocated resources are smaller than the service flow packet size, the scheduling failure will be declared and converted to timeout for quantification processing, which will be explained in detail in the algorithm design below; on the other hand, the cache capacity of the node queue determines the upper limit of the service flow that can be transmitted. in, is scheduled to node v in the same cycle j business flow, For node v j The cache capacity of a single queue, when node v j ∈TSN domain, m∈{1,2}; when node v j ∈DIP domain, m∈{1,2,3,4}.

[0098] 3 Objective function: An objective function that takes both effect and efficiency into consideration is established, with the successful transmission of time-sensitive flows as the first objective and the auxiliary objective of improving transmission efficiency while obtaining a higher benefit value.

[0099] maxF=α*|P|+β / D

[0100] Where α+β=1, |P| is the number of time-sensitive flows that are successfully scheduled, and D is the total transmission delay.

[0101] Step S4: Based on the above constraint problem, the end-to-end deterministic scheduling process of the service flow is modeled as a Markov decision process.

[0102] The state space consists of the remaining buffer capacity of each network node queue and the service flow transmission requirements:

[0103] The status includes two aspects: the remaining cache capacity of each network node queue and business flow transmission requirements in It can be expressed as:

[0104]

[0105] in, Represents node v i The remaining cache capacity of queue j.

[0106] 2 The action space consists of paths and queues:

[0107] The action space is characterized by a two-dimensional tuple of (path, queue), The agent takes an action vector a t Guide the transmission of business flows. At the beginning of the transmission, extract the transmission start and end points of the business flow and obtain all optional transmission paths of the business flow. This can avoid redundant exploration processes. i ∈PATH i For business flow f i All optional transmission paths, For node v i The queue number of the .

[0108] 3 Consider the reward function of transmission delay ratio:

[0109]

[0110] Step S5: Based on the above Markov decision process, an online hybrid service flow end-to-end transmission scheduling method based on D3QN (Dueling Double Deep Q Network) is proposed to obtain the optimal resource allocation and deployment result. In order to reduce the impact of environmental state fluctuations on the decision-making of the agent. The transmission path allocated to each service flow and the queue number allocated to each node in the path are obtained.

[0111] Initialize D3QN algorithm parameters: Initialize controller parameters, including the experience replay pool D size N, initialize the current Q network parameters θ and target Q network parameters θ′, and the initial exploration rate ε ini , the final exploration rate ε fin , the number of samples taken at a time is m, and the number of synchronization rounds is L.

[0112] The agent conducts action exploration: selects actions based on the current strategy and exploration mechanism Where x∈[0,1) is a random variable generated at each step. The agent performs action a t , get real-time rewards t and the new environment state s t+1 , d t Indicates whether this exploration is accepted.

[0113] Experience replay mechanism: convert the quintuple (st ,a t ,r t ,s t+1 ,d t ) is stored in the playback memory and the system status is updated: t ←s t+1 , if the memory is full, discard the old tuple. Randomly extract m samples from the memory and calculate y j and Loss.

[0114]

[0115]

[0116] Neural network parameter update: Perform gradient descent on the Loss function to update the current Q network parameters, and synchronize the current Q network parameters to the target Q network every L steps.

[0117] Finally, the above process is repeated until the iteration upper limit reaches the set value, and finally the optimal hybrid business resource allocation deployment can be obtained.

[0118] In summary, the embodiment of the present invention provides a hybrid business deterministic transmission scheduling method for digital twins, which realizes all-round high-precision mapping from physical space to virtual space, accurately simulates the railway operation and maintenance environment in the physical world, and observes and analyzes it in the virtual space. Through this mapping, the system can predict the state changes of physical entities and make corresponding transmission scheduling decisions in the virtual world. In this way, the scheduling strategy can be tested and optimized without affecting the actual physical system, thereby improving the accuracy and effectiveness of the decision.

[0119] The present invention provides a hybrid service deterministic transmission scheduling method for digital twins, designs a differentiated scheduling strategy, divides service flows into three categories: monitoring and data acquisition flow, control and execution service flow, and data analysis and service optimization flow, and performs special scheduling according to the characteristics and requirements of each type of service flow. This differentiated scheduling method not only meets the different requirements of different service flows for transmission delay and reliability, but also optimizes the use of network resources, improves transmission efficiency and the overall performance of the system.

[0120] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0121] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0122] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0123] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A hybrid service deterministic transmission scheduling method for digital twins, characterized in that: include: Constructing a network scenario of the network architecture of TSN+DIP, a time-sensitive network for virtual-to-real mapping; Based on the TSN+DIP network architecture of virtual-real mapping, a deterministic scheduling model and business flow model under the digital twin architecture are proposed; Based on the deterministic scheduling model and business flow model under the digital twin architecture, the cross-domain problem of hybrid business flows in the railway operation and maintenance scenario is transformed into a constraint problem; Based on the constraint problem, the end-to-end deterministic scheduling process of the business flow is modeled as a Markov decision process; Based on the above Markov decision process, an online hybrid service flow end-to-end transmission scheduling method based on dueling dual deep Q network D3QN is proposed to obtain the optimal hybrid service resource allocation and deployment result.

2. The method according to claim 1, characterized in that The network scenario of constructing the network architecture of the time-effective network TSN+deterministic internet interconnection protocol DIP for virtual-real mapping includes: Set the physical network to consist of source TSN1, DIP domain and peer TSN2 domain. The switches in the TSN domain run the cyclic queuing and forwarding CQF mechanism. The DIP domain runs the cyclic specific queuing and forwarding CSQF mechanism. Time synchronization is implemented in the TSN domain and frequency synchronization is implemented in the DIP domain. Both CQF and CSQF divide time into multiple equal-length time slots and abstract the entire network into a directed connected graph G = {V, E}, where V is the set of network nodes and E = {e|e = (v i ,v j ), i≠j} is a link set consisting of nodes, node v i ,v j are the source endpoint and end point of link e respectively, and the delay on link e is d e , delay d e Includes processing delay and queuing delay at upstream nodes; The virtual space is mapped from the physical space. By acquiring data from various sensors deployed in the physical space, the data access side, core network and data analysis side that depict the geographic location are constructed to determine the topological structure of the TSN network and the DIP network.

3. The method according to claim 2, characterized in that The TSN+DIP network architecture based on the virtual-to-real mapping proposes a deterministic scheduling model and a business flow model under the digital twin architecture, including: Setting up a deterministic scheduling model under the digital twin architecture includes: the service flow is transmitted from the source TSN1 domain across the DIP domain to the peer TSN2 domain, and the transmission path of the service flow is p = {v0,v1,…,v p }, where the node set (v0,v1,…,v a ) is located in the TSN1 domain, and the set of nodes in the DIP domain in the transmission path is (v a+1 ,v a+2 ,…,v b ), the set of nodes in the TSN2 domain in the transmission path is (v b+1 ,v b+2 ,…,v p ), the transmission periods of the super-periodic HP source TSN1 and the peer TSN2 are defined as T TSN1 and T TSN2 , the transmission period of DIP is T DIP , the mapping relationship is: T HP =T TSN1 N TSN1 =T TSN2 N TSN2 =T DIP N DIP Among them, N TSN1 、N TSN2 、N DIP is the number of cycles forwarded by different domain nodes within a supercycle; In the TSN domain, the CQF mechanism is run to synchronize queue entry and exit, and two queues are reserved. In the DIP domain, the CSQF mechanism is used for macro scheduling. The service flow scheduling consists of TSN1 domain transmission, TSN1-DIP cross-domain mapping, DIP domain transmission, DIP-TSN2 cross-domain mapping, and TSN2 domain transmission. The business flow model under the digital twin architecture includes: F = {f1,f2,...,f F } is a set of business flows, each business flow f sent by the source station i By quintuple definition, They are the transmission starting point and end point of the business flow respectively. is the data size contained in the business flow, is the maximum end-to-end delay of the service flow, For the benefits of successful scheduling; The service flows are roughly divided into control and execution service flows that are sensitive to latency, monitoring and data collection flows that have relatively high bandwidth requirements, and services for data analysis and service optimization, which correspond to the time-triggered TT flow, audio and video bridge AVB flow, and BE flow in TSN respectively. The importance of each service flow is distinguished by the benefit value of successful transmission:

4. The method according to claim 3, characterized in that The deterministic scheduling model and business flow model based on the digital twin architecture transform the cross-domain problem of hybrid business flows in the railway operation and maintenance scenario into a constraint problem, including: Based on the deterministic scheduling model and business flow model under the digital twin architecture, the hybrid business flow cross-domain problem in the railway operation and maintenance scenario is transformed into a constraint problem. The constraint problem includes: 1: Delay constraint: Business Flow i The total delay of cross-domain transmission is are the transmission delays within the three domains, It is the cross-domain transmission delay; In the TSN1 domain, service flow f i From arriving at v0 to leaving v a The delay is When TSN1-DIP is transmitted across domains, the service flow f i From leaving v a To leave v a+1 The delay is: In the DIP domain, service flow f i From leaving v a+1 To leave v b The delay is: When DIP-TSN2 transmits across domains, the service flow f i From leaving v b To leave v b+1 The delay is: In the TSN2 domain, service flow f i From leaving v b To leave The delay is For business flow i The end-to-end delay constraint is 2: Queue capacity constraint: The cache capacity of the node queue determines the upper limit of the business flow that can be transmitted. is scheduled to node v in the same cycle j business flow, For node v j The cache capacity of a single queue, when node v j ∈TSN domain, m∈{1,2}; when node v j ∈DIP domain, m∈{1,2,3,4}; 3: Objective function: maxF = α*|P| + β / D Where α+β=1, |P| is the number of time-sensitive flows that are successfully scheduled, and D is the total transmission delay.

5. The method according to claim 4, characterized in that Based on the constraint problem, the end-to-end deterministic scheduling process of the service flow is modeled as a Markov decision process, including: The state space is composed of the remaining buffer capacity of each network node queue and the service flow transmission requirements. The state space includes: the remaining buffer capacity of each network node queue and business flow transmission requirements in It is expressed as: in, Represents node v i The remaining cache capacity of queue j. The action space consists of paths and queues: The action space is characterized by a two-dimensional tuple (path, queue). The agent takes an action vector a t Guide the transmission of business flows. At the beginning of the transmission, extract the transmission start and end points of the business flow, and obtain all optional transmission paths for the business flow. i ∈PATH i For business flow f i All optional transmission paths, For node v i The queue number of the . The reward function considering the transmission delay ratio is set as:

6. The method according to claim 5, characterized in that Based on the above Markov decision process, an online hybrid service flow end-to-end transmission scheduling method based on D3QN is proposed to obtain the optimal hybrid service resource allocation and deployment result, including: Initialize D3QN algorithm parameters: Initialize controller parameters, including the experience replay pool D size N, initialize the current Q network parameters θ and target Q network parameters θ′, and the initial exploration rate ε ini , the final exploration rate ε fin , the number of samples taken at a time is m, and the number of synchronization rounds is L; The agent conducts action exploration: selects actions based on the current strategy and exploration mechanism Random variables generated for each step; The agent performs action a t , get real-time rewards t and the new environment state s t+1 , d t Indicates whether this exploration is accepted; Experience replay mechanism: convert the quintuple (s t ,a t ,r t ,s t+1 ,d t ) is stored in the playback memory and the system status is updated: t ←s t+1 , if the memory is full, discard the old tuple, randomly extract m samples from the memory, and calculate y j and Loss; Neural network parameter update: Perform gradient descent on the Loss function to update the current Q network parameters, and synchronize the current Q network parameters to the target Q network every L steps. The above process is repeated until the iteration upper limit reaches the set value, and finally the optimal hybrid service resource allocation scheme can be obtained, which includes the transmission path allocated to each service flow and the queue number allocated to each node in the path.

Citation Information

Patent Citations

  • 5G industrial time delay sensitive service resource scheduling method and device, and electronic equipment

    CN115996403A

  • Time-sensitive data stream cross-domain aggregation method, system and device and storage medium

    CN116599909A

  • General calculation fusion network architecture and resource adaptation method

    CN117596122A

  • Display synchronization for time sensitive networking

    US20230262281A1

Cited By

  • Virtual-real cooperative communication scheduling method suitable for digital twin workshop

    CN119906746A

  • A virtual-real collaborative communication scheduling method suitable for a digital twin workshop

    CN119906746B

  • Industrial real-time data transmission guarantee method and system based on TSN

    CN120343043A

  • A TSN-based industrial real-time data transmission guarantee method and system

    CN120343043B

  • Transmission scheduling method of cross-domain deterministic network and related device

    CN121462432A