A Hybrid Service Deterministic Transmission Scheduling Method for Digital Twin

Abstract

The present invention provides a deterministic transmission scheduling method for hybrid services oriented to digital twins. The method includes: constructing a network scenario with a virtual-real mapping TSN+DIP network architecture, and proposing a deterministic scheduling model and a service flow model under the digital twin architecture; transforming the cross-domain problem of hybrid service flows in the constructed railway operation and maintenance scenario into a constraint problem; modeling the end-to-end deterministic scheduling process of service flows as a Markov decision process, and proposing an online hybrid service flow end-to-end transmission scheduling method based on the duel double deep Q network D3QN to obtain an optimal resource allocation and deployment result. The method of the present invention can, in the railway operation and maintenance scenario, for the case of end-to-end transmission of hybrid service flows, consider the different differentiated requirements of service flows, and achieve a lower overall end-to-end delay and a higher transmission gain under the condition of ensuring that the service flows are successfully scheduled.

Description

Technical Field

[0001] The present invention relates to the technical field of railway operation and maintenance, and particularly to a deterministic transmission scheduling method for hybrid services oriented to digital twins. Background Art

[0002] In recent years, technologies such as big data and artificial intelligence have developed vigorously, gradually becoming research hotspots and being applied to railway intelligent operation and maintenance, etc. The data in these scenarios is often relatively precise and requires efficient and reliable transmission. However, traditional "best effort" networks are difficult to meet the requirements of multiple services for low latency, high reliability, and differentiated services. In this context, deterministic scheduling technology has been proposed. The key to deterministic scheduling technology lies in achieving deterministic latency and jitter.

[0003] However, the existing research has focused on the transmission scheduling of physical networks. On the one hand, the algorithm lacks the perception of real-time changes. In fact, the communication environment in the actual network is non-stationary, and the pre-trained scheduling strategy cannot ensure the requirements of Quality of Service (QoS). On the other hand, the service flows in railway operation and maintenance have strict requirements for transmission. Directly applying the transmission scheduling method in the physical world may lead to low efficiency and even cause serious network oscillations. As an efficient solution, digital twin technology, through the mutual mapping between the physical world and the virtual world, can accurately reflect and predict the state of entities in the real space by observing counterparts in the virtual space, and is widely used in the manufacturing industry.

[0004] Digital twin technology also provides a new opportunity for the intelligent development of railway operation and maintenance. By widely deploying Internet of Things (IoT) sensing devices at various levels such as railway stations, the digitization of railway operation and maintenance elements can be realized. Through the data transmission and processing of the IoT, communication network, and bearer network, a digital railway "twin" of the physical railway can be formed. Based on the twin body, reliable decisions are made and after obtaining QoS guarantees and then synchronizing the decisions to the entity, the intelligent structural monitoring and management decision-making can be achieved. Summary of the Invention

[0005] Embodiments of the present invention provide a deterministic transmission scheduling method for hybrid services oriented to digital twins to effectively improve the end-to-end transmission benefit of hybrid services.

[0006] To achieve the above object, the present invention adopts the following technical solutions.

[0007] A deterministic transmission scheduling method for hybrid services oriented to digital twins includes:

[0008] Constructing a network scenario with a network architecture of a time-sensitive network (TSN) + deterministic interconnection protocol (DIP) that maps between the virtual and the real;

[0009] Based on the TSN+DIP network architecture with virtual-real mapping, a deterministic scheduling model and a service flow model under the digital twin architecture are proposed;

[0010] Based on the deterministic scheduling model and the service flow model under the digital twin architecture, the cross-domain problem of hybrid service flows in the railway operation and maintenance scenario is transformed into a constraint problem;

[0011] Based on the constraint problem, the end-to-end deterministic scheduling process of the service flow is modeled as a Markov decision process;

[0012] Based on the above Markov decision process, an online hybrid service flow end-to-end transmission scheduling method based on the duel double deep Q-network D3QN is proposed to obtain the optimal hybrid service resource allocation and deployment results.

[0013] Preferably, the network scenario for constructing the time-sensitive network TSN+deterministic interconnection protocol DIP network architecture with virtual-real mapping includes:

[0014] Set the physical network to consist of a source TSN1, a DIP domain, and a peer TSN2 domain. The switches in the TSN domain run the cyclic queuing and forwarding CQF mechanism, the DIP domain runs the cyclic specific queuing and forwarding CSQF mechanism, time synchronization is achieved in the TSN domain, and frequency synchronization is achieved in the DIP domain;

[0015] Both the CQF and CSQF mechanisms divide time into multiple equal-length time slots, and the entire network is abstracted as a directed connected graph G={V,E}, where V is the set of network nodes, and E={e|e=(v i ,v j ),i≠j} is the set of links composed of nodes. The nodes v i ,v j are the source endpoint and the end endpoint of e respectively, and the delay on the link e is d e , and the delay d e includes the processing delay and queuing delay at the upstream node;

[0016] The virtual space is mapped from the physical space. Data is obtained through various sensors deployed in the physical space to construct a data access side, a core network, and a data analysis side that depict geographical locations, and the topological structures of the TSN network and the DIP network are determined.

[0017] Preferably, the proposal of the deterministic scheduling model and the service flow model under the digital twin architecture based on the TSN+DIP network architecture with virtual-real mapping includes:

[0018] Set the deterministic scheduling model under the digital twin architecture to include: The service flow is transmitted from the source TSN1 domain across the DIP domain to the peer TSN2 domain, and the transmission path of the service flow is p={v 0, v 1 ,..., v p}, where the set of nodes (v 0 , v 1 ,..., v a ) is located within the TSN1 domain, and the set of nodes located within the DIP domain in the transmission path is (v a+1 , v a+2 ,..., v b ). The set of nodes located within the TSN2 domain in the transmission path is (v b+1 , v b+2 ,..., v p ). Define the transmission periods of the source end TSN1 and the peer end TSN2 of the hypercycle HP as T TSN1 and T TSN2 respectively, and the transmission period of the DIP is T DIP . The mapping relationship is:

[0019] T HP = T TSN1 N TSN1 = T TSN2 N TSN2 = T DIP N DIP

[0020] where N TSN1 , N TSN2 , N DIP are the number of forwarding periods of nodes in different domains within a hypercycle;

[0021] Within the TSN domain, the CQF mechanism is run to synchronize enqueueing and dequeueing, and two queues are reserved. Within the DIP domain, the CSQF mechanism is used for macro scheduling, and the service flow scheduling consists of TSN1 domain transmission, TSN1 - DIP cross - domain mapping, DIP domain transmission, DIP - TSN2 cross - domain mapping, and TSN2 domain transmission;

[0022] Set the service flow model under the digital twin architecture to include: F = {f 1 , f 2 ,..., f |F|} as the service flow set. Each service flow f i sent from the source station is defined by the five - tuple . are the transmission start and end points of the service flow respectively, is the data size included in this service flow, is the maximum end - to - end delay of this service flow, is the benefit of successful scheduling;

[0023] The service flow is generally divided into a control and execution service flow sensitive to latency, a monitoring and data collection flow with relatively high bandwidth requirements, and a service for data analysis and service optimization, corresponding to the time-triggered TT flow, audio-video bridging AVB flow, and BE flow in TSN respectively. The importance of each service flow is distinguished by the revenue value of successful transmission:

[0024]

[0025] Preferably, based on the deterministic scheduling model and service flow model under the digital twin architecture, the cross-domain problem of the hybrid service flow in the railway operation and maintenance scenario is transformed into a constraint problem, including:

[0026] Based on the deterministic scheduling model and service flow model under the digital twin architecture, the cross-domain problem of the hybrid service flow in the railway operation and maintenance scenario is transformed into a constraint problem, and the constraint problem includes:

[0027] 1: Latency constraint:

[0028] The service flow f i The total latency of cross-domain transmission is

[0029] They are the transmission latencies within the three domains respectively, is the cross-domain transmission latency;

[0030] In the TSN1 domain, for the service flow f i from arriving at v 0 to leaving v a the latency is

[0031] When performing TSN1-DIP cross-domain transmission, for the service flow f i from leaving v a to leaving v a+1 the latency is:

[0032]

[0033] In the DIP domain, for the service flow f i from leaving v a+1 to leaving v b the latency is:

[0034]

[0035] When performing DIP-TSN2 cross-domain transmission, for the service flow f i from leaving v b to leaving v b+1 the latency is:

[0036]

[0037] In the TSN2 domain, traffic flow f i from leaving v b to leaving the time delay is

[0038] For traffic flow f i the end-to-end delay constraint has

[0039] 2: Queue capacity constraint: The buffer capacity of the node queue determines the upper limit of the traffic flow that can be transmitted. For the traffic flows scheduled to node v in the same cycle j For node v j the buffer capacity of a single queue. When node v j ∈TSN domain, m ∈ {1, 2}; when node v j ∈DIP domain, m ∈ {1, 2, 3, 4};

[0040] 3: Objective function: maxF = α * |P| + β / D

[0041] where α + β = 1, |P| is the number of time-sensitive flows with successful scheduling, and D is the total transmission delay.

[0042] Preferably, based on the above constraint problem, the end-to-end deterministic scheduling process of the traffic flow is modeled as a Markov decision process, including:

[0043] The state space consists of the remaining buffer capacities of each network node queue and the traffic flow transmission requirements, and this state space includes: the remaining buffer capacities of each network node queue and the transmission requirements of the traffic flow where is expressed as:

[0044]

[0045] where represents the remaining buffer capacity of queue j of node v i

[0046] The action space consists of paths and queues: This action space is characterized by a two-dimensional tuple (path, queue). The agent guides the traffic flow transmission with action vector a t At the start of the transmission, the transmission start point and end point of the traffic flow are extracted to obtain all the optional transmission paths of this traffic flow, p i ∈PATH i is all the optional transmission paths of traffic flow f i ​​​ is the queue serial number for node v i .

[0047] Set the reward function considering the transmission delay ratio as:

[0048]

[0049] Preferably, based on the above Markov decision process, an online hybrid traffic flow end-to-end transmission scheduling method based on D3QN is proposed to obtain the optimal hybrid service resource allocation and deployment results, including:

[0050] Initialize the D3QN algorithm parameters: Initialize the parameters of the controller, including the size N of the experience replay pool D, initialize the current Q-network parameters θ and the target Q-network parameters θ′, and the initial exploration rate ε ini , and the final exploration rate ε fin , the number of samples taken per time m, and the synchronization round L;

[0051] The agent conducts action exploration: Select actions according to the current policy and exploration mechanism x ∈ [0, 1) is a random variable generated at each step;

[0052] The agent executes action a t , and obtains the real-time reward r t and the new environmental state s t+1 , d t represents whether this exploration is accepted;

[0053] Experience replay mechanism: Store the five-tuple (s t , a t , r t , s t+1 , d t ) into the replay memory bank and update the system state: s t ← s t+1 , and if the memory bank is full, discard the old tuples, randomly extract m samples from the memory bank, and calculate y j and Loss;

[0054]

[0055]

[0056] Neural network parameter update: Perform gradient descent on the Loss function to update the current Q-network parameters, and synchronize the current Q-network parameters to the target Q-network every L steps.

[0057] Repeat the above process until the iteration upper limit reaches the set value. Finally, an optimal hybrid service resource allocation scheme can be obtained, which includes the transmission path assigned to each service flow and the queue sequence number assigned to each node in the path.

[0058] As can be seen from the technical solutions provided by the embodiments of the present invention above, the present invention provides a hybrid service deterministic transmission scheduling method for digital twins, which is used in the railway operation and maintenance scenario. For the end-to-end transmission of hybrid service flows, considering the different differentiated requirements of different service flows, while ensuring that the service flows are successfully scheduled, it realizes a lower overall end-to-end delay and a higher transmission gain.

[0059] Additional aspects and advantages of the present invention will be given in part in the following description, which will become apparent from the following description, or can be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0061] Figure 1 It is a schematic diagram of the implementation of a hybrid service deterministic transmission scheduling method for digital twins in an embodiment of the present invention;

[0062] Figure 2 It is a processing flow chart of a hybrid service deterministic transmission scheduling method for digital twins in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] The following details the embodiments of the present invention. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present invention, and cannot be construed as a limitation of the present invention.

[0064] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the description of the present invention means the presence of the stated features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when the present invention states that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The phrase "and / or" used herein includes any unit and all combinations of one or more of the associated listed items.

[0065] Those skilled in the art of the present technology can understand that, unless otherwise defined, all terms used herein (including technical terms and scientific terms) have the same meaning as the general understanding of those of ordinary skill in the art to which the present invention pertains. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with their meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless defined as here.

[0066] For the convenience of understanding the embodiments of the present invention, the following will further explain with several specific embodiments in conjunction with the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0067] The embodiment of the present invention provides a schematic implementation diagram of a method for scheduling deterministic transmission of hybrid services for digital twins as Figure 1 shown, which can be used in scenarios such as railway operation and maintenance. The processing flow of this method is as Figure 2 shown and includes the following processing steps:

[0068] Step S1, construct a network scenario of a network architecture of "TSN (Time-Sensitive Networking, time-sensitive network) + DIP (Deterministic Internet Protocol, deterministic Internet protocol) that maps virtual and physical entities.

[0069] The entity network consists of the source TSN1, the DIP domain, and the peer TSN2 domain. Among them, the TSN domain has fewer hops, and each sensor is directly connected to the TSN switch, and the TSN switch runs the cyclic queuing and forwarding (CQF) mechanism. While the DIP domain converges service flows and has more hops, and runs the cyclic specified queuing and forwarding (CSQF) mechanism to achieve multi-queue caching.

[0070] Time synchronization is achieved in the TSN domain, while frequency synchronization is achieved in the DIP domain. Both the CQF and CSQF mechanisms divide time into multiple equal-length time slots. For formal description, the entire network is abstracted as a directed connected graph G = {V, E}, where V is the set of network nodes, and E = {e|e = (v i ,v j ), i≠j} is the set of links composed of nodes. The nodes v i ,v j are the source endpoint and the end endpoint of e respectively, and the delay on the link e is d e , and the delay d e includes the processing delay and queuing delay at the upstream node.

[0071] The virtual space is obtained by high-precision mapping of the physical space. Data is acquired through various sensors deployed in the physical space to construct the data access side, the core network, and the data analysis side that depict the geographical location, determine the topological structures of the TSN network and the DIP network, and extract the performance of sensors, switches, routers, etc. to obtain the model data of the virtual space. At the same time, a deterministic end-to-end transmission strategy based on transmission knowledge and evaluation knowledge is deployed to achieve all-round high-precision synchronization between the virtual and physical spaces. In the virtual space, E2ETSM-OMT is used to perform centralized scheduling of deterministic service flows under limited resources, minimizing the overall delay and maximizing the revenue while ensuring the scheduling success rate, and finally synchronizing the strategy with stable training to the physical space to meet the requirements for the effectiveness and reliability of transmission in actual railway operation and maintenance.

[0072] Step S2: Based on the "TSN + DIP" network architecture with virtual-real mapping, a deterministic scheduling model and a service flow model under the digital twin architecture are proposed.

[0073] A deterministic scheduling model proposed in an embodiment of the present invention includes: The service flow is transmitted from the source TSN1 domain across the DIP domain to the peer TSN2 domain. The transmission path of the service flow is p = {v 0 ,v 1 ,…,v p}, where the node set (v 0, v 1 , …, v a ) is located within the TSN1 domain, (v a+1 , v a+2 , …, v b ) is located within the DIP domain, (v b+1 , v b+2 , …, v p ) is located within the TSN2 domain. In order to execute the time slot mapping and rectification mechanism across the network gateway, the transmission periods of the source - side TSN1 and the peer - side TSN2 of the hyper - period (HP) are defined as T TSN1 and T TSN2 , the transmission period of the DIP is T DIP , and the mapping relationship is:

[0074] T HP = T TSN1 N TSN1 = T TSN2 N TSN2 = T DIP N DIP

[0075] Wherein, N TSN1 , N TSN2 , N DIP are the number of forwarding periods of nodes in different domains within a hyper - period.

[0076] The present invention defines that the link e = (v i , v i+1 ) spans different domains D 1 and D 2 , then the traffic flow sent by the upstream node v i in the period c is mapped to the period Φ i+1 (c) of the downstream node v e , that is, the period mapping is:

[0077]

[0078] Abstract the time slots / periods of the two mechanisms as node queues for unified expression. The traffic flow assigned to queue 0 will be forwarded immediately, and the traffic flow assigned to queue k will be forwarded after waiting for k periods (0 < k ≤ N), where N is the number of queues reserved for this node.

[0079] Within the TSN domain, the CQF mechanism is run to synchronize enqueueing and dequeueing, and two queues are reserved.

[0080] Within the DIP domain, due to the co - existence of a large number of mixed traffic flows converging in the network, the multiple time slots of different flows overlap, resulting in a reduction in the number of schedulable flows. The CSQF mechanism is adopted for macro - scheduling. A period offset r k∈{0, 1,..., n - 1} to schedule the service flow to a specific CSQF queue, where n is the number of reserved queues. When the service flow is sent within the c cycle of the DIP domain v m node, it will be forwarded within the (c + r m+1 ) % N k cycle of the DIP domain v DIP node.

[0081] As described above, it can be seen that the service flow scheduling consists of TSN1 domain transmission, TSN1 - DIP cross - domain mapping, DIP domain transmission, DIP - TSN2 cross - domain mapping, and TSN2 domain transmission.

[0082] A service flow model under a digital twin architecture proposed in an embodiment of the present invention includes: F = {f 1 , f 2 ,..., f F} is the service flow set. Each service flow f i sent from the source station is defined by a five - tuple . are the transmission start point and end point of the service flow respectively, is the data size included in this service flow, is the maximum end - to - end delay of this service flow, is the benefit of successful scheduling. According to the characteristics of the transmission service flow in railway operation and maintenance, the service flow is generally divided into control and execution service flows that are sensitive to delay, monitoring and data acquisition flows with relatively high bandwidth requirements, and services for data analysis and service optimization, corresponding to time - triggered (TT) flows, audio / video bridging (AVB) flows, and BE flows in TSN respectively. The present invention reflects its importance by the benefit value of successful transmission.

[0083]

[0084] Step S3, based on the deterministic scheduling model and service flow model under the digital twin architecture, transform the problem of cross - domain hybrid service flow in the railway operation and maintenance scenario into a constraint problem.

[0085] 1 Delay constraint:

[0086] The total delay of cross - domain transmission of the service flow f i is where,

[0087] are the transmission delays within the three domains respectively, is the cross - domain transmission delay.

[0088] In the TSN1 domain, due to the operation of the CQF mechanism, at node vm The traffic flow sent in the c cycle will be received at node v m+1 in the c cycle and then will be forwarded in the c+1 cycle at node v m+1 Therefore, the latency of traffic flow f i from arriving at v 0 to leaving v a is

[0089] When TSN1-DIP cross-domain transmission occurs, for traffic flow f i from leaving v a to leaving v a+1 the latency is

[0090]

[0091] In the DIP domain, due to the operation of the CSQF mechanism, for traffic flow f i from leaving v a+1 to leaving v b the latency is

[0092]

[0093] When DIP-TSN2 cross-domain transmission occurs, for traffic flow f i from leaving v b to leaving v b+1 the latency is

[0094]

[0095] In the TSN2 domain, for traffic flow f i from leaving v b to leaving the latency is

[0096] For the end-to-end latency constraint of traffic flow f i there are

[0097] 2 queue capacity constraints: The mixed traffic flow is transmitted along the determined end-to-end link. On the one hand, the traffic flow can be transmitted only when the queue buffer resources allocated for each hop on the assigned path are sufficient. When the allocated resources are less than the traffic flow packet size, it will be declared that the scheduling fails and is quantified as a timeout, which will be elaborated in detail in the following algorithm design. On the other hand, the cache capacity of the node queue determines the upper limit of the traffic flow that can be transmitted. Among them, is the traffic flow scheduled to node v in the same cycle j , is the cache capacity of a single queue of node v. When node v j ​j When in the TSN domain, m ∈ {1, 2}; when node v j is in the DIP domain, m ∈ {1, 2, 3, 4}.

[0098] 3 Objective function: An objective function that takes into account both effectiveness and efficiency is established. The first objective is to successfully transmit time-sensitive flows, the secondary objective is to improve transmission efficiency, and at the same time, a relatively high revenue value is obtained.

[0099] maxF = α * |P| + β / D

[0100] where α + β = 1, |P| is the number of time-sensitive flows successfully scheduled, and D is the total transmission delay.

[0101] Step S4. Based on the above constraint problem, the end-to-end deterministic scheduling process of service flows is modeled as a Markov decision process.

[0102] State space composed of the remaining buffer capacities of each network node queue and the transmission requirements of service flows:

[0103] The state includes two aspects: the remaining buffer capacity of each network node queue and the transmission requirements of service flows where can be expressed as:[[]]

[0104]

[0105] where represents the remaining buffer capacity of queue j of node v i .

[0106] 2 Action space composed of paths and queues:[[]]

[0107] The action space is characterized by a two-dimensional tuple (path, queue). The agent uses the action vector a t to guide the transmission of service flows. At the beginning of the transmission, the transmission start point and end point of the service flow are extracted to obtain all the optional transmission paths of the service flow, which can avoid redundant exploration processes. p i ∈ PATH i is all the optional transmission paths of service flow f i . is the queue serial number of node v i .

[0108] 3 Reward function considering the transmission delay ratio:[[]]

[0109]

[0110] Step S5: Based on the above Markov decision process, an online hybrid traffic flow end-to-end transmission scheduling method based on D3QN (Dueling Double Deep Q Network) is proposed to obtain the optimal resource allocation and deployment results, so as to reduce the impact of fluctuations in the environmental state on the decisions made by the agent. The transmission path assigned to each traffic flow and the queue sequence number assigned to each node in the path are obtained.

[0111] Initialize the parameters of the D3QN algorithm: Initialize the parameters of the controller, including the size N of the experience replay pool D, initialize the parameters θ of the current Q network and the parameters θ' of the target Q network, and the initial exploration rate ε ini , the final exploration rate ε fin , the number of samples m taken each time, and the synchronization round L.

[0112] The agent conducts action exploration: Select actions according to the current policy and exploration mechanism where x ∈ [0, 1) is the random variable generated at each step. The agent executes action a t , and obtains the real-time reward r t and the new environmental state s t+1 , d t represents whether this exploration is accepted.

[0113] Experience replay mechanism: Store the five-tuple (s t , a t , r t , s t+1 , d t ) into the replay memory bank, and update the system state: s t ← s t+1 , and discard the old tuple if the memory bank is full. Randomly extract m samples from the memory bank, and calculate y j and Loss.

[0114]

[0115]

[0116] Update the neural network parameters: Perform gradient descent on the Loss function to update the parameters of the current Q network, and synchronize the parameters of the current Q network to the target Q network every L steps.

[0117] Finally, repeat the above process until the iteration upper limit reaches the set value, and finally the optimal hybrid service resource allocation and deployment can be obtained.

[0118] In summary, the embodiment of the present invention provides a method for deterministic transmission scheduling of hybrid services for digital twins, which realizes an all-round high-precision mapping from the physical space to the virtual space, accurately simulates the railway operation and maintenance environment in the physical world, and conducts observations and analyses in the virtual space. Through this mapping, the system can predict the state changes of physical entities and make corresponding transmission scheduling decisions in the virtual world. In this way, the scheduling strategy can be tested and optimized without affecting the actual physical system, thereby improving the accuracy and effectiveness of decisions.

[0119] The present invention provides a method for deterministic transmission scheduling of hybrid services for digital twins, designs a differentiated scheduling strategy, divides the service flows into three categories: monitoring and data collection flows, control and execution service flows, and data analysis and service optimization flows, and conducts specialized scheduling for the characteristics and requirements of each type of service flow. This differentiated scheduling method not only meets the different requirements of different service flows for transmission delay and reliability, but also optimizes the use of network resources, improves the transmission efficiency and the overall performance of the system.

[0120] Those of ordinary skill in the art can understand that the drawings are only schematic diagrams of an embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present invention.

[0121] From the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes a contribution to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

[0122] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, they are described relatively simply, and the relevant parts can refer to the partial descriptions of the method embodiments. The device and system embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0123] As described above, it is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A hybrid service deterministic transmission scheduling method for digital twins, characterized in that: include: Constructing a network scenario of the network architecture of the time-sensitive network TSN + deterministic internetworking protocol DIP for virtual-real mapping; Based on the TSN+DIP network architecture of virtual-real mapping, a deterministic scheduling model and a business flow model under the digital twin architecture are established; Based on the deterministic scheduling model and business flow model under the digital twin architecture, the cross-domain problem of hybrid business flows in the railway operation and maintenance scenario is transformed into a constraint problem; Based on the constraint problem, the end-to-end deterministic scheduling process of the business flow is modeled as a Markov decision process; Based on the above Markov decision process, an online hybrid service flow end-to-end transmission scheduling method based on dueling dual deep Q network D3QN is adopted to obtain the optimal hybrid service resource allocation and deployment result; Based on the constraint problem, the end-to-end deterministic scheduling process of the service flow is modeled as a Markov decision process, including: The state space is composed of the remaining buffer capacity of each network node queue and the service flow transmission requirements. The state space includes: the remaining buffer capacity of each network node queue and business flow transmission requirements in It is expressed as: in, Represents node v i The remaining cache capacity of queue j; The action space consists of paths and queues: The action space is characterized by a two-dimensional tuple (path, queue). The agent takes an action vector a t Guide the business flow transmission. At the beginning of the transmission, extract the transmission start and end point of the business flow, and obtain all optional transmission paths of the business flow. i ∈PATH i For business flow f i All optional transmission paths, For node v i The queue number of x∈[0,1) is the random variable generated at each step; The reward function considering the transmission delay ratio is set as: is the maximum end-to-end delay of the service flow, For the benefits of successful scheduling, For business flow f i The total latency of cross-domain transmission; The service flows are divided into control and execution service flows that are sensitive to latency, monitoring and data collection flows that have relatively high bandwidth requirements, and services for data analysis and service optimization, which correspond to the time-triggered TT flow, audio and video bridge AVB flow, and BE flow in TSN respectively. The importance of each service flow is distinguished by the benefit value of successful transmission:

2. The method according to claim 1, characterized in that: The network scenario of constructing the network architecture of the time-sensitive network TSN+deterministic internetworking protocol DIP for virtual-real mapping includes: Set the physical network to consist of source TSN1, DIP domain and peer TSN2 domain. The switches in the TSN domain run the cyclic queuing and forwarding CQF mechanism. The DIP domain runs the cyclic specific queuing and forwarding CSQF mechanism. Time synchronization is implemented in the TSN domain and frequency synchronization is implemented in the DIP domain. Both CQF and CSQF divide time into multiple equal-length time slots and abstract the entire network into a directed connected graph G = {V, E}, where V is the set of network nodes and E = {e|e = (v i ,v j ), i≠j} is a link set consisting of nodes, node v i ,v j are the source endpoint and end point of link e respectively, and the delay on link e is d e , delay d e Includes processing delay and queuing delay at upstream nodes; The virtual space is mapped from the physical space. By acquiring data from various sensors deployed in the physical space, the data access side, core network and data analysis side that depict the geographic location are constructed to determine the topological structure of the TSN network and the DIP network.

3. The method according to claim 2, characterized in that The TSN+DIP network architecture based on the virtual-to-real mapping establishes a deterministic scheduling model and a business flow model under the digital twin architecture, including: Setting up a deterministic scheduling model under the digital twin architecture includes: the service flow is transmitted from the source TSN1 domain across the DIP domain to the peer TSN2 domain, and the transmission path of the service flow is p = {v0,v1,…,v p }, where the node set (v0,v1,…,v a ) is located in the TSN1 domain, and the set of nodes in the DIP domain in the transmission path is (v a+1 ,v a+2 ,…,v b ), the set of nodes in the TSN2 domain in the transmission path is (v b+1 ,v b+2 ,…,v p ), the transmission periods of the super-periodic HP source TSN1 and the peer TSN2 are defined as T TSN1 and T TSN2 , the transmission period of DIP is T DIP , the mapping relationship is: T HP =T TSN1 N TSN1 =T TSN2 N TSN2 =T DIP N DIP Among them, N TSN1 、N TSN2 、N DIP is the number of cycles forwarded by different domain nodes within a supercycle; In the TSN domain, the CQF mechanism is run to synchronize queue entry and exit, and two queues are reserved. In the DIP domain, the CSQF mechanism is used for macro scheduling. The service flow scheduling consists of TSN1 domain transmission, TSN1-DIP cross-domain mapping, DIP domain transmission, DIP-TSN2 cross-domain mapping, and TSN2 domain transmission. The business flow model under the digital twin architecture includes: F = {f1,f2,...,f F } is a set of business flows, each business flow f sent by the source station i By quintuple definition, They are the transmission starting point and end point of the business flow respectively. The data size contained in the business flow.

4. The method according to claim 3, characterized in that: The deterministic scheduling model and business flow model based on the digital twin architecture transform the cross-domain problem of hybrid business flows in the railway operation and maintenance scenario into a constraint problem, including: Based on the deterministic scheduling model and business flow model under the digital twin architecture, the hybrid business flow cross-domain problem in the railway operation and maintenance scenario is transformed into a constraint problem. The constraint problem includes: 1: Delay constraint: Business Flow i The total delay of cross-domain transmission is are the transmission delays within the three domains, It is the cross-domain transmission delay; In the TSN1 domain, service flow f i From arriving at v0 to leaving v a The delay is When TSN1-DIP is transmitted across domains, the service flow f i From leaving v a To leave v a+1 The delay is: In the DIP domain, service flow f i From leaving v a+1 To leave v b The delay is: When DIP-TSN2 transmits across domains, the service flow f i From leaving v b To leave v b+1 The delay is: In the TSN2 domain, service flow f i From leaving v b To leave The delay is For business flow i The end-to-end delay constraint is 2: Queue capacity constraint: The cache capacity of the node queue determines the upper limit of the business flow that can be transmitted. is scheduled to node v in the same cycle j business flow, For node v j The cache capacity of a single queue, when node v j ∈TSN domain, m∈{1,2}; when node v j ∈DIP domain, m∈{1,2,3,4}; 3: Objective function: maxF = α*|P| + β / D Where α+β=1, |P| is the number of time-sensitive flows that are successfully scheduled, and D is the total transmission delay.

5. The method according to claim 1, characterized in that Based on the above Markov decision process, an online hybrid service flow end-to-end transmission scheduling method based on D3QN is proposed to obtain the optimal hybrid service resource allocation and deployment result, including: Initialize D3QN algorithm parameters: Initialize controller parameters, including the experience replay pool D size N, initialize the current Q network parameters θ and target Q network parameters θ′, and the initial exploration rate ε ini , the final exploration rate ε fin , the number of samples taken at a time is m, and the number of synchronization rounds is L; The agent conducts action exploration: selects actions based on the current strategy and exploration mechanism The agent performs action a t , get real-time rewards t and the new environment state s t+1 , d t Indicates whether this exploration is accepted; Experience replay mechanism: convert the quintuple (s t ,a t ,r t ,s t+1 ,d t ) is stored in the playback memory and the system status is updated: t ←s t+1 , if the memory is full, discard the old tuple, randomly extract M samples from the memory, and calculate y j and Loss; Neural network parameter update: Update the current Q network parameters by gradient descent on the Loss function, and synchronize the current Q network parameters to the target Q network every L steps; The above process is repeated until the iteration upper limit reaches the set value, and finally the optimal hybrid service resource allocation scheme can be obtained, which includes the transmission path allocated to each service flow and the queue number allocated to each node in the path.

Images