Information deterministic scheduling method and device

Abstract

The invention relates to an information deterministic scheduling method and device, and specifically, the method comprises the steps: building a clustering domain based on each aircraft and industrial equipment in a coverage range of the aircraft; in the current time step, each aircraft updates parameters of a local decision reasoning model based on the shared sample trajectory tuple; the sample track tuple comprises a first sample state (including deterministic index information before sample business flow scheduling), a sample scheduling decision, a sample reward value and a second sample state (including deterministic index information after scheduling); and each aircraft determines a first state of the service flow sent by the cluster domain, reasones a scheduling decision for the first state based on the updated model and schedules the service flow to obtain a reward value and a second state. And if the second state meets the deterministic scheduling condition, generating and sharing a new sample track tuple according to the first state, the scheduling decision, the reward value and the second state, and entering a new current time step to continue iteration to obtain an inference model for realizing information deterministic scheduling.

Description

Technical Field

[0001] This application relates to the field of industrial network technology, specifically to an information deterministic scheduling method and apparatus. Background Technology

[0002] With the development of the aircraft industry, aircraft are increasingly involved in communication scenarios, and aircraft (such as electric vertical take-off and landing aircraft) play a very important role in network communication.

[0003] In traditional communication scenarios not involving aircraft, time-sensitive networks (TSNs) are primarily based on Ethernet, providing deterministic guarantees for hop-by-hop forwarding of time-sensitive service flows. However, for wireless networks, deterministic scheduling guarantees on multi-hop links are typically not achievable.

[0004] Communication scenarios involving aircraft often involve wireless communication and complex network topologies. Furthermore, aircraft are highly dynamic, and wireless communication has a wide range; data sent to aircraft during communication may need to traverse multiple wireless links. Therefore, achieving deterministic scheduling of service flow information in communication scenarios involving aircraft is a pressing challenge for the industry. Summary of the Invention

[0005] Based on this, it is necessary to provide an information deterministic scheduling method, apparatus, aircraft, computer-readable storage medium, and computer program product that can achieve deterministic scheduling of service flow information in communication scenarios involving aircraft, in order to address the above-mentioned technical problems.

[0006] Firstly, this application provides an information deterministic scheduling method, which includes:

[0007] For each aircraft in the scheduling system, a sub-cluster domain is constructed, with the aircraft as the cluster head node and wireless industrial equipment within the aircraft's signal coverage area as member nodes;

[0008] In the current time step, each aircraft iteratively updates the parameters of its locally deployed decision reasoning model based on the shared sample trajectory tuples. Each sample trajectory tuple includes a first sample state, a sample scheduling decision, a sample reward value, and a second sample state. The first sample state includes deterministic indicator information before the sample business flow information is scheduled, and the second sample state includes deterministic indicator information after the scheduling.

[0009] Each aircraft determines the first state corresponding to the service flow information sent by the wireless industrial equipment in its cluster at the current time step, infers the first scheduling decision corresponding to the first state based on the locally updated decision reasoning model, schedules the service flow information according to the first scheduling decision, and obtains the first reward value and the second state after scheduling.

[0010] If the second state satisfies the deterministic scheduling condition, a new sample trajectory tuple is generated and shared based on the first state, the first scheduling decision, the first reward value, and the second state. The iteration continues in a new current time step until the iteration stops, thus obtaining the target reasoning model for achieving information deterministic scheduling.

[0011] In one embodiment, each aircraft shares its generated sample trajectory tuple with other aircraft in the scheduling system; each aircraft stores its generated sample trajectory tuple and the shared sample trajectory tuple in a local experience pool; the method further includes:

[0012] In the current time step, if the first number of sample trajectory tuples in the local experience pool is greater than or equal to the preset sample number threshold, each aircraft performs an iterative update of the parameters of the locally deployed decision reasoning model based on the mutually shared sample trajectory tuples, and then performs subsequent steps.

[0013] If the first quantity is less than the sample quantity threshold, each aircraft determines the first state, and infers the second scheduling decision corresponding to the first state based on the initial decision reasoning model. The business flow information is scheduled according to the second scheduling decision to obtain the second reward value and the third state after scheduling.

[0014] If the third state satisfies the deterministic scheduling conditions, based on the first state, the second scheduling decision, the second reward value, and the third state, a new sample trajectory tuple is generated and shared. The newly generated sample trajectory tuple and the newly shared sample trajectory tuple at the current time step are added to the local experience pool, and the process enters a new iteration at the current time step.

[0015] In one embodiment, the deterministic metric information includes end-to-end deterministic characterization information and link capacity utilization; the deterministic characterization information includes at least one of end-to-end latency, jitter, or packet loss rate; the link capacity utilization rate is used to characterize the resource utilization of the link within each time slot; and / or, both the first sample state and the second sample state also include basic attribute information of the sample service flow information; the basic attribute information includes at least one of service flow period, source address, destination address, service flow size, or number of packets sent; and / or, the sample reward value is determined based on the difference in the number of successfully scheduled service flow information in adjacent time steps.

[0016] In one embodiment, in each iteration, each aircraft updates the parameters of the decision reasoning model once based on the optimization objective; the optimization objective is to maximize the amount of service flow information that the network can accommodate; the deterministic scheduling conditions include at least one of transmission capacity constraints, wireless link transmission delay constraints, or end-to-end deterministic demand constraints.

[0017] In one embodiment, in the scheduling system, the sending end and the receiving end communicate with their respective direct communication devices via a wired network, and each aircraft communicates with the direct communication device and with different aircraft via a wireless network.

[0018] The optimization objective and deterministic scheduling conditions are determined based on the queue traffic shaping model; the queue traffic shaping model includes the circular queue shaping mechanism for wired networks and the time slot orchestration rules for wireless networks;

[0019] Among them, each device communicating through the wired network maintains a pair of ping-pong queues on its sending interface. In each time slot, only one queue sends service flow information to the downstream node, and the other queue is used to receive service flow information forwarded by the upstream node. The circular queue shaping mechanism requires that the service flow information sent by the upstream node in a time slot must be completely received by the downstream node in the same time slot. When the next time slot arrives, the downstream node continues to forward the service flow information.

[0020] Time-sensitive network translators are embedded on the transmitting interfaces of various devices communicating via wireless networks. The time slot scheduling rules require that the service flow information that arrives in advance of the scheduled time slot be cached by the time-sensitive network translator and delayed until the correct pre-scheduled time slot for transmission.

[0021] In one embodiment, each aircraft performs an iterative update of the parameters of its locally deployed decision reasoning model based on a shared set of sample trajectory tuples, including:

[0022] Each aircraft samples a sample trajectory tuple from the shared sample trajectory tuple. For each sampled sample trajectory tuple, a probability ratio of a first probability to a second probability is determined. The first probability is the probability of inferring a first sample scheduling decision based on the first sample state in the sample trajectory tuple using the current parameters of the decision reasoning model. The second probability is the probability of inferring a sample scheduling decision based on the first sample state using the parameters of the last update of the decision reasoning model.

[0023] The advantage value is calculated based on the sample reward value in the sample trajectory tuple, and the first state evaluation value corresponding to the first sample state and the second sample state, respectively.

[0024] The first loss value is determined based on the probability ratio and the advantage value;

[0025] Based on the first loss value corresponding to each sample trajectory tuple, the parameters of the locally deployed decision reasoning model are updated and adjusted.

[0026] In one embodiment, the first state evaluation value is predicted by the evaluation neural network in the previous iteration; in each iteration, the method further includes:

[0027] Based on the first state evaluation value and advantage value corresponding to the first sample state, determine the target state evaluation value corresponding to the first sample state;

[0028] Based on the current evaluation neural network in this iteration, predict the second state evaluation value of the first sample state;

[0029] The second loss value is determined based on the difference between the target state evaluation value and the second state evaluation value.

[0030] Based on the second loss value corresponding to each sample trajectory tuple, the parameters of the current evaluation neural network are updated iteratively.

[0031] In one embodiment, the method further includes:

[0032] After receiving the target service flow information sent by the sender, the target aircraft obtains the target status corresponding to the target service flow information; the target aircraft can be any one of multiple aircraft.

[0033] The target aircraft inputs the target status into the locally deployed target inference model and outputs the target scheduling decision corresponding to the target service flow information; the target scheduling decision includes the target scheduling time slot and the target scheduling path;

[0034] During the target scheduling time slot, the target aircraft forwards the target service flow information to the designated receiving end according to the target scheduling path.

[0035] In one embodiment, the sending end is a first user equipment, and the designated receiving end is a second user equipment communicating with a base station; the first user equipment communicates directly with wireless industrial equipment within the cluster domain where the aircraft is located via a wired network; the wireless industrial equipment forwards the target service flow information sent by the first user equipment to the target aircraft via a wireless network; the target service flow information is forwarded by the target aircraft to the second user equipment via the base station; or, the sending end is the second user equipment, and the designated receiving end is the first user equipment; the second user equipment communicates directly with the corresponding base station via a wired network to send the target service flow information to the base station, and the base station forwards the target service flow information to the target aircraft via a wireless network; the target service flow information is forwarded by the target aircraft to the second user equipment via the wireless industrial equipment within its cluster domain.

[0036] Secondly, this application provides an information deterministic scheduling device, which is installed in each aircraft in the scheduling system, comprising:

[0037] The clustering domain construction module is used to construct a clustering domain with the aircraft as the cluster head node and wireless industrial equipment within the signal coverage area of ​​the aircraft as member nodes.

[0038] The model parameter update module is used to iteratively update the parameters of the locally deployed decision reasoning model at the current time step based on the sample trajectory tuples shared by each aircraft. Each sample trajectory tuple includes a first sample state, a sample scheduling decision, a sample reward value, and a second sample state. The first sample state includes deterministic indicator information before the scheduling of sample business flow information. The second sample state includes deterministic indicator information after scheduling.

[0039] The information scheduling module is used to determine the first state corresponding to the service flow information sent by the wireless industrial equipment in the current time step in the cluster domain, infer the first scheduling decision corresponding to the first state based on the locally updated decision reasoning model, schedule the service flow information according to the first scheduling decision, and obtain the first reward value and the second state after scheduling.

[0040] The deterministic scheduling constraint module is used to generate and share new sample trajectory tuples based on the first state, the first scheduling decision, the first reward value, and the second state if the second state satisfies the deterministic scheduling conditions, and then enter a new current time step to continue iterating until the iteration stop condition is reached, thus obtaining the target reasoning model for achieving information deterministic scheduling.

[0041] Thirdly, this application also provides an electronic device. This electronic device can be an aircraft, and it includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the embodiments of this application.

[0042] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps in the embodiments of this application.

[0043] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps in the embodiments of this application.

[0044] In the aforementioned deterministic information scheduling methods, devices, electronic equipment, computer-readable storage media, and computer program products, each aircraft and the industrial equipment within its coverage area construct the same cluster domain, which is managed by the aircraft. Each aircraft locally deploys a decision-making reasoning model to be trained. To enable the decision-making reasoning model to achieve deterministic information scheduling, a Markov decision process is used, where deterministic index information related to deterministic scheduling at each time step is taken as the state, scheduling decisions are taken as actions, and deterministic scheduling conditions that act as constraints are combined. Multiple aircraft iteratively and collaboratively train the decision-making reasoning model. Specifically, each aircraft schedules the service flow information sent within its cluster domain at each iteration time step, generates sample trajectory tuples based on the state changes before and after scheduling and the reward value, and shares them with other aircraft. Thus, at each current time step, each aircraft performs a parameter iteration update training of its local decision-making reasoning model based on the shared sample trajectory tuples, thereby achieving collaborative training. Furthermore, based on the updated model, the aircraft makes and executes scheduling decisions based on the state of the service flow information sent within its cluster domain at the current time step, obtaining a new reward value and the state after scheduling. To constrain the decision-making reasoning model to optimize towards enhanced deterministic scheduling capabilities, the state of the business flow information after scheduling at the current time step is compared with the deterministic scheduling conditions. If the conditions are met, new sample trajectory tuples are generated and shared, and the process continues iterating into a new time step. In this way, a target reasoning model capable of achieving deterministic information scheduling can be iteratively trained step by step, and deterministic information scheduling can be realized through this target reasoning model. Attached Figure Description

[0045] Figure 1 This is a schematic diagram illustrating an application scenario of the information deterministic scheduling method in one embodiment;

[0046] Figure 2 This is a flowchart illustrating an information deterministic scheduling method in one embodiment;

[0047] Figure 3 This is a block diagram of an information deterministic scheduling device in one embodiment;

[0048] Figure 4 This is a block diagram of the information scheduling module in one embodiment;

[0049] Figure 5 This is a block diagram of the internal structure of an electronic device in one embodiment;

[0050] Figure 6 This is a block diagram of a computer-readable storage medium in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] Figure 1 These are schematic diagrams illustrating application scenarios in some embodiments of this application. For example... Figure 1 As shown, the various devices form a scheduling system. Within this system, each aircraft (Aircraft 1 to Aircraft 3) acts as a cluster head node, maintaining its respective cluster domain. Industrial equipment (also known as wireless industrial equipment) within the signal coverage area of ​​each aircraft are member nodes within its respective cluster domain. Communication between aircraft, and between each aircraft and the base station and industrial equipment within its cluster domain, is conducted via a wireless network. Communication between the user equipment on the industrial equipment side (which can be referred to as the first user equipment, not shown in the figure) and the industrial equipment, and between the user equipment on the base station side (i.e.,...) and the industrial equipment... Figure 1 The wired equipment (also known as the second user equipment) and the base station communicate via a wired network. This scheduling system enables end-to-end (from the first user equipment to the second user equipment or from the second user equipment to the first user equipment) deterministic scheduling of time-sensitive service flow information (i.e., related service information that requires strict timeliness, determinism, and reliability of transmission and is transmitted in a streaming manner; for ease of description, it will be referred to as "service flow information" below)

[0053] Taking the deterministic scheduling process of service flow information from the first user equipment to the second user equipment as an example, if the first user equipment communicates with industrial equipment through a wired network, causing the industrial equipment to generate service flow information, the wireless nodes deployed in its respective cluster domain forward the service flow information generated by the industrial equipment to the aircraft managing the cluster domain through a wireless network (such as WiFi). Then, the aircraft directly or indirectly (such as through other aircraft) forwards the service flow information to the second user equipment through a base station (such as a 5G base station).

[0054] To achieve deterministic scheduling, each aircraft is equipped with a decision reasoning model to be trained and a multi-domain collaborative scheduling algorithm based on deep reinforcement learning. Each aircraft completes the collaborative training of its own deployed decision reasoning model through an experience pool sharing mechanism, so that each aircraft trains a target reasoning model that is suitable for deterministic scheduling of information within the managed cluster domain.

[0055] To facilitate understanding, specific numerical examples of the relevant parameters are provided (without limitation). For instance, industrial equipment in each cluster sends 1000 time-sensitive service streams. The service stream data frame size is between 50B and 1KB, and the service stream period includes 4ms, 8ms, and 16ms. The network transmission time slot size (which can be simply referred to as time slot size or time slot length) is set to the greatest common divisor of the above service stream periods. The wired network bandwidth is 1000Mbps, the 5G network frequency band is 3.7–3.8GHz, the frequency bandwidth is 40MHz, the aircraft transmit power is 40dBm, and the path loss is PL1(dB) = 32.4 + 17.3*lg(l1) + 20*lg(fc1). l1 is the distance from the aircraft to the base station, fc1 is the frequency of the 5G network, and the noise variance of the AWGN channel is -104dBm.

[0056] The WiFi network accessed by the industrial equipment uses the 5GHz band with a bandwidth of 40MHz. The transmit power of the industrial equipment is 20dBm, and the path loss is PL2(dB)=40.05+20lg(fc2 / 2.4)+20lg(min(l2,10))+(l2>10)*35lg(l2 / 10), where fc2 is the frequency of the WiFi network, l2 is the distance from the industrial equipment to the aircraft, and the noise power is -110dBm.

[0057] During the scheduling process, the Adam optimizer is used to update the model with a learning rate of 1e-4, an experience pool size of 2048, and a sampling batch of 64 samples each, with a discount factor of 0.95 for all samples. This embodiment does not specifically limit the type and number of aircraft or the type of wireless network included in the multi-domain cooperative deterministic network scheduling scenario. For example, the aircraft in this embodiment can be an electric vertical takeoff and landing (EVTOL) aircraft, that is, an aircraft that uses electricity as its power source and has vertical takeoff and landing capabilities.

[0058] like Figure 2 As shown, in one embodiment, an information deterministic scheduling method is provided, which is executed by an aircraft in a scheduling system, and specifically includes the following steps:

[0059] S21, for each aircraft in the scheduling system, a sub-cluster domain is constructed with the aircraft as the cluster head node and the wireless industrial equipment within the signal coverage range of the aircraft as member nodes.

[0060] Specifically, the scheduling system contains multiple aircraft. Each aircraft periodically sends probe signals to the wireless industrial equipment within its signal coverage area to obtain information on the number, distribution location, and distance of the wireless industrial equipment within its signal coverage area. This establishes a cluster domain (also known as a cluster network) with each aircraft as the cluster head node and the wireless industrial equipment within its signal coverage area as member nodes, thereby improving the manageability of the aircraft for the wireless industrial equipment within its respective cluster domain and the scalability of the connection scale.

[0061] Multiple clusters establish wireless communication connections via cluster head aircraft. Clock synchronization occurs between the multiple clusters. Specifically, after a connection is established, one cluster head node is designated as the master clock, and the cluster head nodes of the remaining clusters become slave clocks. The master clock sends clock synchronization signals to the wireless industrial equipment within its managed cluster and to the slave clock cluster head nodes. The slave clocks then further distribute the received clock synchronization signals to their respective clusters to achieve network-wide clock synchronization. When the wireless communication connection between cluster head nodes is broken, the disconnected cluster head node becomes an independent master clock managing its respective cluster.

[0062] S22, in the current time step, each spacecraft iteratively updates the parameters of its locally deployed decision reasoning model based on the shared sample trajectory tuples.

[0063] The sample trajectory tuples are generated from historical scheduling processes. The service flow information scheduled in the historical scheduling is the sample service flow information. In this embodiment, a Markov decision process is used to train the decision reasoning model through reinforcement learning. Therefore, each sample trajectory tuple includes a first sample state, a sample scheduling decision, a sample reward value, and a second sample state. The first and second sample states represent the states of the sample service flow information before and after scheduling.

[0064] The first sample state / second sample state includes deterministic indicator information before and after the scheduling of sample business flow information. Deterministic indicator information refers to information about indicators related to deterministic scheduling. Some deterministic indicator information may change before and after the scheduling of business flow information.

[0065] It should be understood that the first sample state and the second sample state include the same state items, but the specific values ​​of some state items differ. State items can be understood as various information items or reference indicators useful for making future decisions. For example, in addition to deterministic indicator information, state items may also include basic attribute information of the scheduled business flow.

[0066] Sample scheduling decisions are inferred from sample service flow information in historical scheduling. A scheduling decision is a specific resource allocation strategy when scheduling service flow information. Scheduling decisions include scheduling time slots and scheduling paths. Scheduling decisions are equivalent to actions in Markov decision-making. The action space A includes the scheduling decision inferred for each service flow, including the scheduling time slot, scheduling path (or forwarding path), and frequency bandwidth allocated to that service flow. For example, the action space A can be expressed as: A = {slot} i (t),B i (t),path i (t)};where, slot i (t) is the scheduling time slot for the i-th service flow information in time step t, with a frequency bandwidth of B. i (t) is the frequency bandwidth allocated for the transmission of the i-th service flow information in time step t (or the t-th time step), and the scheduling path is path. i (t) is the path used to schedule and forward the i-th service flow information at time step t. The scheduling path can include intra-domain paths and / or cross-domain paths. It should be understood that i and t are generic terms.

[0067] The sample reward value refers to the reward value obtained after scheduling sample service flow information in historical scheduling. In principle, the more service flow information successfully scheduled at each time step, the greater the reward. Therefore, in actual calculations, the reward value is positively correlated with the difference in the number of successfully scheduled service flow information in adjacent time steps. For example, the formula for obtaining the reward value is as follows:

[0068]

[0069] Among them, |F sch (t) represents the number of service flow information successfully scheduled at time step t, |F sch (t-1)| represents the number of business flow information successfully scheduled at time step t, β∈[0,1] is the discount factor, F is the set of business flow information that needs to be scheduled within the entire business flow scheduling cycle, and |F| represents the total number of business flow information that needs to be scheduled within the entire business flow scheduling cycle.

[0070] It should be understood that each aircraft has a locally deployed decision-making inference model to be trained. Each aircraft, based on a Markov decision process, iteratively generates sample trajectory tuples at multiple time steps, and shares these tuples with other aircraft in the scheduling system via a wireless network. This allows each aircraft to use the shared sample trajectory tuples to iteratively update the parameters of its locally deployed decision-making inference model, thus achieving collaborative training across multiple aircraft. For example, each aircraft can store its generated sample trajectory tuples and the shared sample trajectory tuples in a local experience pool. During each iteration of model parameter update training, a predetermined number of sample trajectory tuples can be sampled from the local experience pool, and the model parameter update training is performed based on these sample trajectory tuples.

[0071] Specifically, at each time step t, wireless industrial devices in each cluster domain send service flow information to the aircraft. The aircraft, acting as an intelligent agent, observes the state s(t) of its cluster domain—the first state before scheduling. Based on the current parameters (equivalent to the current policy) of its locally deployed decision-making inference model, and combined with this state s(t), the aircraft infers the corresponding scheduling decision a(t) for the service flow information. Based on this scheduling decision (also known as the resource scheduling policy), the aircraft schedules the service flow information. That is, each aircraft executes the scheduling decision through interaction with its cluster domain network environment, obtains the corresponding reward value r(t), and transitions to the next state s(t+1)—the second state after scheduling. This completes one scheduling cycle. The aircraft can also determine whether state s(t+1) meets preset deterministic scheduling conditions and add a corresponding scheduling identifier based on the determination result. The deterministic scheduling conditions are constraints that must be met to ensure or guarantee the deterministic scheduling of information. Considering these deterministic scheduling conditions as training constraints during iterative processing can better train the decision-making inference model to possess deterministic scheduling capabilities.

[0072] For example, if the deterministic scheduling conditions are met, a scheduling completion flag (done=0) is added, and the process proceeds to the next time step to continue the current iteration. If the conditions are not met, a scheduling failure flag (done=1) is added, the network environment is reinitialized (e.g., clearing network resource usage in each time slot and clearing the recorded cumulative number of scheduled service flows), and the current iteration ends, entering a new iteration. After each iteration, the aircraft will use the trajectory tuple generated in that step...<s(t),a(t),s(t+1),done> Store it in the local experience pool and share it with other aircraft.

[0073] In some embodiments, when the number of samples in the local experience pool is less than a sample number threshold, each aircraft can perform only the process of generating sample trajectory tuples in each time step without updating the parameters of the decision reasoning model. Specifically, in each current time step, if the first number of sample trajectory tuples in the local experience pool is less than the sample number threshold, each aircraft determines the first state corresponding to the service flow information sent by the wireless industrial equipment in its cluster at the current time step, infers the second scheduling decision corresponding to the first state based on the initial decision reasoning model, schedules the service flow information according to the second scheduling decision, and obtains the second reward value and the third state after scheduling. If the third state satisfies the deterministic scheduling condition, a new sample trajectory tuple is generated and shared according to the first state, the second scheduling decision, the second reward value, and the third state. The newly generated sample trajectory tuple and the newly shared sample trajectory tuple in the current time step are added to the local experience pool, and the process enters a new current time step to continue iterative processing.

[0074] If the number of samples in the local experience pool is greater than or equal to the preset sample number threshold (equivalent to a sufficient number of samples), the aircraft can first iteratively update the parameters of the locally deployed decision reasoning model based on the sample trajectory tuple in each subsequent time step, and then use the decision reasoning model with updated parameters to schedule the business flow information to be scheduled in that time step to generate a new sample trajectory tuple (i.e., execute steps S23-S24).

[0075] For example, in each iteration, each aircraft updates the parameters of the decision reasoning model once based on the optimization objective; the optimization objective is to maximize the amount of service flow information that the network can accommodate |F sch (That is, maximizing the amount of deterministically successful service flow information). In this way, the decision-making and reasoning model in the aircraft can be gradually trained to balance the network resources within its cluster domain, thereby ensuring the end-to-end deterministic transmission of service flow information as much as possible, deterministically scheduling more service flow information, and improving the success rate of service flow information scheduling and network resource utilization.

[0076] In other embodiments, the aircraft may iteratively update the model parameters at each time step, without being limited to waiting until the number of samples in the local experience pool is greater than or equal to a preset sample number threshold before iteratively updating the model parameters.

[0077] S23, each aircraft determines the first state corresponding to the service flow information sent by the wireless industrial equipment in its cluster at the current time step, infers the first scheduling decision corresponding to the first state based on the locally updated decision reasoning model, schedules the service flow information according to the first scheduling decision, and obtains the first reward value and the second state after scheduling.

[0078] Specifically, after updating the model parameters of the locally deployed decision-making inference model, the aircraft can, based on the updated model, infer a first scheduling decision corresponding to the first state of the service flow information sent at the current time step. This first scheduling decision includes a scheduling time slot and a scheduling path. The aircraft can then send the service flow information according to the scheduled path within the determined scheduling time slot, thereby achieving the scheduling of the service flow information. Subsequently, a first reward value and a second state after scheduling can be obtained. In other examples, the first scheduling decision may also include frequency bandwidth.

[0079] It should be understood that the state items in the first and second states corresponding to the service flow information at the current time step can be found in the description above. That is, the first and second states can include deterministic indicator information before and after this scheduling, as well as basic attribute information of the service flow information at the current time step.

[0080] S24. If the second state satisfies the deterministic scheduling condition, based on the first state, the first scheduling decision, the first reward value, and the second state, generate and share new sample trajectory tuples, and enter a new current time step to continue iterating until the iteration stop condition is met, thus obtaining the target reasoning model used to achieve information deterministic scheduling.

[0081] The aircraft can determine whether the second state after scheduling meets the pre-set deterministic scheduling conditions. If it does, it can generate a new sample trajectory tuple based on the first state, the first scheduling decision, the first reward value, and the second state at the current time step. This new sample trajectory tuple is then updated in the local experience pool and synchronized with other aircraft. The sample trajectory tuples synchronized by other aircraft are stored in the local experience pool. Further, the next time step is taken as the new current time step, and the process returns to iterative execution steps S22 to S24 to continue iteratively updating the locally deployed decision reasoning model and iteratively generating and sharing new sample trajectory tuples. This continues until the iteration stops, resulting in a decision reasoning model for achieving deterministic scheduling of the aircraft's cluster domain, which can be denoted as the target reasoning model.

[0082] In the above method, each aircraft and the industrial equipment within its coverage area construct the same cluster domain, which is managed by the aircraft. Each aircraft locally deploys a decision-making reasoning model to be trained. To enable the decision-making reasoning model to achieve deterministic scheduling, a Markov decision process is used. Deterministic index information related to deterministic scheduling at each time step is treated as a state, and scheduling decisions are treated as actions. Combined with deterministic scheduling conditions that act as constraints, multiple aircraft iteratively and collaboratively train the decision-making reasoning model. In this way, each aircraft can train a target reasoning model applicable to its own cluster domain, which can solve the end-to-end deterministic scheduling problem of differentiated service flows across multiple cluster domains. It can reason more suitable resource scheduling decisions (i.e., adapting sufficient time slot capacity, forwarding paths, frequency bandwidth, and other resources) for each service flow information transmitted on the network, thereby better ensuring the deterministic scheduling of service flow information, ensuring deterministic transmission latency, jitter, and zero packet loss for differentiated service flows. In addition, it can also improve the utilization rate of various network resources.

[0083] In some embodiments, the deterministic metric information includes end-to-end deterministic characterization information and link capacity utilization. End-to-end deterministic characterization information is used to measure or characterize whether end-to-end transmission of service flow information meets deterministic guarantees, including at least one of latency, jitter, or packet loss rate throughout the entire scheduling process (from the first user equipment to the second user equipment or from the second user equipment to the first user equipment). Link capacity utilization is used to characterize the resource utilization of the link within each time slot. It should be understood that scheduling a service flow consumes time slot resources and link resources; therefore, the link capacity utilization changes before and after service flow scheduling, equivalent to a state transition.

[0084] As mentioned above, the status item can also include basic attribute information. Basic attribute information includes at least one of the following: service flow cycle, source address, destination address, service flow size, or number of packets sent. Among these, the service flow cycle is the period at which service flow information is sent; that is, service flow information is sent periodically according to the service flow cycle.

[0085] In some examples, the entire state space S includes the business flow information state space S arriving at each time step. f and link capacity utilization state space S o That is, S = S f ∪S o .

[0086] Among them, record Let S represent the resource utilization rate of link L in the κ-th time slot at the t-th time step. Then the link capacity utilization state space S o The formula is expressed as:

[0087] The basic attribute information and end-to-end deterministic representation information of the business flow information arriving at each time step constitute the business flow information state space S. f For example, the business flow information state space S f The formula is expressed as:

[0088] S f ={f i .period(t),f i .src(t),f i .dst(t),f i .size(t),f i .pkt(t),Δd i ,Δj i ,Δl i};

[0089] That is, the basic attribute information of the i-th business flow information arriving at time step t includes the aforementioned business flow period f. i .period(t), source address f i .src(t), destination address f i .dst(t), Business flow size f i .size(t), number of packets f i .pkt(t); End-to-end deterministic characterization information includes the end-to-end transmission delay Δd of the i-th service flow information. i jitter Δj i Packet loss rate Δl i .

[0090] In some embodiments, the service flow information state space S f It may also include wireless link transmission latency, that is, the latency that occurs from when the service flow information is sent by the previous hop node to when it is successfully received by the wireless logical node in the wireless link.

[0091] In the above embodiments, the state space can be determined more accurately from the perspective of the service flow itself and the end-to-end deterministic transmission demand indicators. This allows for more accurate prediction of corresponding scheduling decisions (i.e. actions) based on the state space, thereby improving the success rate of deterministic scheduling.

[0092] In some embodiments, deterministic scheduling conditions include at least one of the following: transmission capacity constraints, wireless link transmission delay constraints, or end-to-end deterministic demand constraints.

[0093] The aforementioned deterministic scheduling condition is used to constrain dynamic state items (i.e., state items whose values ​​change after each scheduling). Therefore, it is determined whether the new state (i.e., the second state) after each scheduling satisfies the deterministic scheduling condition.

[0094] In addition, deterministic scheduling conditions may include at least one of the following: service flow scheduling period constraints and network time slot size constraints. These two conditions are not used to constrain dynamic state items; therefore, the new state after each scheduling is not compared with these two conditions. However, these two conditions are used to set the values ​​of two key parameters related to deterministic scheduling: the service flow scheduling period P and the time slot length T.

[0095] The following text will explain each deterministic scheduling condition.

[0096] The service flow scheduling period P is a deterministic scheduling period. It should be understood that the scheduling system processes service flow information periodically, that is, it schedules service flow information periodically. The service flow scheduling period P determines the number of time steps within a period. Since one time step schedules one service flow piece of information, it also determines the total number of service flow pieces of information that need to be scheduled within a period.

[0097] The service flow scheduling period constraint is used to ensure that the length of the service flow scheduling period P is equal to the least common multiple of the service flow periods. It should be understood that the sending end continuously sends service flow information according to a preset period (i.e., periodically sends service flow information). Each service flow information has its own corresponding service flow period, which refers to the period during which the service flow information is continuously sent. Therefore, taking the least common multiple of the service flow periods as the service flow scheduling period ensures that within each service flow scheduling period, the periodically sent service flow information is scheduled to the same time slot.

[0098] For example, the formula for the service flow scheduling cycle constraint is described as follows:

[0099] P = LCM(f i (period), i = 1, 2, ..., |F|;

[0100] Among them, f i .period represents the business flow f i The service flow cycle, LCM(·) is a function to calculate the least common multiple. The set of service flow information sent / scheduled within a service flow scheduling cycle is represented by F. The total number of service flow information is modulo the set F, denoted as |F|.

[0101] The network time slot size constraint requires that the time slot length T does not exceed the greatest common divisor (GCD) of all service flow cycles, thereby avoiding competition for the same transmission time slot for service flows from different cycles. Let GCD(·) denote the function for calculating the GCD. The formula for the network time slot size constraint is described as follows:

[0102] T≤GCD(f i(period), i = 1, 2, ..., |F|;

[0103] The transmission capacity constraint requires that the total size of the service flow information transmitted in each time slot does not exceed the maximum capacity of that time slot. Specifically, let the function ψ κ (·) indicates whether the service flow information has been scheduled to the κ-th time slot. If the scheduling is successful, then ψ κ (·) = 1, otherwise 0. Therefore, the formula for the transmission capacity constraint is described as follows:

[0104]

[0105] Where F is a set of service flow information that needs to be scheduled within a service flow scheduling cycle, and M is the service flow information f i The set of data packets, f i .size is the i-th business flow information f i Size, f i .pkt m Represents business flow information f i The m-th data packet, ψ κ (f i .pkt m ) represents business flow information f i Whether the m-th data packet was successfully scheduled to the κ-th time slot, and C represents the maximum capacity of a single time slot. On the wired network side, the time slot capacity is related to the link bandwidth and the time slot length; therefore, the maximum capacity C of a single time slot is described as C = η × B × T, where η is the bandwidth utilization, B is the frequency bandwidth, and T is the time slot length. On the wireless network side, the time slot capacity is related to the frequency bandwidth, transmit power, channel interference, and the time slot length; therefore, the maximum capacity C of a single time slot is described as C = η × B × log2(1+γ) × T, where η is the bandwidth utilization, B is the frequency bandwidth, γ is the signal-to-noise ratio, and T is the time slot length.

[0106] The wireless link transmission delay constraint is used to constrain the magnitude of the wireless link transmission delay. Specifically, the wireless link transmission delay constraint requires that the wireless link transmission delay (i.e., the delay incurred from the time the service flow information is sent by the previous hop node to the time it is successfully received by the wireless logical node in the wireless link) does not exceed the time slot length T, to ensure the determinism of the service flow information forwarding delay at the wireless logical node. Let the time of transmission of the service flow information at the previous hop node of the wireless logical node be denoted as . The moment when the service flow information is successfully received by the wireless logical node is The formula for the wireless link transmission delay constraint is described as follows:

[0107] End-to-end deterministic requirement constraints refer to the conditions that constrain the deterministic representation information at each end during the entire process of end-to-end transmission of service flow information. Specifically, this constraint requires a time delay Δd for end-to-end transmission. i jitter Δj i The packet loss rate Δl is required to not exceed the corresponding upper bound value. i Approximately equal to the preset packet loss rate threshold, the formula for this constraint is described as follows:

[0108]

[0109] Among them, f i .delay and f i .jitter represents the business flow f respectively. i The upper bound of latency and the upper bound of jitter, and the packet loss rate Δl i The corresponding preset packet loss rate threshold is 0.

[0110] In some embodiments, the sender and receiver of service flow information in the scheduling system communicate with their respective corresponding direct communication devices via a wired network. It should be understood that the sender and receiver can be a first user equipment or a second user equipment. Regardless of whether it is a first user equipment or a second user equipment, it communicates with its corresponding direct communication device (wireless industrial equipment or base station) via a wired network. Each aircraft communicates with the direct communication device corresponding to the first or second user equipment via a wireless network, and different aircraft also communicate with each other via a wireless network (i.e., as shown in the image). Figure 1 (As shown).

[0111] In iterative training, the optimization objectives and deterministic scheduling conditions are determined based on the queue traffic shaping model; the queue traffic shaping model includes the circular queue shaping mechanism for wired networks and the time slot orchestration rules for wireless networks.

[0112] The wired network used by the sending or receiving end to communicate with the corresponding direct communication device is based on a circular queue shaping mechanism to ensure hop-by-hop deterministic forwarding of service flow information. Specifically, each device communicating via the wired network maintains a pair of ping-pong queues on its transmitting interface, ensuring that in each time slot, exactly one queue sends service flow information to downstream nodes, while the other queue receives service flow information forwarded by upstream nodes. The transmit / receive status of each queue cycles through different time slots. This circular queue shaping mechanism requires that service flow information sent by upstream nodes within a time slot be completely received by downstream nodes within the same time slot, and that downstream nodes continue forwarding service flow information when the next time slot arrives.

[0113] Each segment of the aircraft's wireless network acts as a logical forwarding node. Wireless network communication ensures deterministic forwarding of service flow information across multiple hops based on time slot orchestration rules. Specifically, a Time-Sensitive Network Translator (TSN translator) is embedded on the transmitting interface of each device communicating through the wireless network. The time slot orchestration rules require that service flow information arriving in advance of the orchestrated time slot be buffered by the TSN translator and delayed until the correct pre-arranged time slot for transmission, thereby achieving deterministic scheduling and forwarding of service flow information within the wireless network.

[0114] In some embodiments, after constructing the cluster domains, a queue traffic shaping model can be built. Based on the queue traffic shaping model, a cross-domain deterministic resource scheduling constraint optimization problem is established, that is, the optimization objective and deterministic scheduling conditions are defined. Then, a Markov decision process is designed in conjunction with the scenario of this case (i.e., the state space, action space, and reward value specific to this scenario are designed). Then, a multi-domain cooperative scheduling algorithm is used to train the target inference model corresponding to each aircraft and adapted to its respective cluster domain through deep reinforcement learning. Subsequently, when sending service flow information within the cluster domain of each aircraft, the network resources (including time slot resources, path resources, and bandwidth resources) of the cluster domain can be reasonably allocated through its adapted target inference model.

[0115] Therefore, the mechanisms and rules in the aforementioned queue traffic shaping model can serve as a prerequisite for formulating optimization objectives and deterministic scheduling conditions. That is, the optimization objectives and deterministic scheduling conditions are formulated based on the overall architecture provided by the queue traffic shaping model. By implementing the mechanisms and rules required by the queue traffic shaping model through optimization objectives and deterministic scheduling conditions, and based on the ping-pong queues set up for wired networks and the time-sensitive network translator set up for wireless networks in the queue traffic shaping model, the feasibility of formulating optimization objectives and deterministic scheduling conditions is increased.

[0116] In some embodiments, step S22 includes: each aircraft sampling sample trajectory tuples from the shared sample trajectory tuples; for each sampled sample trajectory tuple, determining a probability ratio of a first probability to a second probability; the first probability is the probability of inferring a first sample scheduling decision for a first sample state in the sample trajectory tuple using the current parameters of the decision inference model; the second probability is the probability of inferring a sample scheduling decision for a first sample state using the parameters of the last updated decision inference model; calculating an advantage value based on the sample reward value in the sample trajectory tuple and the first state evaluation value corresponding to the first and second sample states respectively; determining a first loss value based on the probability ratio and the advantage value; and updating and adjusting the parameters of the locally deployed decision inference model based on the first loss value corresponding to each sampled sample trajectory tuple.

[0117] For example, the loss function corresponding to the decision reasoning model is shown below (Equation 1):

[0118]

[0119] Among them, E t (·) is a function for calculating the expectation. It's a probability ratio, π θ [a(t)|s(t)] is the first probability, π θ0 [a(t)|s(t)] is the second probability, θ is the current parameter of the decision reasoning model, θ0 is the old parameter of the decision reasoning model in the last update, and the clip(·) function is used to limit the probability ratio between [1-ε, 1+ε] to reduce jitter generated during training. This represents the advantage value. `min()` is a function that takes the minimum value, which is the first loss value. That is, the expected first total loss value is calculated for the first loss value corresponding to each sample trajectory tuple. During iterative training, the parameters of the decision-making inference model are updated in the direction of maximizing the first total loss value.

[0120] In some examples, the first-state evaluation values ​​corresponding to the first and second sample states are predicted by the evaluation neural network in the previous iteration. The evaluation neural network is used to evaluate the scheduling decisions inferred by the decision reasoning model and guide the decision reasoning model to update in the direction of higher evaluation values.

[0121] For example, advantage value The calculation formula is shown below (Equation 2):

[0122]

[0123] Where λ,ω∈[0,1] are discount factors; This represents the temporal difference error (TD error) at time step t. r(t) is the sample reward value. It is the first state evaluation value predicted by the evaluation neural network in the previous iteration (the parameter of the evaluation neural network in the previous iteration is μ0) for the second sample state. This is the first state evaluation value predicted by the evaluation neural network for the first sample state in the previous iteration. l is the offset relative to time step t. Equation (2) integrates the temporal difference error at time step t and the temporal difference error at time step t in the future l steps, thus more accurately determining the dominance value. T' is the total step size within a business flow scheduling cycle.

[0124] In some embodiments, in each iteration, the aircraft can also determine the target state evaluation value corresponding to the first sample state based on the first state evaluation value and the advantage value corresponding to the first sample state; predict the second state evaluation value of the first sample state based on the current evaluation neural network in this iteration; determine the second loss value based on the difference between the target state evaluation value and the second state evaluation value; and perform an iterative update on the parameters of the current evaluation neural network based on the second loss value corresponding to each sample trajectory tuple.

[0125] In some embodiments, the loss function for evaluating the neural network is shown in Equation 3 below:

[0126]

[0127] Where μ is the parameter of the current evaluation neural network in this iteration, and μ0 is the old parameter of the evaluation neural network in the previous iteration. It is the first state evaluation value corresponding to the first sample state. This is the advantageous value calculated in this iteration. It is the target state evaluation value, V μ [s(t)] is the second-state evaluation value of the first sample state. The second loss value corresponds to a tuple of sample trajectories; E t (·) is the function for calculating the expectation. That is, the expected second total loss value is calculated for the second loss value corresponding to each sample trajectory tuple. During iterative training, the parameters of the evaluation neural network are updated in the direction that minimizes the second total loss value.

[0128] In some embodiments, the method further includes: after receiving target service flow information sent by the transmitter, the target aircraft obtains the target state corresponding to the target service flow information; the target aircraft is any one of multiple aircraft; the target aircraft inputs the target state into a locally deployed target inference model and outputs the target scheduling decision corresponding to the target service flow information; the target scheduling decision includes the target scheduling time slot and the target scheduling path; the target aircraft forwards the target service flow information to the designated receiver in the target scheduling time slot according to the target scheduling path.

[0129] For example, the target state includes basic attribute information of the target service flow information, end-to-end deterministic characterization information, and link capacity utilization. The target scheduling decision may also include target frequency bandwidth. The target aircraft allocates target frequency bandwidth for the target service flow information in the target scheduling time slot, thereby forwarding the target service flow information to the designated receiving end according to the target scheduling path based on the target frequency bandwidth. This target scheduling path may be a multi-hop cross-domain scheduling path.

[0130] In some embodiments, the sending end may be a first user equipment, and the designated receiving end is a second user equipment communicating with a base station. The first user equipment communicates directly with wireless industrial equipment within the cluster domain where the aircraft is located via a wired network. The wireless industrial equipment forwards the target service flow information sent by the first user equipment to the target aircraft via a wireless network. The target service flow information is then forwarded by the target aircraft to the second user equipment via the base station. That is, the target aircraft transmits the target service flow information to the base station via a wireless network, and the base station then sends the target service flow information to the second user equipment via a wired network.

[0131] In some embodiments, the sending end can be a second user equipment, and the designated receiving end is a first user equipment. The second user equipment communicates directly with the corresponding base station via a wired network to send target service flow information to the base station. The base station forwards the target service flow information to the target aircraft via a wireless network. The target service flow information is then forwarded by the target aircraft to the second user equipment via wireless industrial equipment in its cluster domain. That is, the target aircraft transmits the target service flow information to the wireless industrial equipment in its cluster domain via a wireless network, and the wireless industrial equipment then sends the target service flow information to the first user equipment via a wired network.

[0132] Although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. At least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least a portion of steps or stages in other steps.

[0133] Based on the same inventive concept, this application also provides an information deterministic scheduling apparatus for implementing the aforementioned method. The solution provided by this apparatus is similar to the solution described in the above method; therefore, the specific limitations in one or more information deterministic scheduling apparatus embodiments provided below can be found in the limitations of the information deterministic scheduling method described above, and will not be repeated here.

[0134] In one embodiment, such as Figure 3 As shown, an information deterministic scheduling device is provided, which is installed in an aircraft and includes:

[0135] Clustering domain construction module 302 is used to construct a clustering domain with the aircraft as the cluster head node and wireless industrial equipment within the signal coverage area of ​​the aircraft as member nodes.

[0136] The model parameter update module 304 is used to iteratively update the parameters of the locally deployed decision reasoning model based on the sample trajectory tuples shared by each aircraft in the current time step. Each sample trajectory tuple includes a first sample state, a sample scheduling decision, a sample reward value, and a second sample state. The first sample state includes deterministic indicator information before the scheduling of sample business flow information. The second sample state includes deterministic indicator information after the scheduling.

[0137] Information scheduling module 306 is used to determine the first state corresponding to the service flow information sent by the wireless industrial equipment in the current time step in the cluster domain, infer the first scheduling decision corresponding to the first state based on the locally updated decision reasoning model, schedule the service flow information according to the first scheduling decision, and obtain the first reward value and the second state after scheduling.

[0138] The deterministic scheduling constraint module 308 is used to generate and share new sample trajectory tuples based on the first state, the first scheduling decision, the first reward value, and the second state if the second state satisfies the deterministic scheduling conditions, and then enter a new current time step to continue iterating until the iteration stop condition is reached, thereby obtaining the target reasoning model for achieving information deterministic scheduling.

[0139] In some embodiments, such as Figure 4 As shown, the information scheduling module 306 includes:

[0140] The scheduling decision reasoning module 3061 is used to receive the target service flow information sent by the sending end and obtain the target status corresponding to the target service flow information; the target aircraft is any one of multiple aircraft; the target aircraft inputs the target status into the locally deployed target reasoning model and outputs the target scheduling decision corresponding to the target service flow information; the target scheduling decision includes the target scheduling time slot and the target scheduling path;

[0141] The scheduling strategy execution module 3062 is used to forward the target service flow information to the designated receiving end according to the target scheduling path during the target scheduling time slot.

[0142] Each module in the aforementioned deterministic scheduling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the electronic device in hardware form or independent of it, or stored in the memory of the electronic device in software form, so that the processor can call and execute the operations corresponding to each module.

[0143] In one embodiment, an electronic device is provided, which may be an aircraft. Figure 5 As shown, this electronic device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a deterministic information scheduling method.

[0144] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0145] In one embodiment, an electronic device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the embodiments of this application.

[0146] In one embodiment, such as Figure 6 As shown, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in the embodiments of this application.

[0147] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the embodiments of this application.

[0148] It should be noted that the user information (including but not limited to user device information, user attribute content, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0149] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited thereto. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. As long as the combination of these technical features does not contradict each other, it should be considered as within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, but should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. The scope of protection of this application shall be determined by the appended claims.

Claims

1. An information deterministic scheduling method, characterized by, The method comprises: For each aircraft in the scheduling system, taking the aircraft as a cluster head node and the wireless industrial devices within the signal coverage range of the aircraft as member nodes, a clustering domain is constructed; In the current time step, each aircraft updates the parameters of the locally deployed decision reasoning model based on the sample trajectory tuples shared by each other; each sample trajectory tuple comprises a first sample state, a sample scheduling decision, a sample reward value and a second sample state; the first sample state comprises deterministic index information of sample traffic flow information before scheduling; the second sample state comprises deterministic index information after scheduling; Each aircraft determines the first state corresponding to the traffic flow information sent by the wireless industrial devices in the clustering domain at the current time step, reasons the first scheduling decision corresponding to the first state based on the locally updated decision reasoning model, schedules the traffic flow information according to the first scheduling decision, and obtains a first reward value and a second state after scheduling; If the second state satisfies the deterministic scheduling condition, a new sample trajectory tuple is generated and shared according to the first state, the first scheduling decision, the first reward value and the second state, and the iteration continues in a new current time step until the iteration stopping condition is reached, and a target reasoning model for realizing information deterministic scheduling is obtained.

2. The method of claim 1, wherein, Each aircraft shares the sample trajectory tuples generated by itself to each aircraft in the scheduling system; each aircraft stores the sample trajectory tuples generated by itself and shared to the local experience pool; The method further comprises: In the current time step, if the first number of sample trajectory tuples in the local experience pool is greater than or equal to a preset sample number threshold, the each aircraft updates the parameters of the locally deployed decision reasoning model based on the sample trajectory tuples shared by each other and the subsequent steps are executed; If the first number is less than the sample number threshold, each aircraft determines the first state, reasons the second scheduling decision corresponding to the first state based on the initial decision reasoning model, schedules the traffic flow information according to the second scheduling decision, and obtains a second reward value and a third state after scheduling; If the third state satisfies the deterministic scheduling condition, a new sample trajectory tuple is generated and shared according to the first state, the second scheduling decision, the second reward value and the third state, the sample trajectory tuples newly generated in the current time step and the sample trajectory tuples newly shared are added to the local experience pool, and the iteration processing is entered in a new current time step.

3. The method of claim 1, wherein The deterministic index information comprises end-to-end deterministic representation information and link capacity utilization; the deterministic representation information comprises at least one of end-to-end delay, jitter degree or packet loss rate; and the link capacity utilization is used to represent the resource utilization of the link in each time slot; And / or, The first sample state and the second sample state also include basic attribute information of the sample traffic flow information; the basic attribute information includes at least one of traffic flow period, source address, destination address, traffic flow size, or packet number; And / or, The sample reward value is determined based on a difference in the number of successfully scheduled traffic flow information at adjacent time steps.

4. The method of claim 1, wherein, In each iteration, each aircraft iteratively updates parameters of the decision reasoning model based on an optimization objective; the optimization objective is to maximize the number of network accommodated traffic flow information; and the deterministic scheduling condition includes at least one of a transmission capacity constraint condition, a wireless link transmission delay constraint condition, or an end-to-end deterministic demand constraint condition.

5. The method of claim 4, wherein, In the scheduling system, the sending end and the receiving end communicate with respective corresponding direct communication devices through a wired network, and each aircraft and the direct communication devices and different aircrafts communicate through a wireless network; The optimization objective and the deterministic scheduling condition are determined based on a queue flow shaping model; the queue flow shaping model includes a cyclic queue shaping mechanism corresponding to the wired network and a time slot arrangement rule corresponding to the wireless network; Each device communicating through the wired network maintains a pair of ping-pong queues on the sending interface of each device, and in each time slot, only one queue sends traffic flow information to a downstream node, and the other queue is used to receive traffic flow information forwarded by an upstream node; the cyclic queue shaping mechanism requires that the traffic flow information sent by the upstream node in a time slot be completely received by the downstream node in the same time slot, and the downstream node continues to forward the traffic flow information until the next time slot arrives; Each device communicating through the wireless network embeds a time limit sensitive network translator on the sending interface of each device, and the time slot arrangement rule requires that the traffic flow information arriving earlier than the arranged time slot be buffered and delayed to the correct time slot arranged in advance through the time limit sensitive network translator.

6. The method of claim 1, wherein, Each aircraft iteratively updates parameters of the locally deployed decision reasoning model based on the mutually shared sample trajectory tuples, including: Each aircraft samples a sample trajectory tuple from the shared sample trajectory tuples, and determines a probability ratio of a first probability and a second probability for each sampled sample trajectory tuple; the first probability is a probability of reasoning the first sample scheduling decision for the first sample state in the sample trajectory tuple using the current parameters of the decision reasoning model; and the second probability is a probability of reasoning the sample scheduling decision for the first sample state using the last updated parameters of the decision reasoning model; An advantage value is calculated according to the sample reward value in the sample trajectory tuple and first state evaluation values corresponding to the first sample state and the second sample state; A first loss value is determined according to the probability ratio and the advantage value; Parameters of the locally deployed decision reasoning model are updated based on the first loss value corresponding to each sampled sample trajectory tuple.

7. The method of claim 6, wherein, The first state evaluation value is predicted by a judgment neural network in the last iteration. In each iteration, the method further comprises: determining a target state evaluation value corresponding to the first sample state according to the first state evaluation value corresponding to the first sample state and the advantage value; predicting a second state evaluation value of the first sample state according to the current evaluation neural network in this iteration; determining a second loss value according to the difference between the target state evaluation value and the second state evaluation value; performing an iterative update on the parameters of the current evaluation neural network based on the second loss value corresponding to each sample trajectory tuple.

8. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: After the target aircraft receives the target service flow information sent by the sending end, the target aircraft obtains a target state corresponding to the target service flow information; the target aircraft is any one of the plurality of aircrafts; The target aircraft inputs the target state into the target inference model deployed locally, and outputs a target scheduling decision corresponding to the target service flow information; the target scheduling decision includes a target scheduling time slot and a target scheduling path; The target aircraft forwards the target service flow information to the specified receiving end according to the target scheduling path in the target scheduling time slot.

9. The method of claim 8, wherein: the sending end is a first user equipment, and the specified receiving end is a second user equipment in communication with a base station; the first user equipment directly communicates with a wireless industrial equipment in a cluster domain where the aircraft is located through a wired network; the wireless industrial equipment forwards the target service flow information sent by the first user equipment to the target aircraft through a wireless network; and the target service flow information is forwarded by the target aircraft to the second user equipment through the base station; alternatively, the sending end is the second user equipment, and the specified receiving end is the first user equipment; the second user equipment directly communicates with a corresponding base station through a wired network to send the target service flow information to the base station, and the base station forwards the target service flow information to the target aircraft through a wireless network; and the target service flow information is forwarded by the target aircraft to the second user equipment through a wireless industrial equipment in a cluster domain where the target aircraft is located.

10. An information deterministic scheduling apparatus characterized by comprising: The device is arranged in each aircraft in a scheduling system, and comprises: a cluster domain construction module, configured to construct a cluster domain with an aircraft as a cluster head node and wireless industrial equipment in a signal coverage range of the aircraft as member nodes; a model parameter update module, configured to perform an iterative update on parameters of a locally deployed decision inference model based on sample trajectory tuples shared by each aircraft with each other in a current time step; each sample trajectory tuple comprises a first sample state, a sample scheduling decision, a sample reward value, and a second sample state; the first sample state comprises deterministic index information before scheduling of sample service flow information; and the second sample state comprises deterministic index information after scheduling. The information scheduling module is configured to determine a first state corresponding to traffic flow information transmitted by wireless industrial devices in the cluster domain at the current time step, infer a first scheduling decision corresponding to the first state based on a locally updated decision inference model, schedule the traffic flow information according to the first scheduling decision, and obtain a first reward value and a second state after scheduling; and the deterministic scheduling constraint module is configured to, if the second state satisfies a deterministic scheduling condition, generate and share a new sample trajectory tuple according to the first state, the first scheduling decision, the first reward value and the second state, and continue iteration in a new current time step until an iteration stop condition is reached, to obtain a target inference model for implementing information deterministic scheduling.

Images