Joint optimization method for task scheduling and resource deployment in industrial time-sensitive network and related application thereof

Abstract

The invention discloses a task scheduling and resource deployment joint optimization method in an industrial time-sensitive network and related application thereof. The method comprises the steps that an industrial time-sensitive network scheduling model containing multiple industrial production cycles and an agent interaction environment are constructed, an agent is configured to obtain a global embedding state through graph neural network coding in each production cycle, and equipment deployment and task scheduling decisions are sequentially executed through a strategy network, and after the period is ended, calculating a reward value to update the strategy network, and iteratively completing full-period joint optimization, thereby ensuring that all key communication flows are delivered on time, realizing cross-period optimization of equipment deployment and task scheduling, realizing scientific decision making at the beginning of each production period, overcoming the limitation of traditional single-period scheduling, and improving the scheduling efficiency. The overall efficiency of industrial production and the network resource utilization rate are improved in a longer time span, the production efficiency is effectively improved, and the cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of industrial communication network scheduling and optimization, and in particular to a joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks and its related applications. Background Technology

[0002] Time-Sensitive Networking (TSN) is a novel Ethernet communication technology for industrial control and IoT applications. It ensures deterministic transmission and low latency of critical traffic through precise time scheduling. In the context of modern industry, sensors, actuators, and controllers in factories generate periodic critical real-time flows that require strict latency limits to ensure the safety and efficiency of the production process.

[0003] However, in actual industrial production, production demands are dynamic: manufacturing systems typically operate in multiple consecutive production cycles. Different cycles may introduce new production tasks, each requiring specific types of equipment (such as sensors or robot controllers) deployed on the network's end nodes, generating their own time-sensitive communication flows. Whenever a new production cycle begins, network planners need to schedule communication flows for the new tasks of that cycle and may also need to adjust the equipment deployment in the network (adding, removing, or reallocating equipment) to meet task requirements. This creates an optimization problem combining cross-cycle resource deployment with intra-cycle task scheduling.

[0004] Most existing technologies focus on time-sensitive network flow scheduling within a single cycle or assume that network topology and equipment configuration remain fixed. However, when faced with multiple consecutive production cycles and constantly changing task requirements, existing methods fail to consider cross-cycle equipment deployment decisions. For example, past studies have only focused on flow scheduling optimization within a single cycle, neglecting the procurement and reuse of equipment across different cycles. This can lead to the repeated procurement of the same equipment in each cycle or a lack of overall planning for equipment usage, resulting in resource waste and unnecessary costs. Furthermore, tasks generated in different cycles often compete for equipment usage. While some tasks can share existing equipment to save costs, single-cycle optimization cannot coordinate equipment reconfiguration and sharing between cycles. Switching equipment between cycles incurs reinstallation or disassembly configuration overhead, and scheduling strategies lacking cross-cycle coordination may operate independently in each cycle, failing to fully reuse equipment installed in the previous cycle and potentially over-purchasing equipment in the current cycle due to neglecting future needs. Therefore, methods that only consider single-cycle task scheduling are insufficient to meet the overall optimization requirements of multiple production cycles in dynamic industrial time-sensitive network scenarios. An optimization method that can jointly consider task scheduling and equipment deployment across multiple cycles is urgently needed. Summary of the Invention

[0005] The main objective of this invention is to provide a joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks and its related applications. This method aims to solve the technical problem that existing technologies cannot effectively coordinate multi-cycle equipment deployment and task scheduling while ensuring the real-time constraints of critical flows, resulting in low production efficiency and high costs.

[0006] To achieve the above objectives, this invention provides a joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks, the method comprising the following steps: An industrial time-sensitive network (ITN) scheduling model encompassing multiple industrial production cycles is constructed. An agent interaction environment is configured based on this model. The initial input state of the ISN scheduling model includes the task set for each industrial production cycle, the ISN topology, and the device deployment status of each terminal node. The agent interaction environment is configured with agents, which are equipped with a policy network and are configured to perform task scheduling and device deployment. The specific process is as follows: At the beginning of each industrial production cycle, the task requirements of the current production cycle are obtained, and the industrial time-sensitive network topology and the equipment deployment status of each terminal node are encoded by a graph neural network to obtain the global embedding state. Based on the task requirements and the global embedding state, the device deployment decision and task scheduling decision are executed sequentially through the policy network to obtain the device deployment scheme and task scheduling scheme for the current production cycle. At the end of each industrial production cycle, an instant reward value is calculated based on the task completion status and equipment deployment status of the current production cycle, and the policy network is updated with a policy gradient based on the instant reward value. At the start of the next industrial production cycle, the task requirements for obtaining the current production cycle are returned based on the updated policy network, and the state encoding module of the graph neural network is used to encode the topology of the industrial time-sensitive network and the equipment deployment status of each terminal node to obtain the global embedded state. This process continues until all industrial production cycles are completed, thus achieving joint optimization of task scheduling and resource deployment of the industrial time-sensitive network. The policy network includes a device deployment actor network, a task scheduling actor network, and a critic network. The device deployment actor network and the task scheduling actor network are configured to update the device deployment scheme and the task scheduling scheme based on the global embedded state. The critic network is configured to guide the policy gradient update of the actor network.

[0007] Optionally, the step of obtaining the task requirements of the current production cycle and encoding the industrial time-sensitive network topology and the equipment deployment status of each terminal node using a graph neural network to obtain the global embedding state includes: Extract feature information of each node in an industrial time-sensitive network and generate feature vectors corresponding to each node, wherein the nodes include terminal nodes and switching nodes; Based on the topology of industrial time-sensitive networks, a network topology graph is constructed. The network topology graph includes a node set and an edge set. The node set includes all terminal nodes and switching nodes, and the edge set includes the link connection relationships between nodes. The feature vectors of each node and the network topology graph are input into the graph neural network, and the node embedding vectors corresponding to each node are calculated through the message passing mechanism. The mean of all node embedding vectors is calculated to obtain the network embedding vector; The task requirements of the current production cycle are extracted and summarized to obtain the overall task requirement feature vector. The network embedding vector is merged with the overall task requirement feature vector to generate a global embedding state.

[0008] Optionally, the specific process of making the equipment deployment decision and generating the equipment deployment plan includes: Device feature information is extracted from the global embedded state and input into the device deployment actor network. The device feature information includes device cost information and task device requirement information. The fully connected layer of the device deployment actor network calculates the input device feature information to obtain the action preference value of each terminal node corresponding to the optional actions. The optional actions include not performing the operation, installing the device, and removing the deployed device. The action preference values ​​are normalized using a normalized exponential function to obtain the policy probability distribution of the actions corresponding to each terminal node. Based on the probability distribution of the strategy, select the corresponding device operation action for each terminal node, execute the device operation action, and synchronously update the device status and cost statistics information of each terminal node. Once the equipment requirements for all tasks are met, or all terminal nodes have completed their action selections, a device deployment plan for the current production cycle is generated.

[0009] Optionally, the task scheduling decision process includes path selection for executing the task scheduling decision, which includes: Based on the equipment deployment plan of the current production cycle, update the scheduling status of the industrial time-sensitive network and obtain the time-sensitive communication stream set corresponding to all tasks to be scheduled in the current production cycle. For a single time-sensitive communication flow to be scheduled, extract the attribute parameters of the time-sensitive communication flow and the corresponding set of candidate transmission paths, wherein the set of candidate transmission paths contains at least one candidate transmission path; The link feature sequence of each candidate transmission path is input into the stream encoder to extract the path-level embedding vector corresponding to each candidate transmission path. All path-level embedding vectors are fused with the attribute parameters of the time-sensitive communication stream to obtain the task-level embedding vector corresponding to the time-sensitive communication stream; The task-level embedding vector is input into the task scheduling actor network to calculate the score value corresponding to each candidate transmission path. The score value is then processed by a normalized exponential function to obtain the selection probability distribution of each candidate transmission path. Based on the selection probability distribution, a transmission path is selected for the time-sensitive communication stream until all time-sensitive communication streams to be scheduled have completed the selection of transmission paths.

[0010] Optionally, the specific process of generating the task scheduling scheme includes: According to the preset scheduling order, traverse all time-sensitive communication flows of the selected transmission paths and determine all links and corresponding node ports on the transmission path corresponding to a single time-sensitive communication flow. Along the transmission path of the time-sensitive communication stream, at the output port of the source node, the earliest idle time slot that meets the transmission time constraint is allocated for the data frame of the time-sensitive communication stream; Based on the allocated transmission time slots, link propagation delay, and node processing delay, the time for the data frame to reach the next hop node is determined, and an idle time slot that meets the connection requirements is allocated to the output port of the corresponding node for the data frame. The time slot allocation for the entire path of the time-sensitive communication flow is completed hop by hop. When the time slot allocation for the entire path is completed in accordance with the constraints, the time-sensitive communication flow is marked as successfully scheduled, and the time slot occupancy status of the corresponding link and port is updated synchronously. Once all time-sensitive communication streams have completed their time slot allocation attempts, a task scheduling scheme for the current production cycle is generated.

[0011] Optionally, the step of calculating the instant reward value based on the task completion status and equipment deployment status of the current production cycle includes: Count the number of time-sensitive communication flows that are successfully scheduled within the current production cycle, and add the corresponding first positive reward value to each successfully scheduled time-sensitive communication flow; Count the number of tasks in the current production cycle that have been successfully scheduled for all associated time-sensitive communication flows, and add the corresponding second positive reward value to each completed task; The system tracks equipment installation and dismantling actions performed during the current production cycle, and calculates the cost penalty for each equipment operation based on the equipment's purchase cost, reconfiguration cost, and residual value. Calculate the average utilization rate of all links in the industrial time-sensitive network during the current production cycle, and calculate the corresponding load balancing penalty value based on the average utilization rate. Based on the first positive reward value, the second positive reward value, the cost penalty value, and the load balancing penalty value, the real-time reward value for the current production cycle is calculated.

[0012] Optionally, updating the policy gradient of the policy network based on the instant reward value includes: Store the quadruple data corresponding to the current production cycle into the priority experience replay buffer. The quadruple data consists of the current state, the current action, the immediate reward value, and the state of the next cycle. When the amount of data in the experience playback buffer reaches a preset capacity threshold, or when the scheduling process of all industrial production cycles is completed, a preset number of sample data are extracted from the experience playback buffer according to priority. Based on the extracted sample data, the time-series difference target value is calculated through the commentator network, and the parameters of the commentator network are updated with the goal of minimizing the mean square error. Based on the updated critic network, the advantage function value corresponding to the sample data is calculated through the advantage function, and the advantage function value is then standardized. Based on the standardized advantage function value, policy gradient updates are performed on the device deployment actor network and the task scheduling actor network, respectively, and policy constraints are applied based on entropy regularization constraints. Perform a soft update on the target network of the commentator network to complete the parameter iteration of the policy network.

[0013] Furthermore, to achieve the above objectives, this invention also proposes a joint optimization device for task scheduling and resource deployment in industrial time-sensitive networks, which applies the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks as described above. The joint optimization device for task scheduling and resource deployment in industrial time-sensitive networks includes: The environment configuration module is used to construct an industrial time-sensitive network scheduling model containing multiple industrial production cycles, and to configure an agent interaction environment based on the industrial time-sensitive network scheduling model. The initial input state of the industrial time-sensitive network scheduling model includes the task set of each industrial production cycle, the industrial time-sensitive network topology, and the device deployment status of each terminal node. The agent interaction environment is configured with agents, which are configured with a policy network and are configured to perform task scheduling and device deployment. The specific process is as follows: The state encoding module is used to obtain the task requirements of the current production cycle at the beginning of each industrial production cycle, and to encode the industrial time-sensitive network topology and the equipment deployment status of each terminal node through a graph neural network to obtain the global embedded state. The decision execution module is used to execute device deployment decisions and task scheduling decisions sequentially through the policy network according to the task requirements and the global embedded state, so as to obtain the device deployment scheme and task scheduling scheme for the current production cycle. The strategy update module is used to calculate the instant reward value at the end of each industrial production cycle based on the task completion status and equipment deployment status of the current production cycle, and to perform a strategy gradient update on the strategy network based on the instant reward value. The iterative optimization module is used to return the task requirements of the current production cycle based on the updated policy network at the beginning of the next industrial production cycle, and to encode the topology of the industrial time-sensitive network and the equipment deployment status of each terminal node through the graph neural network state encoding module to obtain the global embedded state. This process continues until all industrial production cycles are completed, thus completing the joint optimization of task scheduling and resource deployment of the industrial time-sensitive network. The policy network includes a device deployment actor network, a task scheduling actor network, and a critic network. The device deployment actor network and the task scheduling actor network are configured to update the device deployment scheme and the task scheduling scheme based on the global embedded state. The critic network is configured to guide the policy gradient update of the actor network.

[0014] Furthermore, to achieve the above objectives, this application also proposes a joint optimization device for task scheduling and resource deployment in industrial time-sensitive networks. The device includes: a memory, a processor, and a joint optimization program for task scheduling and resource deployment stored in the memory. The processor is used to run the joint optimization program for task scheduling and resource deployment. The computer program is configured to implement the steps of the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks as described above.

[0016] This invention constructs an industrial time-sensitive network scheduling model with multiple industrial production cycles and an intelligent agent interaction environment. The intelligent agent is configured to obtain a global embedded state through graph neural network encoding in each production cycle, and then execute equipment deployment and task scheduling decisions sequentially through a policy network. After the cycle ends, the reward value is calculated to update the policy network, and the full-cycle joint optimization is completed iteratively. This enables dynamic coordination of equipment reuse and task scheduling during multiple production cycles, improving task completion rate and network resource utilization efficiency. Thus, while ensuring that all critical communication flows are delivered on time, it achieves cross-cycle optimization of equipment deployment and task scheduling, enabling scientific decision-making at the beginning of each production cycle. This overcomes the limitations of traditional single-cycle scheduling, improves the overall efficiency of industrial production and network resource utilization over a longer time span, effectively improves production efficiency, and reduces costs. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of the task scheduling and resource deployment joint optimization device in the industrial time-sensitive network of the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the first embodiment of the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks according to the present invention. Figure 3 This is a flowchart illustrating the second embodiment of the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks according to the present invention. Figure 4 This is a structural block diagram of the first embodiment of the joint optimization device for task scheduling and resource deployment in industrial time-sensitive networks according to the present invention.

[0019] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0020] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0021] Reference Figure 1 , Figure 1 This is a schematic diagram of the device structure for joint optimization of task scheduling and resource deployment in an industrial time-sensitive network, which is the hardware operating environment involved in the embodiments of the present invention.

[0022] like Figure 1As shown, the task scheduling and resource deployment co-optimization device in this industrial time-sensitive network may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; the user interface 1003 may also include standard wired or wireless interfaces. The network interface 1004 may optionally include standard wired or wireless interfaces (such as Wireless-Fidelity (Wi-Fi) interfaces). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0023] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the joint optimization device for task scheduling and resource deployment in industrial time-sensitive networks. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0024] like Figure 1 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a task scheduling and resource deployment joint optimization program.

[0025] exist Figure 1 In the task scheduling and resource deployment co-optimization device for industrial time-sensitive networks shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the task scheduling and resource deployment co-optimization device for industrial time-sensitive networks of the present invention can be set in the task scheduling and resource deployment co-optimization device for industrial time-sensitive networks. The task scheduling and resource deployment co-optimization device for industrial time-sensitive networks calls the task scheduling and resource deployment co-optimization program stored in the memory 1005 through the processor 1001 and executes the task scheduling and resource deployment co-optimization method for industrial time-sensitive networks provided in the embodiments of the present invention.

[0026] This invention provides a joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks according to the present invention.

[0027] In this embodiment, the joint optimization method for task scheduling and resource deployment in the industrial time-sensitive network includes the following steps: Step S1: Construct an industrial time-sensitive network scheduling model that includes multiple industrial production cycles, and configure an intelligent agent interaction environment based on the industrial time-sensitive network scheduling model.

[0028] It should be noted that this embodiment applies to the joint optimization of scheduling and deployment in dynamic industrial time-sensitive networks with multiple production cycles. The joint optimization of scheduling and deployment in industrial time-sensitive networks is a complex combinatorial optimization problem: even in a simplified scenario with fixed equipment and only scheduling flows, the problem has been proven to be NP-hard (Non-deterministic Polynomial-Hard), and its search space expands exponentially with the network size and the number of production cycles. Traditional heuristic algorithms or static optimization methods struggle to obtain high-quality solutions in a timely manner under large-scale, multi-cycle scenarios. Therefore, this embodiment aims to effectively coordinate equipment deployment and task scheduling across multiple cycles using intelligent optimization algorithms, while ensuring the real-time constraints of critical flows, thereby improving production efficiency and reducing costs.

[0029] It should be understood that the executing entity of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a terminal electronic device capable of performing the above functions. The following description uses a task scheduling and resource deployment joint optimization device in an industrial time-sensitive network (hereinafter referred to as the optimization device) as an example to illustrate this embodiment and the following embodiments.

[0030] It should be noted that Industrial Time-Sensitive Networking (TSN) refers to industrial-specific communication networks built on the IEEE 802.1 series of standards. Its core features are microsecond-level time synchronization, deterministic low latency, low jitter, and high reliability. It is specifically adapted to scenarios in industrial automation and intelligent manufacturing where there are stringent requirements for the real-time performance and stability of task transmission, and serves as the core carrier for industrial control commands and production data transmission.

[0031] The industrial production cycle refers to the smallest time unit for completing a standardized operation process in industrial production. It covers the entire process of task issuance, execution, and data feedback. In this embodiment, it is the smallest closed-loop unit for task scheduling, resource deployment, and strategy updates. Its duration is set according to the requirements of industrial production processes.

[0032] It should be noted that the industrial time-sensitive network scheduling model can be a low-level model that mathematically and logically abstracts all elements of task scheduling and resource deployment in industrial TSN scenarios, including multi-production cycle time sequence rules, task attributes, network topology constraints, node resource limitations, etc.

[0033] The agent interaction environment can be a closed-loop interaction carrier built on a reinforcement learning framework. It is responsible for transmitting the observable state of the current network and task to the agent, executing the agent's decision instructions, updating the environmental state and feeding back reward signals, so as to realize the agent's closed-loop cycle of "perception-decision-learning".

[0034] It should be noted that the policy network adopts an actor-critic architecture, which includes a device deployment actor network, a task scheduling actor network, and a critic network. The device deployment actor network and the task scheduling actor network are configured to update the device deployment scheme and the task scheduling scheme based on the global embedded state, and the critic network is configured to guide the policy gradient update of the actor network.

[0035] The device deployment actor network is a sub-network in the policy network responsible for generating device deployment decisions. Its inputs are the global embedding state and task requirements, and its output is the probability distribution of deployment actions, which is used to generate device deployment schemes.

[0036] The task scheduling actor network is a sub-network in the policy network responsible for generating task scheduling decisions. Its inputs are the global embedding state, task requirements, and equipment deployment schemes, and its output is the probability distribution of scheduling actions, which is used to generate task scheduling schemes.

[0037] The critic network is a subnetwork in the policy network responsible for evaluating the value of decision actions. Its inputs are the global embedded state and the actor network's decision, and its output is a value estimate of the state-action pair, which is used to guide the actor network's policy update and reduce the policy optimization variance.

[0038] It should be noted that a task set refers to the collection of all industrial tasks that need to be transmitted and executed in the TSN within a single industrial production cycle. Each task has clear attributes such as cycle, deadline, data volume, priority, and source / destination node, and is divided into types such as time-sensitive control tasks and production data acquisition tasks.

[0039] An industrial time-sensitive network topology can be a structured description of the communication link connections between all network nodes (switches, terminals, controllers, etc.) in a TSN.

[0040] The device deployment status of terminal nodes can be the resource configuration and operation status of each terminal node and switch node in TSN, including node start / stop status, computing power / storage / communication resource occupancy and availability, functional module deployment status, port and queue configuration, and other information.

[0041] In some embodiments, the optimization device models the scheduling process of an industrial time-sensitive network (ITN) over multiple consecutive industrial production cycles as a multi-stage decision-making process to determine the industrial production cycle sequence; extracts parameters of the task set corresponding to each industrial production cycle in the industrial production cycle sequence, including the equipment type and quantity required for the task and the parameters of the time-sensitive communication stream generated by the task; obtains the topology information of the ITN to determine the set of terminal nodes, the set of switching nodes, and the link connection relationships between nodes; obtains the initial equipment deployment status information of each terminal node; integrates the industrial production cycle sequence, task set parameters, topology information, and initial equipment deployment status information to complete the construction of the ITN scheduling model; and configures the environmental rules for agent interaction based on the state space, action space, state transition rules, and optimization objectives defined in the ITN scheduling model to complete the configuration of the agent interaction environment.

[0042] In practical implementation, a multi-cycle scheduling model is constructed to optimize equipment. The scheduling process of the industrial time-sensitive network over multiple consecutive production cycles is modeled as a multi-stage decision-making process. The production cycle sequence is defined. For each production cycle, the system acquires the task set and its parameters, including the type and quantity of equipment required for each task, the set of time-sensitive communication flows generated by the task, and the cycle, deadline, and data length of each flow. Simultaneously, it acquires the industrial time-sensitive network topology, including the set of terminal nodes and time-sensitive network switching nodes, as well as the link connections, and initializes the available deployed equipment status on each terminal node. The initial state includes: the network topology and initial equipment deployment, as well as the task requirements for the current cycle.

[0043] Step S11: At the beginning of each industrial production cycle, obtain the task requirements of the current production cycle, and encode the industrial time-sensitive network topology and the equipment deployment status of each terminal node through a graph neural network to obtain the global embedding state.

[0044] It should be noted that task requirements can be the attributes and execution constraints of all industrial tasks that need to be completed within the current production cycle, including the number of tasks, type, source / destination nodes, data volume, latency requirements, jitter limit, reliability standards, etc.

[0045] The global embedded state can be a unified low-dimensional vector output after being encoded by a graph neural network. It fully includes the global topology features of the TSN, the deployment status features of terminal node devices, and the link constraint features between nodes. It comprehensively represents the global state of the network and resources at the initial moment of the current cycle and provides input for the policy network.

[0046] In practical implementation, for any production cycle, the current system state needs to be represented before making a decision. This invention uses a Graph Neural Network (GNN) to represent the topology and state of the industrial time-sensitive network. Node features are extracted from the network topology: for each terminal node, its deployment cost and the current and historical utilization statistics of the links connected to that node are obtained and combined to form the node's feature vector. The network is represented as a graph. , where the node set Includes terminal nodes and switching nodes, edge set For link connections, the graph neural network takes the feature vectors of all terminal nodes and the graph structure as input, and calculates the embedding representation of each node through a message passing mechanism. .

[0047] It is understood that this embodiment uses graph neural networks (GNNs) to specifically process the graph structure data of TSNs, solving the problem that traditional encoding methods cannot effectively mine the relationships between nodes and the coupling features of resources and topological constraints; the global embedded state realizes the unified and vectorized representation of discrete information, providing complete and low-redundancy global state input for subsequent policy networks, improving the adaptability of decisions to the global state of the network, and avoiding global resource imbalance caused by local decisions.

[0048] Step S12: Based on the task requirements and the global embedding state, the device deployment decision and task scheduling decision are executed sequentially through the policy network to obtain the device deployment scheme and task scheduling scheme for the current production cycle.

[0049] It should be noted that device deployment decisions can be actions taken by an intelligent agent to configure, reserve, and deploy TSN node resources based on the current periodic task requirements and network status. These actions include reserving computing / storage resources, starting and stopping functional modules, configuring switch queues and time slots, etc., which determine the spatial distribution of available network resources.

[0050] Task scheduling decisions can be actions taken by an intelligent agent based on the device deployment plan and the global state to plan the transmission path, allocate time slots, and configure priorities for each time-sensitive task. The core is to ensure that tasks meet constraints such as latency and jitter, while improving resource utilization.

[0051] The equipment deployment scheme can be a set of resource configuration instructions output by the equipment deployment actor network, which clarifies the resource reservation rules and availability boundaries of each node.

[0052] The task scheduling scheme can be a set of scheduling instructions output by the task scheduler network, which specifies the transmission path, time slot allocation, priority and other rules for each task.

[0053] In the specific implementation, at the beginning of each production cycle, the current state vector is obtained by merging the task requirements and the global embedded state. First, equipment deployment decisions are made. These decisions aim to determine which terminal nodes need to be equipped with new equipment, which idle equipment needs to be removed, or which nodes need to remain unchanged, in order to meet the equipment requirements of the current task while taking into account the optimal cost.

[0054] After equipment deployment is completed, the task scheduling phase of the production cycle begins. At this point, the available network equipment configuration for this cycle has been determined, and transmission scheduling needs to be allocated in a time-sensitive network for the communication flows of all tasks in this cycle, so that the data frames of each flow can be delivered to the destination on time.

[0055] Furthermore, the specific process of making equipment deployment decisions and generating equipment deployment plans includes: Step S1201: Extract device feature information from the global embedding state and input the device feature information into the device deployment actor network. The device feature information includes device cost information and task device requirement information. Step S1202: Calculate the input device feature information through the fully connected layer of the device deployment actor network to obtain the action preference value of each terminal node corresponding to the optional actions. The optional actions include not performing the operation, installing the device, and removing the deployed device. Step S1203: Normalize the action preference value using a normalized exponential function to obtain the strategy probability distribution of the corresponding action for each terminal node; Step S1204: Select the corresponding device operation action for each terminal node according to the probability distribution of the strategy, execute the device operation action and synchronously update the device status and cost statistics information of each terminal node; Step S1205: When the equipment requirements of all tasks are met, or all terminal nodes have completed the action selection, generate the equipment deployment plan for the current production cycle.

[0056] It should be noted that the device feature information can be a set of special features that are directly related to device deployment decisions and are extracted and filtered from the global embedded state. It is the exclusive input of the device deployment actor network and includes device cost information and task device requirement information, taking into account both resource constraints and business needs. Equipment cost information can be quantitative cost parameters related to the installation, removal, operation and maintenance of terminal node equipment, including hardware deployment costs, computing power occupation costs, bandwidth consumption costs, energy consumption costs, etc., which are used to constrain the economics of deployment decisions; Task equipment requirements information can be the hard requirements of various tasks in the current production cycle for the equipment type, functional modules, and resource specifications of terminal nodes. These are constraints used to ensure that tasks can be executed normally.

[0057] It should be noted that the action preference value can be a quantitative value output by the fully connected layer that represents the quality of the available actions for each terminal node. The value directly reflects the degree to which the action is adapted to the current network state and task requirements, and is used to compare the quality of actions.

[0058] In practice, the specific equipment deployment decision-making process is as follows: The portion of the state vector related to equipment cost and demand is input into the equipment deployment decision module. This module executes the decisions for this phase, outputting specific deployment actions based on a policy network.

[0059] The device deployment decision module takes the normalized device cost and other characteristics of each terminal node as input, and calculates a set of action preference values ​​for each terminal node through a two-layer fully connected neural network. The optional actions for each terminal node include: not performing an operation, installing the device, and removing the deployed device type.

[0060] These preference values ​​are normalized using the softmax function to obtain the policy probability distribution. The sampling process for device deployment decision-making is as follows: For each terminal node, based on... Select a device action and execute it. For example, if "Install device at node A" is selected, the device status of node A will be updated accordingly, and the incurred costs will be accumulated. The above strategy can be used to perform installation / removal operations on multiple nodes sequentially. When the device deployment decision module outputs the "No Operation" option for a certain node or when all task device requirements have been met, the decision-making process for this stage ends, no new device adjustment actions are executed, and the final terminal node device configuration status for the current production cycle is obtained. .

[0061] Furthermore, the task scheduling decision-making process includes path selection for executing the task scheduling decision, which includes: Step S1211: Based on the equipment deployment plan of the current production cycle, update the scheduling status of the industrial time-sensitive network and obtain the time-sensitive communication stream set corresponding to all tasks to be scheduled in the current production cycle.

[0062] It should be noted that time-sensitive communication streams can be data streams transformed from industrial time-sensitive tasks, with strict constraints on latency, jitter, and reliability. They are the core objects of task scheduling, and each stream corresponds to the transmission requirements of a single industrial task.

[0063] The set of time-sensitive communication flows corresponding to the tasks to be scheduled can be the sum of all time-sensitive communication flows that have not completed transmission path planning and need to be scheduled within the current production cycle, and is the complete set of processing objects for path selection.

[0064] Step S1212: For a single time-sensitive communication flow to be scheduled, extract the attribute parameters of the time-sensitive communication flow and the corresponding candidate transmission path set, wherein the candidate transmission path set contains at least one candidate transmission path.

[0065] It should be noted that the time-sensitive communication flow attribute parameters are a set of parameters that characterize the core transmission requirements of a single time-sensitive communication flow. They include core constraint parameters such as source node, destination node, required bandwidth, maximum allowable end-to-end delay, delay jitter limit, transmission priority, data frame length, and transmission period. These parameters are the core basis for path selection and decision-making.

[0066] Candidate transmission paths refer to reachable communication paths from the source node to the destination node that meet the basic transmission constraints of time-sensitive communication flows, based on the current TSN scheduling state, and are free from loops and resource overload.

[0067] Step S1213: Input the link feature sequence of each candidate transmission path into the stream encoder to extract the path-level embedding vector corresponding to each candidate transmission path.

[0068] It should be noted that the link feature sequence can be a time-series feature sequence formed by arranging the core features of each link segment in a single candidate transmission path in the transmission order, including features such as single link bandwidth, latency, jitter, load rate, and packet loss rate.

[0069] A stream encoder is a neural network module specifically designed to process the characteristics of time-series links, and is responsible for mining the overall transmission performance characteristics of the path.

[0070] The path-level embedding vector can be a dense vector generated after being encoded by a stream encoder. It integrates the overall performance characteristics of the entire link of a single candidate path to characterize the transmission adaptation capability of the path.

[0071] Step S1214: Fuse all path-level embedding vectors with the attribute parameters of the time-sensitive communication stream to obtain the task-level embedding vector corresponding to the time-sensitive communication stream.

[0072] It should be noted that the task-level embedding vector can be a composite feature vector that integrates all the constraints of a single time-sensitive communication flow with the overall performance of all candidate paths. It is the direct input to the task scheduling actor network and represents the core decision-making elements of the current scheduling task.

[0073] Step S1215: Input the task-level embedding vector into the task scheduling actor network, calculate the score value corresponding to each candidate transmission path, and process the score value through the normalized exponential function to obtain the selection probability distribution of each candidate transmission path.

[0074] It should be noted that the path score is a numerical value output by the task scheduler network that quantifies the degree to which a single candidate path is suitable for the corresponding time-sensitive communication flow. The higher the score, the better the path is suited to the task transmission requirements.

[0075] Step S1216: Select a transmission path for the time-sensitive communication stream according to the selection probability distribution until all time-sensitive communication streams to be scheduled have completed the selection of transmission paths.

[0076] In practice, the specific task scheduling decision-making process is as follows: Initialize the scheduling state based on the current updated network state.

[0077] The task scheduling actor network makes scheduling decisions for each task to be scheduled individually. Since each task may consist of multiple CT streams, and these streams usually have a pre-planned set of candidate transmission paths, the task scheduling actor network's decision is to select an actual path for each stream from the candidate paths and attempt to allocate transmission time slots for all frames of that stream on that path.

[0078] For the currently pending tasks The task scheduling actor network collects each of its streams. The corresponding candidate path list and the attributes of the flow are used. Each candidate path is represented as a feature sequence and input into a flow encoder sub-network. The flow encoder is preferably implemented as a single-layer gated recurrent unit (GRU) combined with a multi-head self-attention mechanism: the GRU is used to sequentially process the link feature sequences on the path to extract path-level representations, and the self-attention mechanism is used to synthesize the feature correlations between different paths.

[0079] FlowEncoder for tasks All Each candidate path outputs a series of hidden vectors, which are then linearly transformed and fused with the request parameters of the task. Finally, the average of these vectors is taken to obtain the task-level embedding for the task. , representing the task The feature representation required for scheduling decisions under the current network state. Next, we will... The data is fed into the task scheduling actor network, and after two fully connected layers, the dimension is obtained. The output vector corresponds to the score of each candidate path, and is transformed into a probability distribution using a softmax function. Based on this distribution, the path with the highest probability is selected as the transmission path for the current stream of the task.

[0080] For the task All communication flows are routed independently and identically distributed by the task scheduling actor network. Once all tasks to be scheduled have passed through the path selected by the above decision-making process, the communication flow time slot allocation and scheduling process begins.

[0081] Furthermore, the specific process of generating the task scheduling scheme includes: Step S1221: According to the preset scheduling order, traverse all time-sensitive communication flows of the selected transmission path and determine all links and corresponding node ports on the transmission path corresponding to a single time-sensitive communication flow.

[0082] It should be noted that the preset scheduling order can be a scheduling sorting rule based on the priority of industrial TSN services, such as prioritizing high-priority time-sensitive control flows and postponing ordinary production data flows, to avoid failure of critical control task time slots.

[0083] The transmission path link can be a physical communication link connecting adjacent nodes in the previously selected transmission path. It is the carrier for transmitting time-sensitive communication stream data frames, and each link corresponds to a unique bandwidth and delay constraint.

[0084] A node port refers to the physical interface on a terminal node or switching node used for data transmission and reception. It is the core unit for time slot allocation and link connection, and each port has an independent time slot scheduling table and bandwidth quota.

[0085] Step S1222: Along the transmission path of the time-sensitive communication stream, at the output port of the source node, allocate the earliest idle time slot that meets the transmission time constraint for the data frame of the time-sensitive communication stream; Step S1223: Based on the allocated transmission time slots, link propagation delay, and node processing delay, determine the time for the data frame to arrive at the next hop node, and allocate an idle time slot that meets the connection requirements for the data frame at the output port of the corresponding node; Step S1224: Complete the time slot allocation for the entire path of the time-sensitive communication flow hop by hop. When the time slot allocation for the entire path is completed in accordance with the constraints, mark the time-sensitive communication flow as successfully scheduled and synchronously update the time slot occupancy status of the corresponding link and port. Step S1225: When all time-sensitive communication streams have completed their time slot allocation attempts, generate the task scheduling scheme for the current production cycle.

[0086] It is understood that this embodiment follows the following constraints when allocating time slots: Sending time constraints: for communication streams In its first The data frame generated in the next cycle, if the frame release time is Therefore, it is essential to ensure that the first hop of the transmission occurs at the source node (or the task triggering node). That is, a frame cannot be sent immediately after it is generated; it must wait until the moment it is generated before it can enter the sending queue. Jump-by-jump propagation constraint: If the flow The current link on the path of a certain data frame At the moment Once transmission is complete, the frame reaches the target node. The time should take into account the link propagation delay. and node processing latency Therefore, in the subsequent next-hop link The sending on should meet the following requirements ; End-to-end latency constraints: for each communication stream The specification includes a maximum permissible end-to-end delay. The requirement is that the total time taken for any data frame in the stream to travel from the source to the destination node does not exceed [a certain threshold]. ; Collision-free transmission constraint: For any two different communication flows that pass through the same physical link or switch port within the same production cycle... and The scheduled sending time intervals must not overlap or conflict.

[0087] This embodiment employs a heuristic greedy first-fit scheduling algorithm to allocate time slots for communication flows. The specific process is as follows: Each communication flow requiring scheduling is traversed in a certain order, and transmission time slots are reserved sequentially along the selected transmission path of that flow. At the source node interface, the earliest available time slot, no earlier than the frame generation time and with a length no less than the frame transmission time, is selected and allocated to the first transmission of the frame. Then, based on the propagation delay, the arrival time of the frame at the next switching node is calculated. The next available time slot that can be connected for transmission is found at the output port of that switching node and assigned to the frame. This process continues hop-by-hop towards the destination node. If no available time slot that satisfies both conflict-free scheduling and arrival within the time limit is found at a certain hop's switching port, the communication flow is determined to be unschedulable on the current path, and the scheduling algorithm marks the flow as unscheduled. The next communication flow is processed until all flows have been attempted to be allocated. After the above process, all communication flows for all tasks within this cycle either successfully have their transmission plans allocated or fail to be scheduled as required due to network resource constraints. For successfully scheduled flows, their associated tasks are marked as partially completed or completed. At the end of the task scheduling phase, the scheduling results for that period are generated, including the occupied transmission time slots on each link and a list of completed tasks.

[0088] Step S13: At the end of each industrial production cycle, calculate the instant reward value based on the task completion status and equipment deployment status of the current production cycle, and perform policy gradient update on the policy network based on the instant reward value.

[0089] It should be noted that the task completion status refers to the execution effect of all tasks within the current period, including core indicators such as whether the latency meets the standard, whether the jitter is compliant, transmission success rate, packet loss rate, and deadline timeout rate.

[0090] Equipment deployment status refers to the execution effect of resource deployment within the current period, including node resource utilization, link bandwidth utilization, resource distribution balance, and resource idle / overload status.

[0091] The immediate reward value is a scalar feedback signal that evaluates the quality of the agent's decision in the current cycle. It is calculated by the reward function in combination with the task completion and equipment deployment status. Positive rewards correspond to good decisions, and negative rewards correspond to bad decisions.

[0092] In this implementation, after a production cycle is completed, this embodiment calculates the reward value for reinforcement learning based on the scheduling results of that cycle to evaluate the merits of the current decision-making scheme. The reward design directly relates to the agent's learning direction; therefore, a mechanism combining final rewards and phased rewards is adopted.

[0093] Furthermore, in order to accelerate the learning convergence speed and avoid training stagnation caused by reward sparsity, step S13 above may include: Step S131: Count the number of time-sensitive communication flows that are successfully scheduled in the current production cycle, and add the corresponding first positive reward value to each successfully scheduled time-sensitive communication flow.

[0094] It should be noted that a successfully scheduled time-sensitive communication flow can be a time-sensitive communication flow that meets the TSN deterministic transmission requirements, completes the full path time slot compliant allocation, has no link conflicts, no timeout packet loss, and meets the end-to-end latency and latency jitter standards, and can completely execute the data transmission task.

[0095] The first positive reward value can be a unit positive incentive value set for a single time-sensitive communication flow that has been successfully scheduled. It is a fixed small positive reward used to encourage agents to ensure the basic transmission compliance of a single time-sensitive flow, which is in line with the deterministic transmission requirements of the underlying TSN.

[0096] Step S132: Count the number of tasks that have been successfully scheduled for all associated time-sensitive communication streams in the current production cycle, and add the corresponding second positive reward value to each completed task.

[0097] It should be noted that the associated time-sensitive communication flow can be a set of time-sensitive communication flows derived from a single complete industrial production task. A single task can correspond to multiple associated flows. Only when all associated flows are successfully scheduled can the task loop be considered complete.

[0098] A completed task refers to an industrial task in which all associated time-sensitive communication flows are successfully scheduled within the current period, there are no transmission failures in any link, and the business execution goals are fully achieved. This is different from an incomplete / partially completed task where some flows fail to be scheduled.

[0099] The second positive reward value can be a higher-order unit reward set for a single complete closed-loop task. The reward amount is significantly higher than the first positive reward value. It belongs to the task-level incentive item (additional reward) and is used to guide the agent to pursue the global completion of the task, rather than just fragmented completion of single flow scheduling.

[0100] Step S133: Statistically analyze the equipment installation and dismantling actions performed during the current production cycle, and calculate the cost penalty value corresponding to the equipment operation based on the equipment purchase cost, reconfiguration cost and recovery residual value.

[0101] It should be noted that the purchase cost refers to the one-time total cost of hardware procurement, on-site deployment, computing power adaptation, debugging and calibration incurred by adding and installing terminal equipment and functional modules; Reconfiguration costs refer to the labor, resource losses, and downtime indirect costs incurred by equipment dismantling, node parameter readjustment, and link replanning; residual value refers to the recyclable value of dismantled equipment, which can offset part of the dismantling costs. Cost penalty value: A negative penalty value set for irrational and high-cost equipment operation. The value is negative and is used to constrain the agent from blindly deploying and frequently disassembling equipment, so as to balance scheduling performance and deployment economy.

[0102] Step S134: Calculate the average utilization rate of all links in the industrial time-sensitive network during the current production cycle, and calculate the corresponding load balancing penalty value based on the average utilization rate.

[0103] It should be noted that the average link utilization rate refers to the average ratio of the actual bandwidth usage of all communication links in the TSN network to the total bandwidth capacity of the links within the current period, reflecting the overall utilization of link resources across the entire network. The load balancing penalty value can be a negative penalty value set for link load imbalance, excessive or low utilization. The value is negative, and the more severe the load imbalance, the larger the absolute value of the penalty. It is used to guide the agent to optimize path allocation and device deployment to achieve balanced utilization of network resources.

[0104] Step S135: Based on the first positive reward value, the second positive reward value, the cost penalty value, and the load balancing penalty value, calculate the instant reward value for the current production cycle.

[0105] In the specific implementation, the reward algorithm is as follows: At the end of the entire multi-cycle scheduling process, calculate the total benefit: let each task Successful completion will generate revenue. If the task is not completed, the reward is 0; the total task reward is obtained by summing the rewards of all tasks. ,in Instructions for task completion.

[0106] Then calculate the equipment cost expenditure: for each type of equipment. Statistics on the number of newly purchased devices throughout the entire multi-cycle period. Multiply by purchase cost Obtain the total purchase cost ; and the number of times the equipment is disassembled or reinstalled at the end of each cycle. Multiply by the unit reconfiguration cost And sum over all cycles, device types, and nodes. Obtain the total reconfiguration cost. The ultimate overall goal is to... maximize.

[0107] This embodiment incorporates a phased reward signal during training: a fixed small positive reward is given for each successfully scheduled communication stream. (First positive reward value); whenever all flows of a task are successfully scheduled and completed, an additional reward (i.e., second positive reward value) is given to that task. This ensures that the completion of each flow and each task contributes to the agent's immediate reward, avoiding sparse rewards that are only evaluated after all cycles have ended. At the same time, for tasks that are not successfully completed, neither the aforementioned positive reward nor additional penalties are obtained, incentivizing the agent to complete more tasks without being double-penalized for the failure of a few individual tasks.

[0108] Introduce cost penalties and coverage rewards during the equipment deployment phase: For each equipment installation action, immediately apply a negative reward (i.e., cost penalty value) equivalent to the purchase cost of the equipment to simulate expenditure; for each equipment removal, give a corresponding negative or zero reward based on its salvage value; at the same time, calculate the coverage based on the degree to which the equipment deployed in the current cycle meets the task requirements, and give a certain positive reward accordingly to encourage agents to meet the task resource requirements as much as possible while avoiding over-deployment.

[0109] At the end of the period, a load penalty (i.e., load balancing penalty value) is applied based on the average utilization level of all network links: the average utilization of all links is taken as the result. Set negative rewards ( (As weighting coefficients), guiding the agent to not only complete tasks but also balance network load and improve resource utilization. Furthermore, if all tasks submitted within the current period are successfully completed, a period completion reward is given as a signal for fully meeting real-time requirements. Through this multi-factor reward design, this embodiment ensures that the reinforcement learning agent receives denser and richer feedback signals during training, focusing on both long-term overall gains and short-term phased goals, thereby accelerating learning convergence and avoiding training stagnation caused by sparse rewards.

[0110] Furthermore, in order to improve the success rate of task scheduling, reduce equipment deployment costs, and balance network load, step S13 above also includes: Step S1301: Store the quadruple data corresponding to the current production cycle into the priority experience replay buffer. The quadruple data consists of the current state, the current action, the immediate reward value, and the state of the next cycle. Step S1302: When the amount of data in the experience playback buffer reaches a preset capacity threshold, or when the scheduling process of all industrial production cycles is completed, a preset number of sample data are extracted from the experience playback buffer according to priority. Step S1303: Based on the extracted sample data, calculate the time-series difference target value through the critic network, and update the parameters of the critic network with the goal of minimizing the mean square error; Step S1304: Based on the updated critic network, calculate the advantage function value corresponding to the sample data through the advantage function, and standardize the advantage function value; Step S1305: Based on the standardized advantage function value, perform policy gradient updates on the device deployment actor network and the task scheduling actor network respectively, and perform policy constraints based on entropy regularization constraints; Step S1306: Perform a soft update on the target network of the commentator network to complete the parameter iteration of the policy network.

[0111] In the specific implementation, the strategy update method is as follows: At the start of training, the GNN encoder parameters, device-deployed actor network, multiple task-scheduled actor network, critic network, and target network are initialized. A priority experience replay buffer is used to store state, action, and reward transition samples generated by the agent's interaction with the environment, and different priorities are assigned to the samples based on their TD error so that important samples can be sampled more frequently during training.

[0112] During training, the agent repeatedly interacts with the simulation environment according to steps S1201-S1306: after completing the decision-making and scheduling of each cycle, it receives an immediate reward and enters the state of the next cycle. The quadruple is recorded in the experience pool, where... Indicates the current state. Indicates the current action. Indicates an immediate reward. Indicates the state of the next cycle.

[0113] Once a complete multi-cycle scheduling process ends or the experience pool reaches a certain capacity, the agent's policy is updated.

[0114] First, a batch of transition samples are randomly selected from the experience pool according to sample priority, and the TD target value for each sample is calculated using the critic network. ,in For the target Critic network, The discount factor is used to update the critic network parameters by minimizing the mean squared error, making them approximate the new value estimate.

[0115] Calculate the advantage function for each sample using the updated critic network. Then, standardization is performed. Next, policy gradient updates are applied to the actor network: for both the device-deployed actor network and the per-task-scheduled actor network, the aforementioned advantages are leveraged to estimate... Weighted log probabilities of their respective outputs The expected value, and add an entropy regularization term. Its coefficient The Actor network parameters are updated via gradient ascent, gradually decreasing as training progresses. This increases the probability of actions that bring higher value under the current strategy, while the entropy term ensures that the strategy does not converge too quickly into a greedy approach and lose its exploration ability.

[0116] The target network of the critic network undergoes soft updates, slowly aligning with the latest critic parameters to stabilize the training process. This training process is repeated several times, continuously optimizing the agent's policy network parameters, and gradually teaching the device deployment and task scheduling networks better device deployment and task scheduling strategies.

[0117] Step S14: At the start of the next industrial production cycle, return to step S11 based on the updated policy network until all industrial production cycles are completed, thus completing the joint optimization of task scheduling and resource deployment for the industrial time-sensitive network.

[0118] Understandably, this embodiment employs an Actor-Critic deep reinforcement learning algorithm to train the agent to complete the aforementioned decision-making task. The Actor-Critic method includes a policy network (actor) for outputting action decisions and a value network (critic) for evaluating the long-term value of a given state-action relationship. In this embodiment, the agent comprises two types of Actor networks: DeviceActor (device deployment actor network) responsible for device deployment decisions and TaskActor (task scheduling actor network) responsible for task scheduling decisions; and a centralized Critic network (critic network) evaluating the overall value of the decision. To improve training efficiency, the graph neural network encoder can fix parameters during training, recalculating nodes and global embeddings only when the environment changes, thereby reducing redundant computation.

[0119] In practice, after the agent completes the policy update of the previous cycle, it automatically enters the start time of the next cycle, loads the updated policy network weights, and initializes the decision environment for the current cycle.

[0120] Repeat steps 11 (state acquisition and GNN encoding), 12 (joint decision generation), and 13 (reward calculation and policy update) to complete the closed-loop optimization of the current cycle.

[0121] After each cycle ends, it is determined whether all preset production cycles have been completed; if not, the iteration continues; if all are completed, the loop terminates, and the final optimized strategy network is output for online decision-making in industrial TSN, thus completing joint optimization.

[0122] It is understandable that this embodiment achieves continuous evolution of decision-making strategies through multi-cycle iteration, which can adapt to the dynamic changes of tasks and network states in industrial scenarios and gradually converge to the globally optimal strategy. Taking the production cycle as the iteration unit, it fits the actual time sequence of industrial production, and the optimized strategy has strong engineering feasibility. The closed-loop iteration of the entire process ensures the implementation of the joint optimization goal and improves the practicality and adaptability of the solution.

[0123] It should be noted that this embodiment transforms the multi-period network planning problem of industrial time-sensitive networks into hierarchical decision strategy learning. By combining graph neural networks and deep reinforcement learning, it achieves end-to-end joint optimization of equipment deployment and task scheduling, which can overcome the limitations of traditional methods and significantly improve scheduling performance.

[0124] This embodiment comprehensively considers equipment purchase, reconfiguration costs, and task completion benefits across multiple production cycles, avoiding the situation in the prior art where each cycle operates independently and equipment is purchased repeatedly, thus achieving full reuse of equipment and overall cost reduction; and performs fine time-slice allocation of communication flow through the task scheduling phase.

[0125] This embodiment ensures that all critical traffic meets strict real-time constraints, avoids network congestion and packet collisions, and maximizes network resource utilization while ensuring system security. It uses graph neural networks to extract features from complex topologies and states, and the policy decisions can adapt to industrial networks of different sizes and structures.

[0126] In the specific implementation, after sufficient training, stable and efficient DeviceActor and TaskActor strategies are obtained, which can be used for online scheduling of actual industrial time-sensitive networks. For a given industrial time-sensitive network deployment and task plans for several future production cycles, the optimal or near-optimal equipment deployment and task scheduling schemes for each cycle can be obtained by using the trained agent model and executing steps 11 to 13. When all cycles are processed, the joint optimization of task scheduling and resource deployment over the entire multi-cycle scope is completed. The method of this embodiment can also be used in online rolling optimization scenarios: after each cycle is completed in actual operation, the trained strategy is called again to execute the decision for the next cycle based on the new state, thereby dynamically adapting to changing task requirements and achieving continuous optimization.

[0127] This embodiment adopts a dual-agent architecture of DeviceActor and TaskActor, decoupling cross-cycle device resource planning from intra-cycle task scheduling, which greatly reduces the complexity of a single decision network, avoids input feature incompatibility and gradient conflict problems, and makes the training process more stable and reliable. At the same time, a course learning strategy is introduced to gradually increase the task and network scale of the training environment, enabling the agent to learn step by step, transitioning from small-scale problems to large-scale complex problems, effectively solving the problem of slow convergence and poor performance of direct training in large-scale scenarios.

[0128] This embodiment constructs an industrial time-sensitive network scheduling model with multiple industrial production cycles and an intelligent agent interaction environment. The intelligent agent is configured to obtain a global embedded state through graph neural network encoding in each production cycle, and then execute equipment deployment and task scheduling decisions sequentially through a policy network. After the cycle ends, the reward value is calculated to update the policy network, and the full-cycle joint optimization is completed iteratively. This enables dynamic coordination of equipment reuse and task scheduling during multiple production cycles, improving task completion rate and network resource utilization efficiency. Thus, while ensuring that all critical communication flows are delivered on time, it achieves cross-cycle optimization of equipment deployment and task scheduling, enabling scientific decision-making at the beginning of each production cycle. This overcomes the limitations of traditional single-cycle scheduling, improves the overall efficiency of industrial production and network resource utilization over a longer time span, effectively improves production efficiency, and reduces costs.

[0129] refer to Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks according to the present invention.

[0130] Based on the first embodiment described above, in this embodiment, step S11 further includes: Step S111: Extract the feature information of each node in the industrial time-sensitive network and generate the feature vector corresponding to each node. The nodes include terminal nodes and switching nodes.

[0131] It should be noted that the feature information of a node can be the unique static attributes and dynamic operating status data of terminal nodes and switching nodes in an industrial time-sensitive network. These are the core parameters that characterize the node's resource capabilities and operating status, and are divided into two categories: static fixed features and dynamic real-time features.

[0132] A node feature vector is a one-dimensional dense vector formed by standardizing and normalizing the scattered feature information of a node. It is a standard input format that graph neural networks can directly process.

[0133] In some embodiments, all nodes in the network are first classified into two categories: terminal nodes (industrial sensors, actuators, controllers, etc.) and switching nodes (TSN switches, core gateways, etc.). Then, features are extracted hierarchically: for terminal nodes, features such as computing power margin, storage occupancy rate, deployed functional module type, communication port status, and real-time load rate are extracted; for switching nodes, features such as port number, time slot resource margin, queue configuration parameters, link forwarding latency limit, bandwidth carrying capacity, and cache capacity are extracted. After extraction, all feature data are normalized and discretely encoded to eliminate differences in units. The feature vectors are then concatenated according to fixed dimensions to generate a unique feature vector for each node, ensuring that the feature vectors of all nodes have a unified dimension.

[0134] Step S112: Construct a network topology diagram based on the topology of the industrial time-sensitive network.

[0135] It should be noted that the network topology graph includes a node set and an edge set. The node set includes all terminal nodes and switching nodes, and the edge set includes the link connections between nodes.

[0136] In some embodiments, the physical cabling and logical networking configuration files of the industrial TSN are retrieved, the connection correspondence of all nodes is sorted out, and a directed topology graph is constructed (adapting to the unidirectional data forwarding characteristics of TSN): all terminal nodes and switching nodes are uniformly included in the node set and assigned a unique identifier; the connection relationships between nodes that have direct communication links and can forward data are organized into an edge set, and the link direction is marked on each edge. At the same time, redundant indirect connections are simplified, and only core direct links are retained to form a standardized topology graph without loops or redundancy, which is adapted to the graph data input requirements of graph neural networks.

[0137] Step S113: Input the feature vectors of each node and the network topology graph into the graph neural network, and calculate the node embedding vectors corresponding to each node through the message passing mechanism.

[0138] It should be noted that the message passing mechanism refers to the process by which each node transmits its own characteristic information to neighboring nodes, while simultaneously receiving and aggregating characteristic messages from neighboring nodes, and iteratively updating its own characteristics. This process can deeply explore the relationships and dependencies between nodes.

[0139] Node embedding vectors can be node feature vectors output after message passing and feature aggregation in graph neural networks. They integrate the node's own features with the related features of neighboring nodes and the topological structure, and are more globally relevant than the original feature vectors.

[0140] Step S114: Calculate the mean of all node embedding vectors to obtain the network embedding vector.

[0141] It should be noted that the network embedding vector can be a dense vector generated by aggregating the global mean of the embedding vectors of all nodes in the entire network. This eliminates the local differences of individual nodes, extracts the global resources and topological common features of the entire industrial TSN, and represents the overall state of the network.

[0142] Step S115: Extract and summarize the features of the task requirements for the current production cycle to obtain the overall task requirement feature vector.

[0143] It should be noted that the overall task requirement feature vector can be a vector generated by extracting, organizing, and summarizing the core requirement parameters of all tasks in the current industrial production cycle. It integrates core requirements such as task quantity, latency constraints, priority, data volume, and source / destination node distribution to represent the overall task load and business requirements within the cycle.

[0144] Step S116: Merge the network embedding vector with the overall task requirement feature vector to generate a global embedding state.

[0145] In the implementation, a global self-attention graph neural network is used, calculating the correlation weight for any two nodes to fuse global topology information in each iteration. After several layers of graph neural network computation, a set of node embeddings containing topology and device state information is obtained. The average of this set yields the global network embedding vector. This represents the comprehensive characteristics of the current time-sensitive network and its deployment status.

[0146] Organize the requirements of the current cycle task set: extract parameters such as the required equipment resources, the number of communication streams contained in the task, and the common duration of these streams for each task; concatenate or summarize all task requirement vectors to obtain a representation of the overall task requirement characteristics. Then embed the aforementioned global network. With task requirements and characteristics Combined with other possible state variables, it forms the state vector at the current decision moment. This state vector reflects both the network topology and device configuration, as well as information about the current task load, serving as input for intelligent scheduling decisions.

[0147] This embodiment thoroughly solves the problems of unstructured, difficult-to-process, and low-correlation original network and task information by hierarchical feature extraction, topology structuring transformation, correlation feature mining, global feature aggregation, and bidirectional information fusion. It can accurately mine the implicit correlation features between TSN topology and task requirements. The generated global embedded state has the characteristics of high recognition and low redundancy, which not only simplifies the input pressure of subsequent policy networks, but also provides comprehensive and reliable state support for joint decision-making, further enhancing feature extraction efficiency and decision adaptability.

[0148] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a task scheduling and resource deployment joint optimization program. When the task scheduling and resource deployment joint optimization program is executed by a processor, it implements the steps of the task scheduling and resource deployment joint optimization method in industrial time-sensitive networks as described above.

[0149] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0150] The aforementioned computer-readable storage medium may be included in a task scheduling and resource deployment co-optimization device in an industrial time-sensitive network; or it may exist independently and not be assembled into a task scheduling and resource deployment co-optimization device in an industrial time-sensitive network.

[0151] Furthermore, this invention also proposes a computer program product, including a task scheduling and resource deployment joint optimization program, which, when executed by a processor, implements the steps of the task scheduling and resource deployment joint optimization method in industrial time-sensitive networks as described above.

[0152] The specific implementation of the computer program product of the present invention is basically the same as the embodiments of the above-described joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks, and will not be repeated here.

[0153] Reference Figure 4 , Figure 4This is a structural block diagram of the first embodiment of the joint optimization device for task scheduling and resource deployment in industrial time-sensitive networks according to the present invention.

[0154] like Figure 4 As shown, the joint optimization device for task scheduling and resource deployment in industrial time-sensitive networks proposed in this embodiment of the invention includes: The environment configuration module 10 is used to construct an industrial time-sensitive network scheduling model containing multiple industrial production cycles, and to configure an intelligent agent interaction environment based on the industrial time-sensitive network scheduling model. The initial input state of the industrial time-sensitive network scheduling model includes the task set of each industrial production cycle, the industrial time-sensitive network topology, and the device deployment status of each terminal node. The intelligent agent interaction environment is configured with intelligent agents, which are configured with a policy network and are configured to perform task scheduling and device deployment. The specific process is as follows: The state encoding module 20 is used to obtain the task requirements of the current production cycle at the beginning of each industrial production cycle, and to encode the industrial time-sensitive network topology and the equipment deployment status of each terminal node through a graph neural network to obtain the global embedded state. The decision execution module 30 is used to execute equipment deployment decisions and task scheduling decisions sequentially through the policy network according to the task requirements and the global embedded state, so as to obtain the equipment deployment scheme and task scheduling scheme for the current production cycle. The strategy update module 40 is used to calculate the instant reward value at the end of each industrial production cycle based on the task completion status and equipment deployment status of the current production cycle, and to perform strategy gradient update on the strategy network based on the instant reward value. The iterative optimization module 50 is used to, at the beginning of the next industrial production cycle, return the task requirements for obtaining the current production cycle based on the updated policy network, and encode the topology of the industrial time-sensitive network and the equipment deployment status of each terminal node through the graph neural network state encoding module to obtain the global embedded state, until all industrial production cycles end, thus completing the joint optimization of task scheduling and resource deployment of the industrial time-sensitive network. The policy network includes a device deployment actor network, a task scheduling actor network, and a critic network. The device deployment actor network and the task scheduling actor network are configured to update the device deployment scheme and the task scheduling scheme based on the global embedded state. The critic network is configured to guide the policy gradient update of the actor network.

[0155] This embodiment constructs an industrial time-sensitive network scheduling model with multiple industrial production cycles and an intelligent agent interaction environment. The intelligent agent is configured to obtain a global embedded state through graph neural network encoding in each production cycle, and then execute equipment deployment and task scheduling decisions sequentially through a policy network. After the cycle ends, the reward value is calculated to update the policy network, and the full-cycle joint optimization is completed iteratively. This enables dynamic coordination of equipment reuse and task scheduling during multiple production cycles, improving task completion rate and network resource utilization efficiency. Thus, while ensuring that all critical communication flows are delivered on time, it achieves cross-cycle optimization of equipment deployment and task scheduling, enabling scientific decision-making at the beginning of each production cycle. This overcomes the limitations of traditional single-cycle scheduling, improves the overall efficiency of industrial production and network resource utilization over a longer time span, effectively improves production efficiency, and reduces costs.

[0156] The joint optimization apparatus for task scheduling and resource deployment in industrial time-sensitive networks provided in this application employs the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks described in the above embodiments, and can solve the technical problem of joint optimization of task scheduling and resource deployment in industrial time-sensitive networks. Compared with the prior art, the beneficial effects of the joint optimization apparatus for task scheduling and resource deployment in industrial time-sensitive networks provided in this application are the same as those of the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks provided in the above embodiments, and other technical features in the joint optimization apparatus for task scheduling and resource deployment in industrial time-sensitive networks are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0157] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solutions of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0158] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0159] In addition, for technical details not described in detail in this embodiment, please refer to the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks provided in any embodiment of the present invention, which will not be repeated here.

[0160] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0161] It should be noted that the user information (including but not limited to user device information, user personal information, user location information, user behavior information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0162] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0163] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0164] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks, characterized in that, The method includes: An industrial time-sensitive network (ITN) scheduling model encompassing multiple industrial production cycles is constructed. An agent interaction environment is configured based on this model. The initial input state of the ISN scheduling model includes the task set for each industrial production cycle, the ISN topology, and the device deployment status of each terminal node. The agent interaction environment is configured with agents, which are equipped with a policy network and are configured to perform task scheduling and device deployment. The specific process is as follows: At the beginning of each industrial production cycle, the task requirements of the current production cycle are obtained, and the industrial time-sensitive network topology and the equipment deployment status of each terminal node are encoded by a graph neural network to obtain the global embedding state. Based on the task requirements and the global embedding state, the device deployment decision and task scheduling decision are executed sequentially through the policy network to obtain the device deployment scheme and task scheduling scheme for the current production cycle. At the end of each industrial production cycle, an instant reward value is calculated based on the task completion status and equipment deployment status of the current production cycle, and the policy network is updated with a policy gradient based on the instant reward value. At the start of the next industrial production cycle, the task requirements for obtaining the current production cycle are returned based on the updated policy network, and the state encoding module of the graph neural network is used to encode the topology of the industrial time-sensitive network and the equipment deployment status of each terminal node to obtain the global embedded state. This process continues until all industrial production cycles are completed, thus achieving joint optimization of task scheduling and resource deployment of the industrial time-sensitive network. The policy network includes a device deployment actor network, a task scheduling actor network, and a critic network. The device deployment actor network and the task scheduling actor network are configured to update the device deployment scheme and the task scheduling scheme based on the global embedded state. The critic network is configured to guide the policy gradient update of the actor network.

2. The joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks as described in claim 1, characterized in that, The process involves obtaining the task requirements for the current production cycle and encoding the industrial time-sensitive network topology and the equipment deployment status of each terminal node using a graph neural network to obtain a global embedding state, including: Extract feature information of each node in an industrial time-sensitive network and generate feature vectors corresponding to each node, wherein the nodes include terminal nodes and switching nodes; Based on the topology of industrial time-sensitive networks, a network topology graph is constructed. The network topology graph includes a node set and an edge set. The node set includes all terminal nodes and switching nodes, and the edge set includes the link connection relationships between nodes. The feature vectors of each node and the network topology graph are input into the graph neural network, and the node embedding vectors corresponding to each node are calculated through the message passing mechanism. The mean of all node embedding vectors is calculated to obtain the network embedding vector; The task requirements of the current production cycle are extracted and summarized to obtain the overall task requirement feature vector. The network embedding vector is merged with the overall task requirement feature vector to generate a global embedding state.

3. The joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks as described in claim 1, characterized in that, The specific process of making equipment deployment decisions and generating equipment deployment plans includes: Device feature information is extracted from the global embedded state and input into the device deployment actor network. The device feature information includes device cost information and task device requirement information. The fully connected layer of the device deployment actor network calculates the input device feature information to obtain the action preference value of each terminal node corresponding to the optional actions. The optional actions include not performing the operation, installing the device, and removing the deployed device. The action preference values ​​are normalized using a normalized exponential function to obtain the policy probability distribution of the actions corresponding to each terminal node. Based on the probability distribution of the strategy, select the corresponding device operation action for each terminal node, execute the device operation action, and synchronously update the device status and cost statistics information of each terminal node. Once the equipment requirements for all tasks are met, or all terminal nodes have completed their action selections, a device deployment plan for the current production cycle is generated.

4. The joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks as described in claim 3, characterized in that, The task scheduling decision-making process includes path selection for executing the task scheduling decision, which includes: Based on the equipment deployment plan of the current production cycle, update the scheduling status of the industrial time-sensitive network and obtain the time-sensitive communication stream set corresponding to all tasks to be scheduled in the current production cycle. For a single time-sensitive communication flow to be scheduled, extract the attribute parameters of the time-sensitive communication flow and the corresponding set of candidate transmission paths, wherein the set of candidate transmission paths contains at least one candidate transmission path; The link feature sequence of each candidate transmission path is input into the stream encoder to extract the path-level embedding vector corresponding to each candidate transmission path. All path-level embedding vectors are fused with the attribute parameters of the time-sensitive communication stream to obtain the task-level embedding vector corresponding to the time-sensitive communication stream; The task-level embedding vector is input into the task scheduling actor network to calculate the score value corresponding to each candidate transmission path. The score value is then processed by a normalized exponential function to obtain the selection probability distribution of each candidate transmission path. Based on the selection probability distribution, a transmission path is selected for the time-sensitive communication stream until all time-sensitive communication streams to be scheduled have completed the selection of transmission paths.

5. The joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks as described in claim 4, characterized in that, The specific process of generating the task scheduling scheme includes: According to the preset scheduling order, traverse all time-sensitive communication flows of the selected transmission paths and determine all links and corresponding node ports on the transmission path corresponding to a single time-sensitive communication flow. Along the transmission path of the time-sensitive communication stream, at the output port of the source node, the earliest idle time slot that meets the transmission time constraint is allocated for the data frame of the time-sensitive communication stream; Based on the allocated transmission time slots, link propagation delay, and node processing delay, the time for the data frame to reach the next hop node is determined, and an idle time slot that meets the connection requirements is allocated to the output port of the corresponding node for the data frame. The time slot allocation for the entire path of the time-sensitive communication flow is completed hop by hop. When the time slot allocation for the entire path is completed in accordance with the constraints, the time-sensitive communication flow is marked as successfully scheduled, and the time slot occupancy status of the corresponding link and port is updated synchronously. Once all time-sensitive communication streams have completed their time slot allocation attempts, a task scheduling scheme for the current production cycle is generated.

6. The joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks as described in any one of claims 1 to 5, characterized in that, The calculation of the instant reward value based on the task completion status and equipment deployment status of the current production cycle includes: Count the number of time-sensitive communication flows that are successfully scheduled within the current production cycle, and add the corresponding first positive reward value to each successfully scheduled time-sensitive communication flow; Count the number of tasks in the current production cycle that have been successfully scheduled for all associated time-sensitive communication flows, and add the corresponding second positive reward value to each completed task; The system tracks equipment installation and dismantling actions performed during the current production cycle, and calculates the cost penalty for each equipment operation based on the equipment's purchase cost, reconfiguration cost, and residual value. Calculate the average utilization rate of all links in the industrial time-sensitive network during the current production cycle, and calculate the corresponding load balancing penalty value based on the average utilization rate. Based on the first positive reward value, the second positive reward value, the cost penalty value, and the load balancing penalty value, the real-time reward value for the current production cycle is calculated.

7. The joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks as described in claim 6, characterized in that, The step of updating the policy gradient of the policy network based on the instant reward value includes: Store the quadruple data corresponding to the current production cycle into the priority experience replay buffer. The quadruple data consists of the current state, the current action, the immediate reward value, and the state of the next cycle. When the amount of data in the experience playback buffer reaches a preset capacity threshold, or when the scheduling process of all industrial production cycles is completed, a preset number of sample data are extracted from the experience playback buffer according to priority. Based on the extracted sample data, the time-series difference target value is calculated through the commentator network, and the parameters of the commentator network are updated with the goal of minimizing the mean square error. Based on the updated critic network, the advantage function value corresponding to the sample data is calculated through the advantage function, and the advantage function value is then standardized. Based on the standardized advantage function value, policy gradient updates are performed on the device deployment actor network and the task scheduling actor network, respectively, and policy constraints are applied based on entropy regularization constraints. Perform a soft update on the target network of the commentator network to complete the parameter iteration of the policy network.

8. A joint optimization apparatus for task scheduling and resource deployment in industrial time-sensitive networks, applying the joint optimization method for task scheduling and resource deployment in industrial time-sensitive networks according to any one of claims 1 to 7, characterized in that, The device includes: The environment configuration module is used to construct an industrial time-sensitive network scheduling model containing multiple industrial production cycles, and to configure an agent interaction environment based on the industrial time-sensitive network scheduling model. The initial input state of the industrial time-sensitive network scheduling model includes the task set of each industrial production cycle, the industrial time-sensitive network topology, and the device deployment status of each terminal node. The agent interaction environment is configured with agents, which are configured with a policy network and are configured to perform task scheduling and device deployment. The specific process is as follows: The state encoding module is used to obtain the task requirements of the current production cycle at the beginning of each industrial production cycle, and to encode the industrial time-sensitive network topology and the equipment deployment status of each terminal node through a graph neural network to obtain the global embedded state. The decision execution module is used to execute device deployment decisions and task scheduling decisions sequentially through the policy network according to the task requirements and the global embedded state, so as to obtain the device deployment scheme and task scheduling scheme for the current production cycle. The strategy update module is used to calculate the instant reward value at the end of each industrial production cycle based on the task completion status and equipment deployment status of the current production cycle, and to perform a strategy gradient update on the strategy network based on the instant reward value. The iterative optimization module is used to return the task requirements of the current production cycle based on the updated policy network at the beginning of the next industrial production cycle, and to encode the topology of the industrial time-sensitive network and the equipment deployment status of each terminal node through the graph neural network state encoding module to obtain the global embedded state. This process continues until all industrial production cycles are completed, thus completing the joint optimization of task scheduling and resource deployment of the industrial time-sensitive network. The policy network includes a device deployment actor network, a task scheduling actor network, and a critic network. The device deployment actor network and the task scheduling actor network are configured to update the device deployment scheme and the task scheduling scheme based on the global embedded state. The critic network is configured to guide the policy gradient update of the actor network.

9. A joint optimization device for task scheduling and resource deployment in industrial time-sensitive networks, characterized in that, The task scheduling and resource deployment joint optimization device in the industrial time-sensitive network includes: a memory, a processor, and a task scheduling and resource deployment joint optimization program stored in the memory. The processor is used to run the task scheduling and resource deployment joint optimization program, which is configured to implement the task scheduling and resource deployment joint optimization method in the industrial time-sensitive network as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a task scheduling and resource deployment joint optimization program, which, when executed by a processor, implements the task scheduling and resource deployment joint optimization method in industrial time-sensitive networks as described in any one of claims 1 to 7.

Images