Intelligent transmission scheduling method for industrial network systems driven by perception and control performance

Abstract

The present invention discloses an intelligent transmission scheduling method for an industrial network system driven by perception and control performance, which relates to the field of industrial network systems and includes the following steps: 1: optimization problem construction and perception model pre-learning; 2: transmission resource reservation; 3: dynamic transmission scheduling based on reinforcement learning; 4: perception and control performance evaluation. The method of the present invention mainly solves the actual needs of balancing system performance and limited transmission resources in complex industrial production. The intelligent transmission scheduling method adopted matches the current expanding industrial production scale and potential unknown system parameters. According to the present invention, the transmission scheduling method can be deployed conveniently and quickly in the absence of precise prior information on the system model, and the weights of various performance and resources can be adjusted according to actual production needs to achieve ideal production effects. It can be effectively extended to a variety of different industrial application scenarios.

Description

Technical Field

[0001] The present invention relates to the field of industrial network systems, and in particular to an intelligent transmission scheduling method for industrial network systems driven by perception and control performance. Background Art

[0002] With the continuous evolution of Industry 4.0, a new generation of industrial network systems integrating key processes such as ubiquitous sensing, transmission, control, and learning are empowering the development of industrial automation. Leveraging advanced edge computing and data transmission technologies, edge computing units can efficiently integrate information from multiple sensors in the field and promptly transmit control instructions to actuators, thereby achieving ultimate production control objectives. Improved sensing and control performance relies on more efficient data exchange, but this comes at the expense of scarce industrial resources such as bandwidth, time slots, and energy. Therefore, it is urgent to design joint scheduling methods for uplink (sensor-to-edge computing unit) and downlink (edge ​​computing unit-to-actuator) transmission to improve overall system performance. Furthermore, the ever-expanding scale of industrial network systems and the partially unknown system models have led to increasing attention for intelligent scheduling methods based on reinforcement learning and other approaches. However, further exploration is needed to integrate system performance indicators such as observability and controllability with the design of intelligent scheduling methods.

[0003] In most existing industrial network system transmission scheduling methods, there is no joint scheduling of uplink and downlink data transmission for perception and control performance; there is also a lack of analysis between the adopted intelligent scheduling method and the comprehensive performance of the system, and the degree of matching between the two still needs to be further improved. For example, the domestic application number 202310602949.8, "Resource Scheduling Method for Wireless Networked Control Systems Based on IEEE802.11ax", proposed a multi-subsystem transmission scheduling method for control performance, but has not yet considered the situation where a single system needs to fuse multi-sensor information to support control decisions; the domestic application number 202211394440.0, "A Method for On-Demand Transmission of Perception Information in Industrial Internet of Things", and the domestic application number 202110453745.3, "A Method for Industrial Edge Perception with Observability Guarantee", both focus only on uplink. The transmission scheduling of perception data does not take into account the on-demand transmission requirements of downlink control instructions; the "A method for intelligent scheduling of cache and communication resources of a single relay in the Internet of Things" with domestic application number 202110824751.5 and the "A method for intelligent scheduling of hybrid business flows in air-based networks based on deep reinforcement learning" with domestic application number 202311674058.X, although they introduced intelligent reinforcement learning methods to obtain the optimal transmission mechanism, have not yet been combined with specific industrial objects to characterize the perception or control performance of the system. Therefore, the relationship between intelligent scheduling design and system control performance still needs further discussion.

[0004] Therefore, technicians in this field are committed to developing an intelligent transmission scheduling method for industrial network systems driven by perception and control performance. Summary of the Invention

[0005] In view of the above-mentioned defects of the prior art, the technical problems to be solved by the present invention are: to reasonably reserve the overall transmission resources required for perception and control performance, and guide the design of dynamic transmission scheduling based on intelligent learning methods; for actual complex and unknown industrial systems, consider the dynamic transmission scheduling scenarios that require the fusion of multi-sensor data for system status perception and require multiple actuators to jointly complete the control objectives; and to finely design the intelligent scheduling method in combination with perception and control performance to ensure the comprehensive performance of the intelligent scheduling method.

[0006] To achieve the above objectives, the present invention provides a method for intelligent transmission scheduling of an industrial network system driven by perception and control performance, characterized in that the method comprises the following steps:

[0007] Step S1: Construct a first model to characterize the dynamics of the industrial object system, formulate a first optimization problem for the industrial network system to optimize the scheduling of uplink and downlink data transmission for perception and control, construct a parameter-based perception model to estimate the state of the industrial system, and iteratively update the perception model parameters based on available historical data;

[0008] Step S2: using the system observability index and controllability index in the selected multiple time periods as constraints, by solving the second optimization problem, determining the total number of uplink and downlink transmission time slots reserved for each period in the subsequent dynamic transmission scheduling process;

[0009] Step S3: Model the dynamic transmission scheduling process of the industrial network system as a Markov decision process, construct a policy network represented by parameters, and use a policy gradient reinforcement learning method to iteratively update the parameters;

[0010] Step S4: After multiple rounds of reinforcement learning training, if the control performance and perception performance meet the production accuracy requirements of the industrial network system, the trained policy network and perception model parameters are saved; if the control or perception performance does not meet the required production accuracy requirements, it is necessary to readjust the values ​​of the relevant weights or constraints to make the system tend to more fully interact with uplink and downlink data, and re-perform the reinforcement learning training process until the performance meets the requirements.

[0011] Furthermore, the first model in step S1 is:

[0012]

[0013] Where, For the time range, 、 and They are Periodic system states, measurements and control inputs, and They are Periodic system noise and measurement noise; is the unknown system model; For the measurement model; and They are The periodic uplink and downlink transmission success indication matrix is ​​determined by the number of uplink and downlink data transmissions and the single transmission success rate of the dynamic transmission scheduling.

[0014] Furthermore,

[0015] The first optimization problem in step S1 is:

[0016]

[0017] in, is the discount factor, and the inequality constraint indicates that the sum of uplink and downlink data transmission times in each cycle must be less than the total number of reserved time slots b; Is the cost function:

[0018]

[0019] Where, for The trajectory equation of the periodic ideal system state is: , is the system state estimation; and are the control performance and control usage weights, and are the uplink and downlink transmission cost weights respectively, and are the number of sensors and actuators, For dynamic scheduling In-cycle sensor The number of transmissions, For dynamic scheduling Edge computing unit to actuator within a cycle The number of transmissions.

[0020] Furthermore, the iterative updating of the perception model parameters based on the available historical data in step S1 is as follows:

[0021]

[0022] Where, for The updated learning rate; ,in is the mean of the initial value of the system state, Characterize the parameters of the system perception model, is the perception model, is the system state estimate, For about The gradient operator symbol.

[0023] Furthermore, the second optimization problem is:

[0024] System observability indicators over multiple time periods and controllability indicators For the constraints, solve:

[0025]

[0026] Where, and are the constraint values ​​of observability and controllability indicators respectively; To reserve sensors for each cycle of the process The number of transmissions, To reserve the edge computing unit to the executor in each cycle of the process The number of transmissions affects and results.

[0027] Furthermore, in step S3, the intelligent transmission scheduling problem of the industrial network system is modeled as a Markov decision process:

[0028] Defining decision states as follows:

[0029]

[0030] Defining decision actions as follows:

[0031]

[0032] Where, is the total transmission scheduling ratio, and Sensors and actuators The transmission scheduling weight is In-cycle sensor and edge computing units to actuators The number of transmissions is determined as follows:

[0033]

[0034] Matching optimization target, definition Cycle Return for .

[0035] Furthermore, the strategy network in step S3 is composed of parameters Representation, indicating the decision state Next action The probability that the edge computing unit collects trajectories in each round after each round of interaction is , for the parameters The iterative updates are as follows:

[0036]

[0037] Where, For parameters The updated learning rate, For about The gradient operator symbol, For about

[0038] The expected operator.

[0039] Furthermore, the step S3 also includes: after each round of interaction, the edge computing unit calculates the value of the collected data based on the collected data. System-aware model parameters To update your learning:

[0040]

[0041] in, For perceived performance.

[0042] Furthermore, the control performance in step S4 is .

[0043] Furthermore, the weights in step S4 include 、 、 、 , the constraint values ​​include 、 .

[0044] This invention proposes an intelligent transmission scheduling method for industrial network systems, driven by perception and control performance. This method addresses the practical need to balance system performance and limited transmission resources in complex industrial production. The intelligent transmission scheduling approach employed matches the ever-expanding scale of industrial production and the potential for unknown system parameters. This method can be quickly and easily deployed even in the absence of precise prior information about the system model, adjusting various performance and resource weights based on actual production needs to achieve optimal production results. This invention closely connects advanced intelligent methods with the perception and control requirements of industrial production, enabling its effective application in a variety of diverse industrial application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 An architecture diagram of an industrial network system supported by edge computing in an embodiment of the perception and control performance-driven intelligent transmission scheduling method for an industrial network system of the present invention;

[0046] Figure 2 A schematic diagram of the data transmission framework of the intelligent transmission scheduling method for an industrial network system driven by perception and control performance of the present invention;

[0047] Figure 3 A schematic diagram of a training process for a specific embodiment of the perception and control performance-driven industrial network system intelligent transmission scheduling method of the present invention;

[0048] Figure 4 This is a flowchart of a specific embodiment of the perception and control performance driven industrial network system intelligent transmission scheduling method of the present invention. DETAILED DESCRIPTION

[0049] The following describes several preferred embodiments of the present invention with reference to the accompanying drawings to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0050] In the drawings, components with identical structures are denoted by the same reference numerals, and components with similar structures or functions are denoted by similar reference numerals. The size and thickness of each component shown in the drawings are arbitrary and are not limited by the present invention. For clarity, the thickness of components in some places in the drawings is appropriately exaggerated.

[0051] The perception in the embodiment of the present invention mainly includes data collection and data uplink transmission performed by the hardware components of the sensors in the industrial network system, and obtaining system state estimation after performing information analysis on the above data; the control in the embodiment of the present invention mainly includes obtaining the system feedback control parameters after performing information analysis on the above data after the sensors in the industrial network system collect data, and sending the control parameters to the actuator.

[0052] Taking a typical industrial network system, the laminar cooling process of industrial hot rolling, as an example, the architecture of the above system consists of two parts: the field equipment layer and the edge computing layer. The field equipment layer mainly includes a sensor network composed of high-temperature guns, thermal imagers and other sensors. The above-mentioned sensors act as sensing elements to upload the collected data to the edge computing layer. The field equipment layer also includes actuators such as nozzles to receive the control parameters issued by the edge computing unit to achieve cooling control. The system architecture is as follows Figure 1 As shown, the mechanical properties of steel are directly affected by the cooling curve, making strip temperature control crucial during this process. Numerous sensors, such as thermal imagers and high-temperature guns, deployed on-site wirelessly transmit measured strip temperature data to edge computing units. Simultaneously, the edge computing units issue control commands to individual cooling water valves to control their openings. Therefore, there is a pressing need to balance strip temperature sensing and control performance with the limited data transmission resources at the hot rolling site.

[0053] like Figure 2 As shown, it is a schematic diagram of the framework composition of the data transmission flow of the intelligent transmission scheduling method of the industrial network system driven by the perception and control performance of the present invention. In the above data flow graphic example, a total of steps need to be taken, including data collection, uplink transmission, intelligent perception, feedback control, downlink transmission, and the update system after control execution to start a new round of data collection in the new state. Among the several main steps of the above data control flow, each process link involves data uploading, and some require the generation of control instructions. In addition, in the process of uplink and downlink transmission of data, observable-controllable performance traction and overall transmission quantity constraints are involved.

[0054] Based on the above Figure 2 The overall data transmission flow process shown in FIG, the training process diagram of the perception and control performance driven industrial network system intelligent transmission scheduling method of the present invention is as follows Figure 3 As shown, the problems solved in each step are:

[0055] Step 1: Optimize problem construction and perception model pre-learning;

[0056] Step 2: Transmission resource reservation;

[0057] Step 3: Dynamic transmission scheduling based on reinforcement learning;

[0058] Step 4: Perception and control performance evaluation.

[0059] Furthermore, according to the perception and control performance driven intelligent transmission scheduling method of industrial network system implemented by the present invention, Figure 4 As shown in , the steps include:

[0060] Step S1: Construct a first model to characterize the dynamics of the industrial object system, formulate a first optimization problem for the industrial network system to optimize the scheduling of uplink and downlink data transmission for perception and control, construct a parameter-based perception model to estimate the state of the industrial system, and iteratively update the perception model parameters based on available historical data;

[0061] Step S2: using the system observability index and controllability index in the selected multiple time periods as constraints, by solving the second optimization problem, determining the total number of uplink and downlink transmission time slots reserved for each period in the subsequent dynamic transmission scheduling process;

[0062] Step S3: Model the dynamic transmission scheduling process of the industrial network system as a Markov decision process, construct a policy network represented by parameters, and use a policy gradient reinforcement learning method to iteratively update the parameters;

[0063] Step S4: After multiple rounds of reinforcement learning training, if the control performance and perception performance meet the production accuracy requirements of the industrial network system, the trained policy network and perception model parameters are saved; if the control or perception performance does not meet the required production accuracy requirements, it is necessary to readjust the values ​​of the relevant weights or constraints to make the system tend to more fully interact with uplink and downlink data, and re-perform the reinforcement learning training process until the performance meets the requirements.

[0064] Specifically, step S1 further includes:

[0065] Step S1.1: Based on the heat conduction equation, the following system model is constructed to describe the temperature evolution process of the strip steel:

[0066]

[0067] Where, For the time range, 、 and They are Periodic system states, measurements and control inputs, and They are Periodic system noise and measurement noise; It is an unknown system model, which is determined by the processes of heat conduction, water cooling and air cooling of the strip; It is a measurement model, which is determined by the layout of various temperature sensors in the industrial site; and They are The periodic uplink and downlink transmission success indication matrix is ​​determined by the number of uplink and downlink data transmissions and the single transmission success rate of the dynamic transmission scheduling.

[0068] Step S1.2: Construct the first optimization problem as follows:

[0069]

[0070] in, is the discount factor, and the inequality constraint indicates that the sum of uplink and downlink data transmission times in each cycle must be less than the total number of reserved time slots b; Is the cost function:

[0071]

[0072] Where, for The trajectory equation of the periodic ideal system state is: , is the system state estimation; and are the control performance and control usage weights, and are the uplink and downlink transmission cost weights respectively, and are the number of sensors and actuators, For dynamic scheduling In-cycle sensor The number of transmissions, For dynamic scheduling Edge computing unit to actuator within a cycle To achieve the overall optimization goal, in addition to the uplink and downlink transmission scheduling, the control input design is also involved in the optimization problem.

[0073] Step S1.3: To obtain the system state estimate , construct with parameters Representation, perception model based on multi-layer neural network , based on the available historical data, the perception model parameters are iteratively updated as follows:

[0074]

[0075] Where, For parameters The updated learning rate; ,in is the mean of the initial value of the system state, For about The gradient operator symbol.

[0076] Furthermore, step S2 further includes:

[0077] With the selected System observability indicators within a time period and controllability indicators As a constraint, the total number of uplink and downlink transmission time slots reserved for each period in the subsequent dynamic transmission scheduling process is determined by solving the following second optimization problem: :

[0078]

[0079] Where, and are specifically taken as the expectation of the trace of the system observability matrix and controllability matrix, and are the constraint values ​​of observability and controllability indicators respectively; To reserve sensors for each cycle of the process The number of transmissions, To reserve the edge computing unit to the executor in each cycle of the process The number of transmissions, which affect and results.

[0080] Furthermore, step S3 further includes:

[0081] Step S3.1: Model the intelligent transmission scheduling problem of the industrial network system as a Markov decision process and define the decision state as follows:

[0082]

[0083] Defining decision actions as follows:

[0084]

[0085] Where, is the total transmission scheduling ratio, and Sensors and actuators The transmission scheduling weight is In-cycle sensor and edge computing units to actuators The number of transmissions is determined as follows:

[0086]

[0087] Matching optimization target, definition Cycle Return for .

[0088] Step S3.2: Construct the Representational Policy Network , which represents the decision state Next action The probability that the edge computing unit collects trajectories in each round after each round of interaction is , using policy gradient reinforcement learning method to adjust the parameters The iterative updates are as follows:

[0089]

[0090] Where, For parameters The updated learning rate, For about The gradient operator symbol, For about In this step, a value network can also be introduced to estimate the state value function of the reinforcement learning process, and the gradient descent method can be used to update the parameters of the value network.

[0091] Step S3.3: After each round of interaction, the edge computing unit collects data System-aware model parameters To update your learning:

[0092]

[0093] in, For perceived performance.

[0094] Furthermore, step S4 further includes:

[0095] After multiple rounds of reinforcement learning training, it is determined whether the control performance and perceived performance If the required strip production accuracy requirements are met, the trained policy network and perception model parameters are saved for online dynamic transmission scheduling after method deployment; if the control or perception performance does not meet the required production accuracy requirements, the relevant weights need to be readjusted, including 、 、 、 , the constraint values ​​include 、 , so that the system tends to have more sufficient uplink and downlink data interaction, and re-enforces the learning training process until the performance meets the requirements.

[0096] The preferred embodiments of the present invention have been described in detail above. It should be understood that numerous modifications and variations based on the concepts of the present invention are possible without inventive effort by those skilled in the art. Therefore, any technical solution that can be derived by one skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A perception and control performance driven intelligent transmission scheduling method for industrial network systems, characterized in that: The above method comprises the following steps: Step S1: Construct a first model to describe the dynamics of the industrial object system. The first model is: in, For the time range, 、 and They are Periodic system states, measurements and control inputs, and They are Periodic system noise and measurement noise; is the unknown system model; For the measurement model; and They are The periodic uplink and downlink transmission success indicator matrix is ​​determined by the number of uplink and downlink data transmissions and the single transmission success rate of the dynamic transmission scheduling; A first optimization problem is constructed for the industrial network system to optimize the scheduling of uplink and downlink data transmission for perception and control. The first optimization problem is: in, is the discount factor, and the inequality constraint indicates that the sum of uplink and downlink data transmission times in each cycle must be less than the total number of reserved time slots b; is the cost function, which is expressed by the formula: in, for The trajectory equation of the periodic ideal system state is: , is the system state estimation; and are the control performance and control usage weights, and are the uplink and downlink transmission cost weights respectively, and are the number of sensors and actuators, For dynamic scheduling In-cycle sensor The number of transmissions, For dynamic scheduling Edge computing unit to actuator within a cycle Number of transmissions; Construct with parameters Perceptual Model of Representation Estimate the state of the industrial system and iteratively update the perception model parameters based on available historical data: in, for The updated learning rate; ,in is the mean of the initial value of the system state, is the system state estimate, For about The gradient operator symbol; Step S2: Using the system observability indicators within the selected multiple time periods and controllability indicators As a constraint, the total number of uplink and downlink transmission time slots reserved for each period in the subsequent dynamic transmission scheduling process is determined by solving the second optimization problem. The second optimization problem is: in, and are the constraint values ​​of observability and controllability indicators respectively; To reserve sensors in each cycle of the process The number of transmissions, To reserve the edge computing unit to the executor in each cycle of the process The number of transmissions affects and the result; Step S3: Model the dynamic transmission scheduling process of the industrial network system as a Markov decision process; Defining decision states as follows: Defining decision actions as follows: in, is the overall transmission scheduling ratio, and Sensors and actuators The transmission scheduling weight is In-cycle sensor and edge computing units to actuators The transmission times are: Matching optimization target, definition Cycle Return for ; Constructed by parameters Representational Policy Network , indicating the decision state Next action The probability of , using the policy gradient reinforcement learning method to iteratively update the parameters: in, For parameters The updated learning rate, For about The gradient operator symbol, For edge computing units to collect trajectories within each round of interaction The expected operator symbol; At the same time, after each round of interaction, the edge computing unit collects data System-aware model parameters To update your learning: in, For perceived performance; Step S4: After multiple rounds of reinforcement learning training, if the control performance and perception performance meet the production accuracy requirements of the industrial network system, the trained policy network and perception model parameters are saved; if the control or perception performance does not meet the required production accuracy requirements, it is necessary to readjust the values ​​of the relevant weights or constraints to make the system tend to more fully interact with uplink and downlink data, and re-perform the reinforcement learning training process until the performance meets the requirements.

2. The perception and control performance driven intelligent transmission scheduling method for industrial network systems according to claim 1, characterized in that: The control performance in step S4 is: .

3. The perception and control performance driven intelligent transmission scheduling method for industrial network systems according to claim 1, characterized in that: The weights in step S4 include 、 、 、 , the constraint values ​​include 、 .

Images