Optimal control method and device for nonlinear networked system, equipment and medium

Abstract

This application discloses an optimal control method, apparatus, device, and medium for a nonlinear networked system, relating to the field of automation control. The method includes quantizing and encoding the system state using a transmitter to obtain a quantized signal; the transmitter includes a dynamic quantizer with adaptive parameters; the quantized signal is transmitted through sensor nodes; only one sensor node has transmission permission at any given time; codeword reconstruction is performed using a receiver based on the quantized signal to obtain a reconstructed state; a generalized policy iterative algorithm is used to determine the system's target optimal performance index function and target optimal control law based on the reconstructed state; a neural network is then used for approximate approximation; finally, the network weights of the neural network are updated until the performance index function of the generalized policy iterative algorithm satisfies monotonically non-increasing and convergent properties, yielding the final optimal control law and state estimation results. This application achieves efficient, safe, and optimal control.

Description

Technical Field

[0001] This application relates to the field of automation control, and in particular to an optimal control method, device, equipment and medium for a nonlinear networked system. Background Technology

[0002] With the rapid development of digital and network technologies, networked systems are widely used in industrial process control, intelligent transportation, unmanned systems, and many other fields. However, most controlled objects in actual industrial processes exhibit nonlinear characteristics, such as the electrode system in the molten magnesium smelting process and the grinding system in a ball mill. Due to the complexity of nonlinear systems, obtaining accurate mathematical models is extremely difficult; therefore, optimal control of nonlinear systems has always been an important research direction in the field of control theory.

[0003] The Hamilton-Jacobi-Bellman (HJB) equations are an important tool for designing nonlinear optimal controllers, but solving them is extremely difficult due to the inherent curse of dimensionality. Adaptive dynamic programming combined with reinforcement learning provides an effective approach to solving HJB equations. Adaptive dynamic programming methods mainly include frameworks such as heuristic dynamic programming and dual heuristic dynamic programming, and their iterative solution methods are mainly divided into value iteration and policy iteration. In recent years, generalized policy iteration, as a novel iterative method, treats value iteration and policy iteration as special cases, thus possessing wider applicability.

[0004] On the other hand, the limited communication bandwidth in networked systems poses a challenge to optimal control. Existing communication protocols, such as event-triggered protocols and weighted one-try-one-drop protocols, can improve bandwidth efficiency to some extent, but the data itself is unprocessed, posing security risks.

[0005] In summary, how to achieve efficient, safe, and optimal control of nonlinear networked systems is a technical problem that needs to be solved in the current control field. Summary of the Invention

[0006] The purpose of this application is to provide an optimal control method, device, equipment, and medium for nonlinear networked systems, which can achieve efficient, safe, and optimal control.

[0007] To achieve the above objectives, this application provides the following solution: Firstly, this application provides an optimal control method for a nonlinear networked system, including: Construct a state-space model of a discrete-time nonlinear networked system and determine the system state; The system state is quantized and encoded using a transmitter to obtain a quantized signal; the system state is transmitted through a protocol frame parsing mechanism; the transmitter includes a dynamic quantizer with adaptive parameters; the quantized signal is transmitted through sensor nodes; only one sensor node obtains transmission permission at any given time. Based on the quantized signal, the receiver is used to reconstruct the codeword to obtain the reconstructed state; Based on the reconstructed state, the system's target optimal performance index function and target optimal control law are determined using a generalized strategy iterative algorithm. Based on the target optimal performance index function and the target optimal control law, an approximate performance index function and an approximate control law are determined using a neural network. The network weights of the neural network are updated according to the approximate performance index function and the approximate control law until the performance index function of the generalized policy iterative algorithm satisfies monotonically non-increasing and convergent properties, thus obtaining the final optimal control law and state estimation results.

[0008] Secondly, this application provides an optimal control device for a nonlinear networked system, comprising: The building module is used to construct the state-space model of a discrete-time nonlinear networked system and determine the system state; The quantization encoding module is used to perform quantization encoding based on the system state using a transmitter to obtain a quantized signal; the system state is transmitted through a protocol frame parsing mechanism; the transmitter includes a dynamic quantizer with adaptive parameters; the quantized signal is transmitted through sensor nodes; only one sensor node obtains transmission permission at a time. The codeword reconstruction module is used to reconstruct the codewords based on the quantized signal using a receiver to obtain the reconstruction state. The target determination module is used to determine the target optimal performance index function and the target optimal control law of the system based on the reconstructed state using a generalized strategy iterative algorithm. An approximation module is used to determine an approximate performance index function and an approximate control law using a neural network based on the target optimal performance index function and the target optimal control law; The update module is used to update the network weights of the neural network according to the approximate performance index function and the approximate control law until the performance index function of the generalized policy iterative algorithm satisfies monotonically non-increasing and convergent properties, thereby obtaining the final optimal control law and state estimation results.

[0009] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the optimal control method for the nonlinear networked system.

[0010] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the optimal control method for the nonlinear networked system.

[0011] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides an optimal control method, apparatus, device, and medium for a nonlinear networked system. It utilizes a protocol frame parsing mechanism through a transmitter and receiver to transmit system states in a small number of codewords, thereby improving transmission efficiency. The protocol frame parsing mechanism not only reduces communication bandwidth consumption but also enhances information transmission security. Furthermore, adaptive parameters are set in the transmitter to further achieve efficient, secure, and optimal control. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a diagram of the adaptive dynamic programming framework. Figure 2 This is a system control block diagram under the protocol frame parsing mechanism; Figure 3 Diagram of the neural network approximation framework structure; Figure 4 The system state and decoding state trajectory diagram; Figure 5 The trajectory graph of the iterative performance index function; Figure 6 The convergence curve of the iterative performance index function; Figure 7 This is the control trajectory diagram for system state 1; Figure 8 This is the control trajectory diagram for system state 2; Figure 9 The graph shows the iterative changes of the control law; Figure 10 A schematic diagram of the optimal control method for a nonlinear networked system; Figure 11 A flowchart illustrating an optimal control method for a nonlinear networked system provided in an embodiment of this application; Figure 12A functional module schematic diagram of an optimal control device for a nonlinear networked system provided in an embodiment of this application; Figure 13 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] The protocol frame parsing mechanism reduces communication bandwidth consumption and enhances information transmission security by compressing and digitizing analog signals.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] In one exemplary embodiment, such as Figure 11 As shown, an optimal control method for a nonlinear networked system is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method includes the following steps.

[0018] Step 101: Construct the state-space model of the discrete-time nonlinear networked system and determine the system state.

[0019] Step 102: Quantize and encode the system state using a transmitter to obtain a quantized signal; the system state is transmitted through a protocol frame parsing mechanism; the transmitter includes a dynamic quantizer with adaptive parameters; the quantized signal is transmitted through sensor nodes; only one sensor node obtains transmission permission at any given time.

[0020] Step 103: Reconstruct the codeword using the receiver based on the quantized signal to obtain the reconstructed state.

[0021] Step 104: Based on the reconstructed state, use the generalized policy iterative algorithm to determine the system's target optimal performance index function and target optimal control law.

[0022] Step 105: Determine the approximate performance index function and approximate control law using a neural network based on the target optimal performance index function and the target optimal control law.

[0023] Step 106: Update the network weights of the neural network according to the approximate performance index function and the approximate control law until the performance index function of the generalized policy iterative algorithm satisfies monotonically non-increasing and convergence, thus obtaining the final optimal control law and state estimation results. The update of the network weights of the neural network according to the approximate performance index function and the approximate control law realizes the iterative process of the generalized policy iterative algorithm.

[0024] In an exemplary embodiment, the reconstructed state is obtained by reconstructing the codeword based on the quantized signal using a receiver. Specifically, this includes: for sensor nodes that have obtained transmission permissions, decoding and matrix representation based on the quantized signal to obtain the reconstructed state; for sensor nodes that have not obtained transmission permissions, a zero-order hold strategy is used to determine the reconstructed state.

[0025] In one exemplary embodiment, the neural network includes a judge network and an action network.

[0026] Based on research on optimal control of nonlinear networked systems, this application overcomes the technical challenges of solving the HJB equations for nonlinear systems, limited network communication bandwidth, and insufficient information transmission security. In another exemplary embodiment, the specific process of applying the optimal control method for nonlinear networked systems in practical applications is provided, such as... Figure 10 As shown, it includes the following steps: Step 1: Establish the state-space model of the discrete-time nonlinear networked system and introduce a protocol frame parsing mechanism as the communication protocol between the sensor and the controller.

[0027] Step 2: Based on the adaptive dynamic programming framework, design a generalized policy iterative algorithm to approximate the target optimal performance index function and the target optimal control law of the system.

[0028] Step 3: Construct a judgment network and an action network using neural networks to approximate the performance index function and control law, respectively, and update the network weights online using the backpropagation algorithm.

[0029] Step 4: Analyze the convergence and optimality of the generalized policy iterative algorithm, and prove that the system has eventual bounded stability under the protocol frame parsing mechanism.

[0030] Step 5: Output the optimal control strategy (i.e., the final optimal control law) and state estimation results of the system.

[0031] Furthermore, in step one, the state-space model of the discrete-time nonlinear networked system is as follows: .

[0032] in, The current state of the system, initialized at... And satisfy , This represents the state at time t+1. This is the initial state of the system. Given a constant; The control law is for controlling the input, i.e., the control law at the current moment; , , All are known real matrices of appropriate dimension; Let be an unknown nonlinear function. t represents time.

[0033] Furthermore, the protocol frame parsing mechanism includes: The protocol frame parsing mechanism of this application adopts a combination of dynamic quantizer and polling (RR) protocol to send, transmit, and receive sensor measurement signals, thereby achieving data compression, bandwidth optimization, and deterministic scheduling. This mechanism consists of two parts: a transmitter process and a receiver process.

[0034] Transmitter process: Design with adaptive parameter adjustment A dynamic quantizer quantizes and encodes the system state to obtain a quantized signal. The quantizer satisfies the following characteristics: Quantization range: .

[0035] The quantization process is represented as follows: .

[0036] The quantized signal Let be the state vector of the system at time k. This represents the quantization range of the static quantizer.

[0037] To ensure the quantizer always operates in the linear region (non-saturation), adaptive parameters are used. Adjustments can be made online according to the following rules: like Then reduce as needed. To improve quantification accuracy.

[0038] like Then increase To avoid saturation.

[0039] There are a total of sensor nodes There are several nodes, each corresponding to a different component of the system state. These nodes share the communication channel via a polling (RR) protocol, with only one node gaining transmission privileges at any given time. Definition Let k be the index of the node that gains permission at time k, and its value is determined by the following formula: .

[0040] At time k, only the first Each sensor node transmits its quantized state components. The transmitter output codeword is: .

[0041] Receiver process: The receiver received the code word Then, extract the node index. With the corresponding quantization value For nodes that have not obtained transmission rights, the decoder adopts a zero-order hold strategy, that is, it holds the decoded value from the previous time step.

[0042] Let the decoded state be... The decoding rules are as follows: .

[0043] in, For sensor node indexes, the initial decoding state satisfies . Let N be the decoded state component of the Nth sensor node at time k. Let be the decoded state component of the m-th sensor node at time k. Let be the decoded state component of the m-th sensor node at time k-1. For ease of subsequent closed-loop system analysis, the above decoding rules can be uniformly represented in matrix form: .

[0044] in, .

[0045] To quantize and decode the state at the current moment, This is the global reconstructed state vector from the previous time step. For decoding sequence, It is the identity matrix. For the Kronecker delta function, satisfying: .

[0046] Receiver: Reconstructs the state based on the codeword, used for controller design.

[0047] Through the aforementioned protocol frame parsing mechanism, the system state is compressed into a small number of codewords for transmission. At the same time, the RR protocol ensures deterministic scheduling of the communication channel and efficient utilization of bandwidth, achieving data compression and secure transmission between the sensor and the controller.

[0048] Furthermore, in step two, the generalized policy iterative algorithm includes the following two iterative phases: Strategy Improvement: .

[0049] Strategy Evaluation: .

[0050] in, It is a value function. To reconstruct the state, For the first The performance metric function for the next iteration. For performance index functions, For the first Control law for the next iteration For the i-th iteration The performance index function of the step, For the i-th iteration The performance index function of the step, In the current state Next, adopt the current control law. Performance metrics at that time , For utility function, It is a positive semi-definite symmetric matrix. It is a positive definite symmetric matrix. In the refactoring state Next, the control law at the current moment will be used. Performance metrics at that time Represents the utility function. This represents the single-step instantaneous cost function.

[0051] Furthermore, in step three, the output of the evaluation network is Its error function is The update rules are as follows: .

[0052] The output of the action network is Its error function is The update rules are as follows: .

[0053] in, To evaluate the network at the nth iteration, the i-th time step, and the... The output value of each output node To reconstruct the state, It is the evaluation of the connection in the network in the nth iteration. The weight matrix of each output node. It is either an offset or a structural parameter. It is the first in the evaluation network The bias term of each node, To evaluate the network's error function, It is the error value used to evaluate the network. It is used to evaluate the learning rate of a network. It outputs the gradient with respect to the weights. To evaluate the activation function of the network, n is the number of iterations. To evaluate the connection in the network during the (n+1)th iteration The weight matrix of each output node; To evaluate the connection in the network in the nth iteration The weight matrix of each output node; For the output of the action network, It is the weight matrix of the action network. It is the activation function of the action network. It is a linear combination of inputs to the hidden layer. It is the error function of the action network. It is the approximation error of the action network. It is the learning rate of the action network. These are the new weights after the action network update. These are the current weights before the action network update. It is the partial derivative of the error with respect to the network output. It is the gradient of the network output with respect to the weights.

[0054] The online update method of the neural network terminates the iteration when the calculation threshold is met or the maximum number of iterations is reached, and outputs the current optimal control law as the final control strategy.

[0055] Furthermore, the closed-loop control system of the system under the protocol frame parsing mechanism is as follows: .

[0056] in, y represents the decoding error, and p represents the controller gain.

[0057] The system satisfies the condition of exponential eventual bounded stability if: there exists a constant. Positive definite matrix , and matrix This makes the linear matrix inequality hold.

[0058] Furthermore, the performance index function sequence generated by the generalized policy iterative algorithm satisfy: Monotonic non-incremental . For the i-th iteration Performance metrics for each step.

[0059] Convergence: ,in This is the optimal performance index function.

[0060] Furthermore, the optimal control strategy described in step five is: .

[0061] The state estimation result is the decoded state. It satisfies: .

[0062] in, For the final optimal control law, To reconstruct the state, For the control law at the current moment, The optimal performance index function. Let the utility function represent the state in the current reconstruction. Take control The immediate cost incurred, where n is the number of iterations. This is the current state. It is a utility function.

[0063] This application addresses nonlinear networked systems by employing a protocol frame parsing mechanism as the communication protocol, which significantly reduces communication bandwidth consumption while enhancing the security of information transmission.

[0064] This application proposes a generalized policy iteration algorithm based on adaptive dynamic programming, which unifies value iteration and policy iteration under the same framework, thus expanding the application scope of adaptive dynamic programming in nonlinear networked systems.

[0065] This application employs neural networks to construct evaluation networks and action networks respectively, enabling online approximation and updating of performance index functions and control strategies, thus overcoming the problem of the difficulty in analytically solving the HJB equation.

[0066] This application presents a rigorous convergence and optimality analysis of the proposed algorithm, proving that the algorithm can converge to the optimal performance index function.

[0067] This application proves the exponentially bounded stability of the system under the protocol frame parsing mechanism using the Lyapunov method.

[0068] In another exemplary embodiment, the method provided in this application will be described in detail with reference to a specific simulation object, a torsion pendulum system.

[0069] In step one, the dynamic equations of the torsional pendulum system are described as follows: .

[0070] in, For gravity, The length of the pendulum rod. For angle, Angular velocity, moment of inertia V is the total mass of the object under study, L is the length of the uniform thin rod, and the coefficient of friction is... , To control the input, a sampling period is used. Discretization yields a discrete-time system model: .

[0071] in, initial state , , This is the control law at the current moment. The current state of the system The second component, The current state of the system The first component.

[0072] Protocol frame parsing mechanism parameter settings: Select quantization parameters , The transmitter quantizes the estimated state error vector to generate codewords, and the receiver reconstructs the state based on the codewords. . Figure 6 The results show that as the node pairing index iterates, the actual state continues to decrease and gradually stabilizes, proving that the model iteration process has good convergence characteristics and provides a reliable foundation for subsequent control.

[0073] In step two, based on the reception status provided in step one... A generalized policy iteration algorithm is designed to approximate the optimal performance index function and optimal control law of the system through alternating iterations of policy evaluation and policy improvement.

[0074] The specific algorithm design is as follows: (a) Definition of utility function: .

[0075] In this embodiment, the settings are as follows: , ,in It is an identity matrix of appropriate dimensions.

[0076] (ii) Generalized strategy iteration process.

[0077] Step 1 (Initialization): Let Select the initial permissible control law Set calculation threshold Construct a sequence of non-negative integers .

[0078] Step 2 (Strategy Improvement): For each iteration index Update the control law: .

[0079] Step 3 (Strategy Evaluation): For each ,make Incrementing from 0 to Repeated updates: .

[0080] Step 4 (Convergence Judgment): If If the algorithm converges, it outputs the current control law; otherwise... Return to step 2.

[0081] In step three, a neural network is used to construct an evaluation network and an action network, which approximate the performance index function and control law in step two, respectively. The backpropagation algorithm is then used to update the network weights online, achieving adaptive optimal control without requiring a precise model. In this embodiment, both the evaluation network and the action network adopt a 1-15-1 structure, i.e., one node in the input layer, 15 nodes in the hidden layer, and one node in the output layer. Activation function: .

[0082] Evaluate network output : .

[0083] Expected output for: .

[0084] Error function: .

[0085] Weight update rules: .

[0086] in, For the vector z, the first The index of a scalar component. The subscript index represents the neuron sequence.

[0087] Action networks for approximate control laws The network output is: .

[0088] Expected output Defined as: .

[0089] Error function: .

[0090] Weight update rules: .

[0091] in For the output of the action network, It is the weight matrix of the action network. It is the activation function of the action network. It is a linear combination of inputs to the hidden layer. It is the error function of the action network. It is the approximation error of the action network. It is the learning rate of the action network. These are the new weights after the action network update. These are the current weights before the action network update. It is the partial derivative of the error with respect to the network output. It is the gradient of the network output with respect to the weights. Represented as an activation function, It is an exponentially decaying function. To evaluate the network at time step i, the... The output value of each output node The error function value. For output error, Output for nodes. As weight.

[0092] In step four, the closed-loop stability of the optimal control system designed in step three under the encoder-decoder mechanism is analyzed. Using the Lyapunov method, it is proven that the system exponent is eventually bounded, providing a theoretical guarantee for the reliability of the control strategy. Under the protocol frame parsing mechanism, the closed-loop control system is: .

[0093] in, Let y represent the decoding error, and p represent the controller gain. A Lyapunov function is chosen. ,in The matrix is ​​a positive definite symmetric matrix. The stability of the system is verified by solving the following linear matrix inequality: .

[0094] In this embodiment, the following is selected: , , , , It is the identity matrix. Let A be a second-order identity matrix and A be the system state matrix. For parameters, , , , , All are matrix parameters. Let be the inverse of the initial state matrix. It has been verified that the above linear matrix inequality has a feasible solution, and the system satisfies the exponentially bounded final stability condition.

[0095] exist Figure 1 This paper demonstrates the basic framework of adaptive dynamic programming, providing a theoretical basis for subsequent solutions to the HJB equations based on neural network approximation. Figure 2 The diagram illustrates the closed-loop control block diagram of a nonlinear networked system, including a transmitter, receiver, execution network, and evaluation network, under the protocol frame parsing mechanism. Figure 2 upper level and As a state input, through the current control law and the next moment control rate The network is updated by calculating the time-series error to complete the iterative training of the generalized policy. The next layer will converge to the optimal control. Send to the controller drive system to generate , obtained through quantization encoding Codeword reconstruction output after network transmission The feedback loop fully demonstrates the entire process of networked adaptive optimal control based on Adaptive Dynamic Programming (ADP) under the protocol frame parsing mechanism. Figure 3 The paper demonstrates the structure of the evaluation network and action network, and how they are used to approximate performance index functions and control laws. Figure 4 The image shows a comparison between the actual system state and the decoding / reconstruction state, verifying that the receiving state can effectively track the real state. Figure 4 (a) in the diagram is the actual system state x1 trajectory diagram. Figure 4 (b) in the diagram is the trajectory diagram of the decoded and reconstructed state x2. Figure 5 The actual system state x1 and the decoding / reconstruction state x2 illustrate the performance index function. The curve showing the change with iteration number i, from Figure 5 It can be observed that as the iteration number i increases, Monotonically decreasing; when hour, It tends to stabilize and converges to the optimal value of approximately 45.2; the convergence process satisfies the monotonically non-increasing property in the theoretical analysis.

[0096] Figure 6 Showing The convergence trajectory was observed. This further validated the algorithm's performance in the dual iteration (outer i, inner i) model. Convergence characteristics under (). Figure 7 and Figure 8 It shows the trajectories of the system state components at different iteration numbers, and the initial control law ( Under the condition that the system state converges slowly and has a large overshoot, after 5 iterations the state response is significantly improved; after multiple iterations the system state converges quickly to the equilibrium point without overshoot. Figure 9 The iterative changes of the control law are demonstrated. Compared with the initial control law, the iterative control law enables the system state to stabilize more quickly, verifying the effectiveness and feasibility of the proposed algorithm.

[0097] Based on research on optimal control of nonlinear networked systems, this application overcomes the technical challenges of solving the HJB equations for nonlinear systems, limited network communication bandwidth, and insufficient information transmission security. Furthermore, the proposed generalized policy iteration algorithm unifies the frameworks of value iteration and policy iteration, providing new theoretical basis and technical support for the application of adaptive dynamic programming methods in complex networked environments. Finally, using a torsion pendulum system as a simulation object, this application verifies the effectiveness of the proposed algorithm. The decoded state can effectively track the actual state, the performance index function monotonically decreases and converges, and the control law gradually transforms into admissible control.

[0098] This application addresses discrete-time nonlinear networked systems, employing a protocol frame parsing mechanism as the communication protocol to improve information transmission security while reducing communication bandwidth consumption. Within an adaptive dynamic programming framework, a generalized policy iteration algorithm is designed. Through alternating iterations of policy evaluation and policy improvement, the algorithm approximates the optimal performance index function and control policy of the system. Neural networks are used to construct evaluation and action networks respectively, enabling the approximation and online updating of the performance index function and control law. Theoretical analysis proves the convergence and optimality of the proposed algorithm and provides the final bounded stability condition of the system exponential based on the Lyapunov method. Simulation results show that this method can effectively achieve stable system state tracking and optimal control, and is suitable for nonlinear networked control systems with limited communication resources and the need to ensure information security.

[0099] Based on the same inventive concept, this application also provides an optimal control device for a nonlinear networked system for implementing the optimal control method for the nonlinear networked system described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the optimal control device for a nonlinear networked system provided below can be found in the limitations of the optimal control method for nonlinear networked systems described above, and will not be repeated here.

[0100] In one exemplary embodiment, such as Figure 12 As shown, an optimal control device for a nonlinear networked system is provided, comprising: The module is used to construct the state-space model of a discrete-time nonlinear networked system and determine the system state.

[0101] The quantization encoding module is used to perform quantization encoding using a transmitter based on the system state to obtain a quantized signal; the system state is transmitted through a protocol frame parsing mechanism; the transmitter includes a dynamic quantizer with adaptive parameters; the quantized signal is transmitted through sensor nodes; only one sensor node obtains transmission permission at a time.

[0102] The codeword reconstruction module is used to reconstruct the codewords based on the quantized signal using a receiver to obtain the reconstruction state.

[0103] The target determination module is used to determine the target optimal performance index function and the target optimal control law of the system based on the reconstructed state using a generalized strategy iterative algorithm.

[0104] An approximation module is used to determine an approximate performance index function and an approximate control law using a neural network based on the target optimal performance index function and the target optimal control law.

[0105] The update module is used to update the network weights of the neural network according to the approximate performance index function and the approximate control law until the performance index function of the generalized policy iterative algorithm satisfies monotonically non-increasing and convergent properties, thereby obtaining the final optimal control law and state estimation results.

[0106] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 13As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores optimal control data for the non-linear networked system. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an optimal control method for the non-linear networked system.

[0107] Those skilled in the art will understand that Figure 13 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment to which the present application is applied. Specific computer equipment may include, for example, [the following is a list of possible additional structures]. Figure 13 The embodiments show more or fewer components, combinations of certain components, or different component arrangements. In one exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the above-described method embodiments.

[0108] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described method embodiments.

[0109] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method embodiments.

[0110] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0111] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0112] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0113] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for optimal control of a nonlinear networked system, characterized in that, include: Construct a state-space model of a discrete-time nonlinear networked system and determine the system state; Based on the system state, the transmitter performs quantization encoding to obtain the quantized signal; The system status is transmitted through a protocol frame parsing mechanism; the transmitter includes a dynamic quantizer with adaptive parameters; the quantized signal is transmitted through sensor nodes; only one sensor node obtains transmission permission at any given time. Based on the quantized signal, the receiver is used to reconstruct the codeword to obtain the reconstructed state; Based on the reconstructed state, the system's target optimal performance index function and target optimal control law are determined using a generalized strategy iterative algorithm. Based on the target optimal performance index function and the target optimal control law, an approximate performance index function and an approximate control law are determined using a neural network. The network weights of the neural network are updated according to the approximate performance index function and the approximate control law until the performance index function of the generalized policy iterative algorithm satisfies monotonically non-increasing and convergent properties, thus obtaining the final optimal control law and state estimation results.

2. The optimal control method for a nonlinear networked system according to claim 1, characterized in that, The quantization process of the dynamic quantizer is as follows: ； in, The quantized signal For adaptive parameters, Let be the state vector of the system at time k. is an unknown nonlinear function of the system.

3. The optimal control method for a nonlinear networked system according to claim 1, characterized in that, The online adjustment rules for the adaptive parameters are as follows: like Then reduce the adaptive parameter as needed; like If so, increase the adaptive parameter; in, Let be the state vector of the system at time k. For adaptive parameters, This represents the quantization range of the static quantizer.

4. The optimal control method for a nonlinear networked system according to claim 1, characterized in that, Based on the quantized signal, codeword reconstruction is performed using a receiver to obtain the reconstructed state, specifically including: For sensor nodes that have obtained transmission permissions, the quantized data is decoded and represented in a matrix to obtain the reconstructed state; For sensor nodes that have not obtained transmission permissions, a zero-order hold strategy is used to determine the reconstruction state.

5. The optimal control method for a nonlinear networked system according to claim 1, characterized in that, The iterative phase of the generalized policy iterative algorithm includes: use Make strategic improvements; use Conduct strategy evaluation; in, It is a value function. To reconstruct the state, For the first The performance metric function for the next iteration. For performance index functions, For the first Control law for the next iteration For the i-th iteration The performance index function of the step, For the i-th iteration The performance index function of the step, In the current state The current control law is then adopted. Performance metrics at that time For the control law at the current moment, In the refactoring state Next, the control law at the current moment will be used. Performance metrics at that time Represents the utility function. This represents the single-step instantaneous cost function.

6. The optimal control method for a nonlinear networked system according to claim 1, characterized in that, The neural network includes a judge network and an action network; The output of the evaluation network is: ； The error function of the evaluation network is: ； The update rules for the evaluation network are as follows: ； in, To evaluate the network at the nth iteration, the i-th time step, and the... The output value of each output node To reconstruct the state, It is to evaluate the connection in the network. The weight matrix of each output node. It is either an offset or a structural parameter. It is the first in the evaluation network The bias term of each node, To evaluate the network's error function, It is the error value used to evaluate the network. It is used to evaluate the learning rate of a network. It outputs the gradient with respect to the weights. To evaluate the activation function of the network, n is the number of iterations. To evaluate the connection in the network during the (n+1)th iteration The weight matrix of each output node; To evaluate the connection in the network in the nth iteration The weight matrix of each output node; The output of the action network is: ； The error function of the action network is: ； The update rules for the action network are as follows: ； in, For the output of the action network, It is the weight matrix of the action network. It is the activation function of the action network. It is a linear combination of inputs to the hidden layer. It is the error function of the action network. It is the approximation error of the action network. It is the learning rate of the action network. These are the new weights after the action network update. These are the current weights before the action network update. It is the partial derivative of the error with respect to the network output. It is the gradient of the network output with respect to the weights.

7. The optimal control method for a nonlinear networked system according to claim 1, characterized in that, The final optimal control law is: ； The state estimation result satisfies ; in, For the final optimal control law, To reconstruct the state, For the control law at the current moment, This is the optimal performance index function. Let the utility function represent the state in the current refactoring. Take control The immediate cost incurred, where n is the number of iterations. This is the current state. It is a utility function.

8. An optimal control device for a nonlinear networked system, characterized in that, include: The building module is used to construct the state-space model of a discrete-time nonlinear networked system and determine the system state; The quantization encoding module is used to perform quantization encoding using a transmitter based on the system state to obtain a quantized signal. The system status is transmitted through a protocol frame parsing mechanism; the transmitter includes a dynamic quantizer with adaptive parameters; the quantized signal is transmitted through sensor nodes; only one sensor node obtains transmission permission at any given time. The codeword reconstruction module is used to reconstruct the codewords based on the quantized signal using a receiver to obtain the reconstruction state. The target determination module is used to determine the target optimal performance index function and the target optimal control law of the system based on the reconstructed state using a generalized strategy iterative algorithm. An approximation module is used to determine an approximate performance index function and an approximate control law using a neural network based on the target optimal performance index function and the target optimal control law; The update module is used to update the network weights of the neural network according to the approximate performance index function and the approximate control law until the performance index function of the generalized policy iterative algorithm satisfies monotonically non-increasing and convergent properties, thereby obtaining the final optimal control law and state estimation results.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the optimal control method for the nonlinear networked system according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the optimal control method for the nonlinear networked system as described in any one of claims 1-7.

Images