An efficient barrier certificate generation method for cyber-physical systems based on collaborative learning

Abstract

The present invention discloses a method for generating an efficient barrier certificate for an information-physical system based on collaborative learning, the method comprising the following steps: Step 1, formally defining the constraint conditions defined in the constraint continuous system formed by abstract modeling of the information-physical system, obtaining the system domain, the initial state set, and the unsafe state set, respectively performing high-frequency sampling and low-frequency sampling on the above sets, and generating training data for obtaining a barrier certificate; Step 2, pre-training the initial model using the data obtained by low-frequency sampling, and obtaining an intermediate model that can approximately reflect the barrier certificate; Step 3, using the intermediate model obtained in Step 2 as the initial model, and using the slave nodes for collaborative learning to obtain the final barrier certificate. Compared with the existing methods, the method of the present invention reduces the generation time, improves the generation success rate, and can synthesize barrier certificates for complex systems that cannot be processed by the most advanced neural network technology at this stage.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology and relates to a method for generating efficient barrier certificates for an information-physical system based on collaborative learning. Background Art

[0002] In recent years, cyber-physical systems (CPS) have been increasingly deployed in various safety-critical areas (e.g., smart grids, autonomous driving, industrial control, and smart cities). Broadly speaking, CPSs can be viewed as continuous dynamic systems on discrete modes. They are hybrid systems whose modeling involves two different dynamics: 1. Continuous dynamics specified by a set of ordinary differential equations (ODEs); 2. Discrete dynamics described by a set of modes that can shift to each other when certain transitions are triggered. However, due to the deep interweaving of continuous and discrete dynamics in CPS behavior modeling, the notorious "state space explosion" problem can easily lead to intolerable verification time or safety property check failures of CPSs. As CPSs need to interact more and more with the complex physical environment around them, the complexity of smart city CPS verification is soaring. Therefore, how to conduct safety verification quickly and efficiently has become an urgent problem to be solved in the design of smart city CPSs.

[0003] There are currently several main classes of methods that are promising for CPS verification. The first uses various approximation methods (i.e., over- or under-approximation) and symbolic methods (especially for ODEs with linear vector fields) to compute the set of reachable states. However, these methods suffer from scalability issues due to the computationally intensive numerical process of solving general ODEs. The second adopts a deductive approach to search for differential invariants, inspired by the concept of inductive invariants (e.g., loop invariants, pre / postconditions of functions) in program verification. For continuous dynamic systems, if differential invariants that satisfy given constraints can be found for all reachable states except the unsafe ones, then the system can be asserted to be safe. Note that the latter class is more effective than the first class in practical safety verification of CPSs, since the invariants are rough approximations of the reachable set and can be discovered based on the ODEs themselves rather than their solutions. Barrier certificates partition the CPS state space into two disjoint parts as a function of the state. Under a given set of initial conditions, all feasible system trajectories should fall into the same part as the initial state, while the unsafe states are in the other part. Various works have been proposed to quickly find barrier certificates for certain CPSs. However, most of them strive to find expressive barrier certificates of specific shapes (e.g., polynomial functions) by reducing the synthesis problem to various optimization problems, such as the sum of squares (SOS) method and semidefinite programming (SDP). Although these fixed-shape-aware methods are powerful in dealing with system dynamics and unsafe sets, they may lead to synthesis failures due to computational intractability. As an alternative, data-driven methods based on neural network training and verification have been studied to achieve efficient barrier certificate synthesis for CPS. Based on limited training data sampled from the state space, a well-designed underlying neural network can quickly approximate the barrier certificate of a given system. Although neural network-based methods are promising, they suffer from the problem of unstable training convergence due to the random weights of the initial NN and the random combination of sampled data during training. In addition, most existing methods rely on a single process to train the sampled data, which greatly limits the potential of neural network-based barrier certificate synthesis to handle high-dimensional continuous systems with a large amount of required sampled data. The serial execution coupled with various randomness can easily lead to extremely long convergence time or even synthesis failure, which is intolerable in the real-time scenarios of many CPS.

[0004] So far, most existing neural network-based methods suffer from serial training on a large amount of sampled data, which can easily lead to two serious problems. First, although the neural network structure is fixed at the beginning of the sampled data training, the weights of the neural network are randomly initialized. In this case, there is a high probability that the initialized neural network cannot find out the barrier certificate. Due to the large number of sampled data, frequently restarting and training with the initialized neural network can be very time-consuming. Second, due to the lack of barrier certificate synthesis methods for distributed machine learning-based methods, the training convergence speed on a large amount of sampled data is very slow when dealing with complex constrained continuous systems. Summary of the invention

[0005] In order to solve the deficiencies of the prior art, the purpose of the present invention is to provide a method for generating efficient barrier certificates for information-physical systems based on collaborative learning, which can accelerate the barrier certificate generation process based on neural networks. The method of the present invention can focus on the generation efficiency of barrier certificates for high-dimensional complex information-physical fusion systems, and quickly generate system barrier certificates through two-stage branch exploration to support large-scale, efficient and secure verification of smart city information-physical fusion systems.

[0006] In order to solve the problem that barrier certificates may not be found and the training convergence speed is slow, this paper proposes a two-stage method for generating barrier certificates, CL-BC (Collaborative learning for Barrier Certificate, CL-BC), which supports collaborative learning between parallel nodes for sampled data training. Figure 1 As shown, the architecture of the method of the present invention has four main parts: 1. Generate training data by high-frequency sampling and low-frequency sampling of the constrained continuous system; 2. Pre-training (i.e. the first stage) to obtain an intermediate model that reflects the approximate barrier certificate; 3. Collaborative learning (i.e. the second stage) to obtain the final barrier certificate; 4. The slave node performs local training on the distributed sampling data, and the distributed sampling data is the data obtained by randomly dividing the data after high-frequency sampling. The first three parts are performed in serial by the master node, and only the fourth part is executed in parallel. Both the master node and the slave node use a convolutional neural network model with the same structure to perform two-stage training and local training respectively. The neural network used in the master and slave nodes is the same, and a convolutional neural network is adopted, generally using three layers. Specifically, it is only necessary to use the characteristics of the neural network fitting to generate the barrier certificate.

[0007] The specific technical solution for achieving the purpose of the present invention is:

[0008] Step 1: Generate training data based on high-frequency and low-frequency sampling complex system

[0009] according to Figure 1, the training of the master node has two stages, namely the pre-training stage and the collaborative learning stage. The introduction of pre-training can quickly obtain a barrier certificate that is close to the optimal solution under the premise that the random initial model does not work well. Since less training data will lead to shorter training time, the sampling data used in pre-training of the present invention is less than that used in collaborative learning, but it does not mean that the model generated by pre-training does not achieve the same effect. In the method of the present invention, the constraints defined in the constraint continuous system abstractly modeled by the information-physical system are formally defined and the system domain, the initial state set, and the unsafe state set are obtained. The above sets are respectively sampled at high frequency and low frequency to generate training data for obtaining the barrier certificate; different frequencies are used for sampling in the implementation process, wherein low frequency sampling is used for pre-training to explore the model, with less training data, and high frequency sampling is used for collaborative learning, with more training data; the high frequency sampling refers to sampling in the same state set in an equidistant manner, and the sampling interval is self-determined according to the complexity of the constraint continuous system. Generally speaking, the more complex the constraint continuous system is, the shorter the sampling interval is, and the simpler the constraint continuous system is, the longer the sampling interval is; the low frequency sampling refers to sampling in the same state set in an equidistant manner, and the sampling frequency is one quarter of the high frequency sampling frequency. In order to avoid sampling twice, by default, the limited sampling data set using low frequency sampling in the method of the present invention is a subset of the data set using high frequency sampling. The sampling method in the present invention performs sampling on all three sets, namely, the system domain X D , initial state set X I and the unsafe state set X U Sampling is performed, and these three state sets are respectively used to model the CPS system itself into three different restrictions of constrained continuous systems. These sets have constraints in high-dimensional complex systems.

[0010] The system domain X D refers to all system regions specified by the constraint continuous system; the initial state set X I It refers to the initial state constraint specified by the constraint continuous system, that is, the initial state of the system; the unsafe state set X U It refers to the unsafe state constraints stipulated by the continuous system, that is, the system enters the unsafe set, that is, the system enters a dangerous or unsafe state.

[0011] Step 2: Loss function modeling method based on barrier certificate characteristics

[0012] Based on the three different constraints defined by the barrier certificate, the key to the barrier certificate generation based on the neural network in the method of the present invention is to encode the constraints in the three high-dimensional complex systems into loss functions. and are finite sampled data from the system domain, the initial state set, and the unsafe state set, respectively. The overall loss function can be expressed as:

[0013]

[0014] Among them, L D , L I and L U They are the sub-loss functions about the system domain, the initial state set and the unsafe state set. If the training can make L(T D ,T I ,T U ) is reduced to 0, then the learned neural network can be used to form a barrier certificate to ensure the security of the system. Let W(x) be the neural network used by both the master node and the slave node. The following three formulas are used to encode the three constraints in the high-dimensional complex continuous system.

[0015] L I (x,τ I )=ReLU(W(x)+τ I ),

[0016] Among them, τ I ≥0 is a small non-negative constant around zero, used as a tolerance, for all x∈T I , ensuring the constraint W(x)≤0, the initial state constraint in the barrier certificate is guaranteed by the nonlinear layer characteristics. This function ensures that all initial state points in the barrier certificate are less than zero and close to a very small non-positive number.

[0017] L U (x,τ U )=ReLU(-W(x)+τ U ),

[0018] Among them, τ U is a non-negative constant close to zero, used as a tolerance to ensure that all x∈T U The constraint W(x)>0 ensures the unsafe state constraint in the barrier certificate through the nonlinear layer characteristics. This function ensures that all unsafe state points in the barrier certificate are greater than zero and close to a very small non-negative number.

[0019]

[0020] Among them, τ D >0 is a very small tolerance around zero to ensure The strict negativity of For all x∈T D , requiring τ BSpecify the width of the upper and lower bounds around W(x). If the point is near the minimum tolerance, the above formula judgment of the barrier certificate can be entered. At the point of the barrier certificate, try to make the reciprocal close to zero in the form of a very small negative number, so that it can be 0.

[0021] Step 3: Method for obtaining an intermediate model of an approximate barrier certificate based on loss value (first stage).

[0022] The pre-training phase attempts to find an approximate barrier certificate in the form of a neural network based on the low-frequency sampled data. Due to failures caused by incorrectly initialized neural networks, this phase may involve more than one training. The constrained continuous system is sampled at a given low frequency, and then Figure 1 As shown, the training rounds are iterated repeatedly for E times. If a loss is found within E rounds that is lower than the preset loss threshold, the model is defaulted to an approximate barrier certificate (i.e. Figure 1 ) and sent it to the next stage. If the loss value in round E is not lower than the set loss threshold, the current model is discarded and a new initialization model is selected. Otherwise, a new round of training will continue. The initial model is a convolutional neural network model with one hidden layer defined in the neural network framework Pytorch; the training is iterated until the overall loss drops to a specified threshold. The loss threshold can be quantitatively adjusted according to the complexity of the constrained continuous system. Generally speaking, the higher the complexity, the higher the loss threshold setting, and the lower the complexity, the lower the loss threshold setting. For example, for a relatively simple two-dimensional constrained continuous system, the threshold is set to 10.

[0023] The pre-training is performed using low-frequency global data, and the quality of the current initialization model is judged based on how fast the loss threshold decreases under the same number of training rounds. Under the same number of training rounds, the faster the loss threshold decreases, the more likely the current initialization model is to generate a true barrier certificate, and the slower the loss threshold decreases, the less likely the current initialization model is to generate a true barrier certificate.

[0024] The newly obtained intermediate model will serve as the "initial model" for the second stage of training.

[0025] Step 4: Collaborative learning-based method for obtaining the final barrier certificate (Phase II)

[0026] After pre-training is completed, CL-BC will perform collaborative learning among slave nodes based on the intermediate model obtained in step 3. At the beginning of collaborative learning, the master node distributes the intermediate model generated by pre-training to all slave nodes for initialization. In one round of collaborative learning, all slave nodes need to send the latest local training information (i.e., gradient) in each training cycle to the master node for knowledge aggregation, and receive the new model from the master node for local model update. The entire collaborative learning process continues for multiple rounds until the total loss L CL-BC Stop when it is equal to 0 or the timeout is triggered. When the total loss L CL-BC When it is equal to 0, the final barrier certificate is obtained. When the timeout is triggered, it means that the barrier certificate has failed to be obtained. The data source of the slave node is the same amount of training data after high-frequency sampling and uniform division. If the barrier certificate is successfully obtained after collaborative learning, it is the final barrier certificate model.

[0027] Total loss L CL-BC The expression formula is as follows:

[0028]

[0029] in, L is the loss function, k is the current kth slave node, N is the total number of slave nodes, T D is the set of system domain training points after high-frequency sampling, T I is the initial state training point set after high-frequency sampling, T U is the set of unsafe state training points after high-frequency sampling. In the above formula, the stopping condition is that all slave node losses are 0.

[0030] like Figure 1 As shown in , the master node periodically collects the gradient information of the slave nodes and distributes the aggregate model to them to achieve knowledge sharing. The sampled data in each partition is dispatched to its corresponding slave node. The slave node trains the data in its own node and obtains the gradient of the updated local model. The gradients between all slave nodes are sent to the gradient cache of the master node and averaged, and then combined with the model parameter cache in the previous round of training to obtain the latest model and send the latest neural network of the master node to all slave nodes. Then calculate the sum of the neural networks of all slave nodes. If the sum of the losses is equal to 0, it means that the barrier certificate is found, proving that the overall constrained continuous system is safe.

[0031] The beneficial effects of the present invention include: the present invention provides an efficient barrier certificate generation method for an information-physical system based on collaborative learning, which can accelerate the barrier certificate synthesis process based on neural networks. The method of the present invention can focus on the generation efficiency of barrier certificates for high-dimensional complex information-physical fusion systems, and quickly generate system barrier certificates through two-stage branch exploration to support large-scale, efficient and secure verification of smart city information-physical fusion systems. Compared with existing related technologies, the method of the present invention reduces generation time and improves generation success rate relative to existing methods. At the same time, the method of the present invention can synthesize barrier certificates for complex systems that cannot be handled by the most advanced neural network technology at this stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is the overall architecture and flow chart of certificate generation of the present invention. DETAILED DESCRIPTION

[0033] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0034] As a security verification method for cyber-physical fusion systems at this stage, barrier certificates are widely used in various security-related cyber-physical fusion systems (e.g., smart grids, autonomous driving, industrial control, and smart cities). Most existing neural network-based barrier certificate synthesis methods cannot handle high-dimensional continuous systems because a large amount of sampled data can easily lead to inaccurate initial models and slow convergence. In order to accelerate the synthesis of barrier certificates based on neural networks, the present invention proposes an effective two-stage method, named CL-BC, which makes full use of the parallel processing capabilities of the underlying hardware and can quickly search for available barrier certificates for current high-order continuous systems. Unlike existing neural network-based methods that use random initial models for barrier certificate synthesis, CL-BC pre-trains the initial generation model based on a small portion of sampled system data in the first stage. In this way, approximate barrier certificates in the form of neural networks can be quickly implemented with little overhead. Based on the collaborative learning scheme proposed in the present invention, CL-BC performs parallel learning on sub-regions in the second stage, where the learning knowledge from different partitions can be aggregated to accelerate the convergence of the global neural network model for barrier certificate generation. In this way, the overall synthesis time of neural network-based barrier certificates can be greatly reduced.

[0035] In the field of machine learning, loss functions are used to represent the difference between model predictions and reality, with the goal of achieving accurate predictions for the model. In deep learning, due to the high requirements for the model, loss functions are used to optimize the prediction accuracy of the neural network model during the training phase. Based on the constraints defined by three different barrier certificates, the key to the generation of barrier certificates based on neural networks is to encode the constraints in the three high-dimensional complex systems into loss functions. and are finite sampled data from the system domain, the initial state set, and the unsafe state set, respectively. The overall loss function can be expressed as:

[0036]

[0037] L D , L I and L U They are the sub-loss functions about the system domain, the initial state set and the unsafe state set. If the training can make L(T D T I ,T U ) is reduced to 0, then the learned neural network can be used to form a barrier certificate. Let W(x) be the neural network used by both the master node and the slave node. The present invention uses the following three formulas to encode the three constraints in the high-dimensional complex continuous system.

[0038] L I (x,τ I )=ReLU(W(x)+τ I ),

[0039] τ I ≥0 is a small non-negative constant around zero, used as a tolerance, for all x∈T I , ensuring the constraint W(x)≤0.

[0040] L U (x,τ U )=ReLU(-W(x)+τ U ),

[0041] τ U is a non-negative constant close to zero, used as a tolerance to ensure that all x∈T U The constraint W(x)>0.

[0042]

[0043] τ D >0 is a very small tolerance around zero to ensure The strict negativity of For all x∈T D , requiring τB Specify the width of the upper and lower bounds around W(x).

[0044] Based on the above loss function, the first stage pre-training algorithm is as follows:

[0045]

[0046] In line 1, sampling is performed on the constrained continuous system at a low frequency. Then lines 2-12 iteratively train until an approximate barrier certificate is found. In lines 3-4, the neural network is initialized along with its master node loss. Lines 5-8 try to calculate an approximation within E training cycles, where line 6 calculates the loss value and line 7 learns the intermediate model M through the stochastic gradient descent algorithm. If the loss of M is less than the threshold T after E training cycles, if an approximate barrier certificate is found in line 10, the search is stopped. Otherwise, a new round of training will continue. The newly obtained intermediate model will be used as the "initial model" for the second stage of training.

[0047]

[0048] Based on the intermediate model obtained above, the algorithm of the main node of the second stage collaborative learning is as follows:

[0049] Algorithm 2 above details the implementation of a slave node with a rank of k. Line 1 obtains local training data from the master node. Receiving data samples from the master node is a blocking function. Lines 2-8 iteratively perform a round of local training. At the beginning of each iteration, line 3 needs to update its local neural network model with the latest model held by the master node. Then, line 4 calculates the loss of the neural network using the local training data. Line 5 calculates the gradient of the slave model, and line 6 uses the tolerance control method to adjust the obtained gradient to avoid generating incorrect barrier certificates. At the end of a round, the slave node sends its newly obtained gradient to the master node.

[0050]

[0051] In the collaborative learning in Algorithm 3 above, the master node periodically collects the gradient information of the slave nodes and distributes the aggregated model to them to achieve knowledge sharing. Algorithm 3 details the master node collaborative learning implementation process. The first line converts the D , X I and X UThe sampled data is divided into N partitions. Lines 2-5 dispatch the sampled data in each partition to its corresponding slave node. Lines 6-21 perform the interaction between the master node and all involved slave nodes. Lines 7-9 send the latest neural network of the master node to all slave nodes. Line 10 initializes the total loss. Lines 11-14 collect the newly trained neural network model gradient and loss information from all slave nodes, where line 13 calculates the sum of the neural network L of all slave nodes CL-BC If L CL-BC If is equal to 0, it means that a usable barrier certificate is found. In this case, Lines 16-17 report the barrier certificate and terminate the entire synthesis process. Otherwise, the master node updates its neural network using the formula provided in Lines 19-20.

[0052] Total loss L CL-BC The expression formula is as follows:

[0053]

[0054] in, L is the loss function, k is the current kth slave node, N is the total number of slave nodes, T D is the set of system domain training points after high-frequency sampling, T I is the initial state training point set after high-frequency sampling, T U is the set of unsafe state training points after high-frequency sampling. In the above formula, the stopping condition is that all slave node losses are 0.

[0055] Example

[0056]

[0057] As can be seen from the above table, the CL-BC method used in the present invention has a shorter generation time and a higher generation success rate than the existing NN-Barrier method. NN-Barrier cannot successfully generate a barrier certificate in the constrained continuous system Exp5 of its dimension, while CL-BC can successfully generate a barrier certificate, which fully demonstrates the excellent performance of the method of the present invention. And compared with the other four different constrained continuous systems, the generation speed of CL-BC is significantly faster than that of NN-Barrier.

[0058] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A method for generating efficient barrier certificates for cyber-physical systems based on collaborative learning, characterized in that: The following steps are involved: Step 1: formally define the constraints defined in the constraint continuous system abstractly modeled by the information-physical system and obtain the system domain, the initial state set and the unsafe state set, perform high-frequency sampling and low-frequency sampling on the above sets respectively, and generate training data for obtaining the barrier certificate; In step 1, the set for sampling includes system domain X D , initial state set X I and the unsafe state set X U ; The set has constraints in the high-dimensional complex system; The system domain X D refers to all system regions specified by the constraint continuous system; the initial state set X I It refers to the initial state constraint specified by the constraint continuous system, that is, the initial state of the system; the unsafe state set X U It refers to the unsafe state constraint specified by the constraint continuous system, that is, the system enters the unsafe set, that is, the system enters a dangerous or unsafe state; The constraints in the high-dimensional complex system are encoded using the following formula, and the overall loss function is expressed as: in, and are finite sample data from the system domain, the initial state set, and the unsafe state set; L D , L I and L U They are the sub-loss functions about the system domain, the initial state set, and the unsafe state set respectively; In the overall loss function representation method, L I (x,τ I )=ReLU(W(x)+τ I ), Where W(x) is the neural network used by both the master node and the slave node, τ I is a small non-negative constant greater than or equal to 0, for all x∈T I , ensure the constraint W(x)≤0; L U (x,τ U )=ReLU(-W(x)+τ U ), Where W(x) is the neural network used by both the master node and the slave node, τ U is a positive constant greater than 0, ensuring that all x∈T U The constraint W(x)>0; Among them, τ D is a tolerance greater than 0, ensuring The strict negativity of For all x∈T D , requiring τ B Specify the width of the upper and lower bounds around W(x); Step 2: Pre-train the initial model using the data obtained by low-frequency sampling to obtain an intermediate model that can approximately reflect the barrier certificate; Step 3: Use the intermediate model obtained in step 2 as the initial model, use the slave nodes for collaborative learning, and obtain the final barrier certificate; In step 3, the intermediate model and the slave nodes are collaboratively learned together. When collaborative learning starts, the master node distributes the intermediate model generated by pre-training to all slave nodes for initialization. In one round of collaborative learning, all slave nodes need to send the latest local training information in each training cycle to the master node for knowledge aggregation, and receive new models from the master node for local model update. When the total loss L CL-BC Stop when it is equal to 0 or the timeout is triggered. When the total loss L CL-BC When it is equal to 0, the final barrier certificate is obtained. When the timeout is triggered, the barrier certificate acquisition fails; The total loss is expressed by the following formula: Among them, L is the loss function, k is the current kth slave node, and N is the total number of slave nodes.

2. The generation method according to claim 1, characterized in that: In step one, the constraint conditions include initial conditions, unsafe conditions and system activity conditions; the high-frequency sampling refers to sampling in the same state set in an evenly spaced manner, and the sampling interval is self-determined according to the complexity of the constrained continuous system; the low-frequency sampling refers to sampling in the same state set in an evenly spaced manner, and the sampling frequency is one-fourth of the high-frequency sampling frequency; the data set obtained by the low-frequency sampling is a subset of the data set obtained by the high-frequency sampling.

3. The generation method according to claim 1, characterized in that: In step 2, the initial model is a convolutional neural network model with one hidden layer defined in the neural network framework Pytorch; the training is iterated until the overall loss drops to a specified threshold, and the loss threshold can be quantitatively adjusted according to the complexity of the limiting continuous system.

4. The generation method according to claim 1, characterized in that: In step 2, the pre-training is performed using low-frequency global data, and the quality of the current initialized model is judged based on how fast the loss threshold decreases under the same number of training rounds. Under the same number of training rounds, the faster the loss threshold decreases, the more likely the current initialized model is to obtain a barrier certificate, and the slower the loss threshold decreases, the less likely the current initialized model is to obtain a barrier certificate. The intermediate model obtained through pre-training is used as the initial model for subsequent steps.

Images