Multi-sensor information transmission scheduling method and system oriented to eavesdropping
By constructing an MDP model and using DDQN to train the optimal value function, the transmission of information from multiple sensors is intelligently scheduled, solving the privacy leakage problem caused by eavesdropping attacks in cyber-physical systems and achieving secure transmission and privacy protection in eavesdropping environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-26
- Publication Date
- 2026-03-10
AI Technical Summary
When cyber-physical systems face eavesdropping attacks, critical status information is easily intercepted in multi-sensor collaborative scenarios, leading to system privacy leaks and security threats, which existing technologies struggle to effectively protect against.
A Markov Decision Process (MDP) model is constructed using a reinforcement learning-based approach. The optimal value function is trained using a dual deep Q-network (DDQN) to extract the optimal transmission scheduling strategy. By intelligently scheduling sensor transmission sequences, the estimation accuracy of eavesdroppers is reduced while protecting the performance of legitimate estimators.
In an eavesdropping attack environment, it effectively reduces the eavesdropper's estimation accuracy, ensures the performance of legitimate estimators, and achieves security and privacy protection for information transmission.
Smart Images
Figure CN121645169A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of information transmission and scheduling technology, and specifically relates to a method and system for transmitting and scheduling multi-sensor information for eavesdropping purposes. Background Technology
[0002] Cyber-physical systems (CPS), as intelligent systems that deeply integrate sensing, computing, communication, and control capabilities, achieve closed-loop interaction between cyberspace and physical processes through the collaborative interconnection of various hardware devices and software modules. This system integrates computer science, automatic control theory, and artificial intelligence methods, constructing a multidisciplinary technical architecture. With technological advancements, CPS, due to its flexible deployment, controllable cost, and strong adaptability, has been widely applied in important scenarios such as intelligent transportation, medical equipment monitoring, industrial production management, and power system dispatching. However, the widespread access of these systems to the internet has also brought about the openness of communication links, making them vulnerable to security threats from cyberspace. More seriously, due to the high coupling between the information domain and the physical domain in CPS, attacks targeting the network layer can further propagate to the physical device level, causing equipment failures or even systemic operational interruptions, resulting in significant socio-economic losses and security risks. Therefore, the security protection of CPS against various cyberattacks has become a key research focus for both academia and industry.
[0003] Among various cyberattacks, eavesdropping attacks, as a typical passive attack, are characterized by their high degree of concealment and difficulty in detection. These attacks do not directly disrupt communication links, but rather illegally obtain sensitive data such as system state estimation information by monitoring wireless channels. Because cyber-physical systems generally employ open wireless communication methods, and their network and physical layers are deeply coupled, the system has a large exposed surface at the communication layer, making it highly vulnerable to eavesdropping. Especially in scenarios where multiple sensors work together, the system needs to schedule multiple nodes to transmit data, which further increases the risk of critical state information being intercepted during transmission. The stolen information not only directly leaks system privacy but may also provide intelligence for subsequent, more complex active attacks, posing a serious threat to the security and stability of the entire cyber-physical system. Summary of the Invention
[0004] The purpose of this invention is to provide a method and system for scheduling the transmission of multi-sensor information for eavesdropping purposes, so as to solve the above-mentioned problems.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for scheduling the transmission of multi-sensor information for eavesdropping purposes, comprising: Input the number of sensor systems and the process parameters of each system; set the failure rate and recovery rate of the normal transmission channel and the eavesdropper channel; and set the trade-off parameters for balancing the performance of legitimate estimation and the weight of privacy protection; and complete the initialization settings of reinforcement learning parameters. For multi-sensor information transmission scenarios under eavesdropping attacks, a Markov Decision Process (MDP) model is constructed. The optimal value function is obtained by training the Markov Decision Process (MDP) model using the Dual Deep Q-Network (DDQN) method. After training, the optimal transmission scheduling strategy is extracted based on the optimal value function obtained from training. This strategy is then applied to the multi-sensor information transmission process to achieve multi-sensor information transmission scheduling.
[0006] Furthermore, the number of input sensor systems and the process parameters of each system, the failure rate and recovery rate of the normal transmission channel and the eavesdropper channel, and the trade-off parameters used to balance the performance of legitimate estimation and the weight of privacy protection, include: With one sensor, the system state equation can be expressed in the following form:
[0007]
[0008] in This represents the corresponding state transition matrix, and the process noise. It belongs to a Gaussian random process. It is the observation matrix, measuring noise. It is a Gaussian random process; At any moment Each sensor Perform the Kalman filter algorithm to obtain the state vector. State estimation And send it to the remote estimator; for the i-th sensor, its and estimation error covariance Defined as follows:
[0009]
[0010] The Kalman filter has converged to a steady state for all , ; In the state estimation process, the legitimate channel and the eavesdropping channel are modeled as the Gilbert-Elliott model:
[0011]
[0012] in This indicates the failure rate, meaning that the transmission was successful in the previous moment but failed in the current moment. This indicates the recovery rate, meaning that the transmission was successful in the previous moment but failed in the current moment. The trace of the estimation error covariance at the illegal eavesdropping end is maintained above that of the legitimate eavesdropper, while the performance of the remote eavesdropper is optimized to balance privacy protection and estimation performance; this is modeled as the following optimization objective function:
[0013]
[0014] in , which is a weighting factor used to measure the performance of the legitimate estimator's state estimation and the weighting metric for protection against eavesdropping attacks at the eavesdropping end.
[0015] Furthermore, after the data packet is transmitted wirelessly to the estimator and the eavesdropper, the remote state estimator's estimation is calculated using the sensor data packet signal. The estimation process is related to the sensor's reception status and is as follows: When the estimator successfully receives data from the sensor... The data will be updated synchronously. Otherwise, the system model will be updated based on the estimation results from the previous time step. That is, the estimator updates the state estimate of sensor i at time k. and the corresponding state estimation error covariance The update formula is:
[0016] The eavesdropper updates the state estimate of sensor i at time k. and the corresponding state estimation error covariance The update formula is: .
[0017] Furthermore, the completion of reinforcement learning parameter initialization settings includes: Set the learning rate, exploration rate and other network parameters of the neural network, initialize the experience replay buffer and its sampling batch size and maximum capacity, set the initial value of the number of training rounds to 0, and complete the initial configuration of the entire transmission scheduling algorithm.
[0018] Furthermore, for the multi-sensor information transmission scenario under eavesdropping attacks, a Markov Decision Process (MDP) model is constructed, including: Constructing a quintuple A Markov Decision Process (MDP) model is constructed for the sensor scheduling decision update process in this scenario: State Space: In a CPS sensor transmission scheduling scenario under an eavesdropping attack, the state is defined by the estimator, the eavesdropper's continuous packet loss time, and the sequence of received data packets.
[0019]
[0020] state space That is, all possibilities A set of combinations; in and This represents the duration of continuous packet loss of sensor i at time k, either at the estimator end or the eavesdropper end. and This indicates a received indication signal; if the estimator or eavesdropper successfully receives a data packet from sensor i at time k, then... ,otherwise ; Action space: Action is defined as the decision to select a sensor, i.e.
[0021] Define the action as , to indicate that sensor i is scheduled to transmit data; State transition probability: The state transition probability is derived based on the channel failure rate, recovery rate, and the setting of related symbol representations.
[0022] The state transition probabilities at the estimator end are shown below:
[0023] The state transition probabilities at the eavesdropper's end are as follows:
[0024] in This indicates whether the transmitted data packet successfully reached the remote estimator or eavesdropper at time k; Cost function: In the MDP process, a certain state Make an action You will receive feedback from the corresponding immediate cost function, as shown below:
[0025] ,
[0026]
[0027]
[0028] in, The function expression is When the system transitions to the state of the next time step, iteratively calculate the instantaneous cost at the next moment; instantaneous reward function. Defined as negative immediate cost:
[0029] The optimization problem is transformed into minimizing the expected value of the immediate cost function and solving the optimal policy of the MDP, where This represents the strategy adopted to achieve the minimum cost, i.e., the optimal transmission scheduling strategy:
[0030] Discount factor: The discount factor γ is a parameter used to weigh current rewards against future rewards, and its value ranges from 0 to 1.
[0031] Furthermore, the optimal value function obtained by training the Markov Decision Process (MDP) model using the Dual Deep Q-Network (DDQN) method includes: Based on MDP modeling and reinforcement learning methods, a state-action value function that maximizes the expected average immediate reward of the discount was designed:
[0032] According to the definition of the optimal value function, it satisfies the Bellman optimality equation:
[0033] in This represents the immediate reward under the current combination of s and a. and Let γ represent the next state and action, respectively, and γ be the discount factor; optimal value function In the process, the optimal strategy is extracted when the state is Optimal action The following relationship must be satisfied: The optimal Q-function is obtained using the D3QN reinforcement learning method, employing a deep neural network. Approximate estimation The network structure was improved by creating two independent neural networks: a target network and a training network, used for action selection and action evaluation, respectively. Gradient descent was used to update the learning weight parameters of the neural networks. Corresponding The network's loss function is constructed as follows:
[0034] The optimization objective in the loss function Calculated by the following formula
[0035] Output the state value function and the advantage function separately, then sum them to obtain the output. value:
[0036] in The dominant function is defined as the state-action value function. With state value function The difference, that is .
[0037] Furthermore, during network training, an experience replay mechanism is employed for state transitions. Stored in the experience playback buffer Random sampling is performed during training for model updates to better evaluate training effectiveness; calculations are performed in each training round. The value is updated via gradient descent using the loss function. And after a fixed number of training steps, synchronize The value is given to the target network Decreasing exploration rate Then begin the next round of training, until the training ends.
[0038] Secondly, the present invention provides a multi-sensor information transmission scheduling system for eavesdropping, comprising: The initialization module is used to input the number of sensor systems and the process parameters of each system, set the failure rate and recovery rate of the normal transmission channel and the eavesdropper channel, as well as the trade-off parameters used to balance the performance of legitimate estimation and the weight of privacy protection; at the same time, it completes the initialization settings of reinforcement learning parameters. The model building module is used to construct Markov Decision Process (MDP) models for multi-sensor information transmission scenarios under eavesdropping attacks. The training module is used to train the Markov Decision Process (MDP) model using the Dual Deep Q-Network (DDQN) method to obtain the optimal value function. The output module is used to extract the optimal transmission scheduling strategy based on the optimal value function obtained after training, and apply the strategy to the multi-sensor information transmission process to realize the transmission scheduling of multi-sensor information.
[0039] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multi-sensor information transmission scheduling method for eavesdropping.
[0040] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the multi-sensor information transmission scheduling method for eavesdropping.
[0041] Compared with the prior art, the present invention has the following technical effects: This invention proposes a secure transmission scheduling method for multi-sensor cyber-physical systems based on reinforcement learning. The aim is to effectively reduce the estimation accuracy of eavesdroppers while ensuring the performance of legitimate estimators in the context of eavesdropping attacks by intelligently scheduling sensor transmission sequences. Attached Figure Description
[0042] Figure 1 This is a state estimation model for cyber-physical systems under eavesdropping attacks.
[0043] Figure 2 This is a graph showing the average cost of training DQN and D3QN.
[0044] Figure 3 This is a diagram illustrating the effect of the algorithm on privacy protection under eavesdropping attacks.
[0045] Figure 4 This is a diagram of the sensor transmission scheduling strategy.
[0046] Figure 5 This is a flowchart. Detailed Implementation
[0047] The present invention will be further described below with reference to the accompanying drawings: Example 1: This invention provides a method for scheduling the transmission of multi-sensor information for eavesdropping purposes, comprising: Input the number of sensor systems and the process parameters of each system; set the failure rate and recovery rate of the normal transmission channel and the eavesdropper channel; and set the trade-off parameters for balancing the performance of legitimate estimation and the weight of privacy protection; and complete the initialization settings of reinforcement learning parameters. For multi-sensor information transmission scenarios under eavesdropping attacks, a Markov Decision Process (MDP) model is constructed. The optimal value function is obtained by training the Markov Decision Process (MDP) model using the Dual Deep Q-Network (DDQN) method. After training, the optimal transmission scheduling strategy is extracted based on the optimal value function obtained from training. This strategy is then applied to the multi-sensor information transmission process to achieve multi-sensor information transmission scheduling.
[0048] This invention proposes a secure transmission scheduling method for multi-sensor cyber-physical systems based on reinforcement learning. The aim is to effectively reduce the estimation accuracy of eavesdroppers while ensuring the performance of legitimate estimators in the context of eavesdropping attacks by intelligently scheduling sensor transmission sequences.
[0049] Example 2, please refer to Figure 5 This invention provides a method for scheduling the transmission of multi-sensor information for eavesdropping purposes, comprising: Step 1: Enter the number of systems Process parameters , , , The value represents the failure rate of the normal transmission channel and the eavesdropper channel. and recovery rate In order to weigh parameters Simultaneously, complete the initial setup of the neural network: set the learning rate. Exploration rate Network parameters. Initialize the experience replay buffer. and its sampling batch size and maximum capacity Set the training epoch number (episode) to 0 to complete the algorithm initialization; Step 2: Initialize the state space set in the MDP model For this round of training, the training steps are set to 0, and training begins in the episode number and step number. Step 3: Randomly generate a number from 0 to 1, following the... The greedy strategy executes the action. Specifically, it uses the exploration rate... The probability is randomly selected from the action space; otherwise, according to equation (19), the probability is used to select an action. Choose the optimal action based on probability; Step 4: Perform the action And complete the estimation process for the entire system. Observe the current reward value. and the next state Complete the state transition for the current step; Step 5: Transition the state Stored in the experience playback buffer The total cost is accumulated and recorded to evaluate the training effectiveness in the final stage; Step 6: Calculate based on (21) The value is updated by gradient descent using the loss function (20). With network parameters As the neural network is continuously updated, its accuracy gradually increases, allowing for more precise scheduling. At this point, steps are incremented by 1, and step 3 is repeated to begin the next training step. Step 7: After 50 training steps, synchronize the parameter values of the training network to the target network, calculate the average cost of that round, and generate a training curve. If training is not yet complete, increment the episode by 1, decrease the exploration rate, and return to Step 2 to start the next round of training. Training ends when the episode count reaches the set maximum number of rounds.
[0050] Consider as Figure 1 The diagram shows a state estimation model for a cyber-physical system under an eavesdropping attack. system representation exist The state vector at time t, It is a system exist The measured value at time, This indicates that the Kalman filter applies to the state vector. The minimum mean square error (MMSE) state estimation, The Kalman filter convergence steady-state value representing the covariance of the state estimation error. , , , Let represent the state estimates of the estimator and the eavesdropper, and the covariance of the estimation error, respectively.
[0051] Consider a model structure with multiple sensors and multiple dynamic processes, where each sensor measures an independent linear physical system. In the case of one sensor, the system state equation can be expressed in the following form:
[0052]
[0053] in This represents the corresponding state transition matrix, and the process noise. It belongs to a Gaussian random process. It is the observation matrix, measuring noise. It is a Gaussian random process.
[0054] At any moment Each sensor Perform the Kalman filter algorithm to obtain the state vector. State estimation And send it to the remote estimator. For the i-th sensor, its and estimation error covariance Defined as follows:
[0055]
[0056] Assume the Kalman filter has converged to a steady state, i.e., for all... , .
[0057] During state estimation, data packets are transmitted to the legitimate estimator via a wireless channel, during which an eavesdropper can intercept this information. The legitimate channel and the eavesdropping channel are modeled using the Gilbert-Elliott model:
[0058]
[0059] in This indicates the failure rate, meaning that the transmission was successful in the previous moment but failed in the current moment. This indicates the recovery rate, meaning that the transmission was successful in the previous moment but failed in the current moment.
[0060] After the data packet is transmitted wirelessly to the estimator and the eavesdropper, the remote state estimator's estimate is calculated using the sensor data packet signal. The estimation process is related to the sensor's reception status and is as follows: When the estimator successfully receives data from the sensor... The data will be updated synchronously. Otherwise, the system model will be updated based on the estimation results from the previous time step. That is, the estimator updates the state estimate of sensor i at time k. and the corresponding state estimation error covariance The update formula is:
[0061] The eavesdropper also uses the same method for updating the state estimate and the estimation error covariance matrix. The eavesdropper updates the state estimate of sensor i at time k. and the corresponding state estimation error covariance The update formula is:
[0062] The research objective of this invention is to find a suitable transmission scheduling strategy that enables the secure scheduling algorithm derived from this strategy to possess privacy protection capabilities. This strategy maintains the trace of the estimation error covariance from the illegal eavesdropper's end above that of the legitimate estimator, while simultaneously optimizing the performance of the remote estimator to balance privacy protection and estimation performance. This problem can be modeled as the following optimization objective function:
[0063]
[0064] in The weighting factor is used to measure the performance of the legitimate estimator's state estimation and the protection against eavesdropping attacks. Specifically, the objective function accumulates the error at each time step, considering both the sensor's estimation error and the eavesdropper's estimation error, and introduces a weighting factor. This allows for a trade-off between the two. Furthermore, the limiting operation and mean expected value of the objective function ensure that the optimization strategy is long-term oriented, rather than just short-term immediate cost optimization.
[0065] In order to design a reasonable and effective secure transmission scheduling algorithm based on reinforcement learning to solve the problem (8) and derive the optimal scheduling strategy, this invention constructs a quintuple. A Markov Decision Process (MDP) model is used to model the update process of sensor scheduling decisions in this scenario.
[0066] State Space: For the CPS sensor transmission scheduling scenario under eavesdropping attack in this study, the state is defined by the estimator and the eavesdropper's continuous packet loss time and data packet reception signal sequence.
[0067]
[0068] Then the state space That is, all possibilities A set of combinations.
[0069] in and This represents the duration of continuous packet loss of sensor i at time k, either at the estimator or the eavesdropper end. and This indicates a received indication signal. If the estimator or eavesdropper successfully receives a data packet from sensor i at time k, then... ,otherwise .
[0070] Action space: Action can be defined as the decision to select a sensor, i.e.
[0071] In the presence of an eavesdropper, the system selects only one sensor to transmit data packets at any given time. By controlling the sparsity of transmission, the probability of an eavesdropper intercepting multiple data packets simultaneously can be reduced, thereby enhancing data privacy protection to some extent. For simplicity, the action can be defined as... , to indicate that sensor i is scheduled to transmit data.
[0072] State transition probability: The state transition probability is derived based on the channel failure rate, recovery rate, and the setting of related symbol representations.
[0073] The state transition probabilities at the estimator end are shown below:
[0074] The state transition probabilities at the eavesdropper's end are as follows:
[0075] in This indicates whether the transmitted data packet successfully reached the remote estimator or eavesdropper at time k.
[0076] Cost function: In the MDP process, a certain state Make an action You will receive feedback from the corresponding immediate cost function, as shown below:
[0077] ,
[0078]
[0079]
[0080] in, The function expression is When the system transitions to the state of the next time step, the instantaneous cost at the next moment can be calculated iteratively. Therefore, the instantaneous reward function... Instantaneous costs that can be defined as negative:
[0081] For the optimization problem (8) in the scenario presented in this paper, it can be transformed into a problem of minimizing the expected value of the instantaneous cost function and solving this optimal MDP strategy, where This represents the strategy adopted to achieve the minimum cost, and the optimal transmission scheduling strategy sought.
[0082] Discount Factor: The discount factor γ is a parameter used to balance current rewards and future rewards, and its value ranges from 0 to 1. A higher discount factor may cause the algorithm to focus more on long-term rewards, but may lead to instability; a lower discount factor may cause the algorithm to focus more on immediate rewards, but may ignore long-term benefits. Therefore, in practice, it should be defined according to the actual situation, and is generally 0.95 or 0.98.
[0083] In order to derive the optimal transmission scheduling strategy for a multi-sensor cyber-physical system from problem (16), this invention, based on MDP modeling and reinforcement learning, designs a state-action value function that maximizes the expected average immediate reward of the discount, in order to achieve the long-term optimal performance of the system.
[0084] According to the definition of the optimal value function, it satisfies the Bellman optimality equation:
[0085] in Represents the immediate reward under the current combination of s and a, which has been declared in the MDP model. and Let represent the next state and action, respectively, and γ be the discount factor. Optimal value function In this process, the optimal strategy can be extracted when the state is... Optimal action The following relationship must be satisfied:
[0086] The optimal Q-function is obtained using the D3QN reinforcement learning method, employing a deep neural network. Approximate estimation The network structure was improved by creating two independent neural networks: a target network and a training network, used for action selection and action evaluation, respectively. Gradient descent was used to update the learning weight parameters of the neural networks. Corresponding The network's loss function is constructed as follows:
[0087] The optimization objective in the loss function Calculated by the following formula
[0088] With this network structure design, the neural network no longer directly outputs... The values are calculated by outputting the state value function and the advantage function respectively, and then summing them to obtain the output. value:
[0089] in The dominant function is defined as the state-action value function. With state value function difference. Right now .
[0090] During network training, an experience replay mechanism is used for state transition. Stored in the experience playback buffer And random sampling is performed during training for model updates to better evaluate training performance. In each training round, calculation is performed according to (21). The value is updated by gradient descent using the loss function (20). And after a fixed number of training steps, synchronize The value is given to the target network Decreasing exploration rate Then begin the next round of training, until the training ends.
[0091] Verify the designed algorithm using simulation examples. Consider a system containing... CPSs of one sensor. State space is 2, process parameters , , , The values are set as follows:
[0092]
[0093]
[0094]
[0095] The steady-state error covariance obtained by the Kalman filter is
[0096] Failure rate of normal transmission channel and recovery rate They are respectively , =0.45. Failure rate of the eavesdropping channel. and recovery rate They are respectively , =0.45, which satisfies the stability condition. Trade-off parameters The value was set to 0.7 to evaluate the relationship between privacy protection and state estimation performance. Since only one sensor is used for scheduling at any given time, there are only two possible actions: scheduling the first sensor or scheduling the second sensor. Therefore, the action space... The elements are only of two types: (0,1) and (1,0).
[0097] The D3QN algorithm was used for training, with 500 epochs and 50 training steps per epoch. After setting the system environment parameters, the algorithm was executed for training. The environment was reset after 50 time steps in each epoch and before the start of the next epoch. The average cost of each epoch was recorded during training to observe convergence and determine the effectiveness of the algorithm.
[0098] Figure 2 The average cost changes of the D3QN and DQN algorithms are shown during 500 training rounds. The DQN algorithm uses the same parameters as the D3QN algorithm.
[0099] As can be clearly seen from the graph, during training, the average cost of the DQN algorithm is consistently higher than that of the D3QN algorithm. Meanwhile, D3QN introduces a competitive network structure, which better captures the advantages of actions, allowing the network to learn the overall value of the state more quickly, resulting in superior convergence performance and smaller overall fluctuations in the average cost during training.
[0100] To verify that the algorithm proposed in this invention effectively provides security protection during the training process. Figure 3 The average evolution trend of the trace of the estimation error covariance at the legitimate estimator and the trace of the estimation error covariance at the eavesdropper were recorded in each training round during the training process.
[0101] The results show that after sufficient training rounds, the policy converges such that, during the simulation, the trace of the eavesdropper's estimation error covariance is consistently higher than the trace of the estimator's estimation error covariance. This demonstrates that the algorithm proposed in this invention can effectively reduce the eavesdropper's performance and protect privacy in the presence of eavesdropping attacks.
[0102] To obtain an effective sensor transmission scheduling strategy, this paper, after 500 rounds of training, iterates through the continuous packet loss times of sensor 1 and sensor 2 at the estimator from 0 to 5. The trained agent is then allowed to select actions for each combination of packet loss times, and the actions are recorded and the strategy graph is drawn as follows. Figure 4 As shown: In the diagram, the blue circles represent the selected sensor 1, and the orange diamonds represent the selected sensor 2. It can be seen that the scheduling strategy exhibits a certain threshold structure, with a clear dividing region between the orange and blue parts of the strategy. That is, after the combined packet loss time reaches a certain magnitude, another sensor will be selected for scheduling. This provides some guidance on the threshold structure of multi-sensor CPS transmission scheduling strategies under eavesdropping attacks.
[0103] In another embodiment of the present invention, a multi-sensor information transmission scheduling system for eavesdropping is provided, which can be used to implement the above-mentioned multi-sensor information transmission scheduling method for eavesdropping. Specifically, the system includes: The initialization module is used to input the number of sensor systems and the process parameters of each system, set the failure rate and recovery rate of the normal transmission channel and the eavesdropper channel, as well as the trade-off parameters used to balance the performance of legitimate estimation and the weight of privacy protection; at the same time, it completes the initialization settings of reinforcement learning parameters. The model building module is used to construct Markov Decision Process (MDP) models for multi-sensor information transmission scenarios under eavesdropping attacks. The training module is used to train the Markov Decision Process (MDP) model using the Dual Deep Q-Network (DDQN) method to obtain the optimal value function. The output module is used to extract the optimal transmission scheduling strategy based on the optimal value function obtained after training, and apply the strategy to the multi-sensor information transmission process to realize the transmission scheduling of multi-sensor information.
[0104] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.
[0105] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to achieve a corresponding method flow or function. The processor described in this embodiment of the present invention can be used for the operation of a multi-sensor information transmission scheduling method for eavesdropping.
[0106] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the multi-sensor information transmission scheduling method for eavesdropping in the above embodiments.
[0107] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0108] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0109] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0110] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A method of transmission scheduling of multi-sensor information oriented to eavesdropping, characterized in that, The input sensor system quantity and process parameters of each system, the failure rate and recovery rate of the normal transmission channel and the eavesdropper channel, and the trade-off parameter for balancing the legal estimation performance and privacy protection weight are set. At the same time, the reinforcement learning parameter initialization setting is completed. A Markov decision process (MDP) model is constructed for the multi-sensor information transmission scenario under eavesdropping attack. The optimal value function is obtained by training the Markov decision process (MDP) model using a double deep Q network (DDQN) method. After training, the optimal transmission scheduling strategy is extracted based on the optimal value function obtained by training, and the strategy is applied to the multi-sensor information transmission process to realize the transmission scheduling of multi-sensor information. The input sensor system quantity and process parameters of each system, the failure rate and recovery rate of the normal transmission channel and the eavesdropper channel, and the trade-off parameter for balancing the legal estimation performance and privacy protection weight are set.
2. The eavesdropping-oriented multi-sensor information transmission scheduling method according to claim 1, characterized in that, In the state estimation process, the legal channel and the eavesdropper channel are modeled as Gilbert-Elliott models: The system state equation is expressed in the following form: wherein denotes the corresponding state transition matrix, process noise is a Gaussian random process, is an observation matrix, measurement noise is a Gaussian random process; At time , each sensor performs a Kalman filter algorithm to obtain a state estimate of the state vector and sends it to the remote estimator; for the i-th sensor, its and the estimation error covariance are defined as follows: The Kalman filter has converged to steady state, and all of , ; The trace of the estimation error covariance of the illegal eavesdropper end is kept above the legal estimator, and the performance of the remote estimator is optimized to balance the privacy protection effect and the estimation performance; the following optimization objective function is modeled: wherein represents the failure rate, i.e. the transmission was successful at the previous time instant and failed at the current time instant; represents the recovery rate, i.e. the transmission failed at the previous time instant and succeeded at the current time instant; The reinforcement learning parameter initialization setting includes: wherein is a weight factor, used to measure the weight metric of the legitimate estimator state estimation performance and the eavesdropper-end eavesdropping attack protection.
3. The eavesdropping-oriented multi-sensor information transmission scheduling method according to claim 2, characterized in that, After the data packet is transmitted wirelessly to the estimator and the eavesdropper, the remote state estimator's estimate is calculated using the sensor data packet signal. The estimation process is related to the sensor's reception status and is as follows: When the estimator successfully receives data from the sensor... The data will be updated synchronously. Otherwise, the system model will be updated based on the estimation results from the previous time step; that is, the estimator will update the state estimate of sensor i at time k. and the corresponding state estimation error covariance The update formula is: The eavesdropper's state estimate update for sensor i at time k and the corresponding state estimation error covariance The update formula is 。 4. The eavesdropping-oriented multi-sensor information transmission scheduling method according to claim 1, characterized in that, The learning rate, exploration rate and other network parameters of the neural network are set, the experience replay buffer and its sampling batch size, maximum capacity are initialized, and the initial value of the training round number is set to 0 to complete the initialization configuration of the entire transmission scheduling algorithm. The Markov decision process (MDP) model is constructed for the multi-sensor information transmission scenario under eavesdropping attack, including:
5. The eavesdropping-oriented multi-sensor information transmission scheduling method according to claim 1, characterized in that, State space: CPS sensor transmission scheduling scenario under eavesdropping attack, the definition of state is composed of continuous packet loss time and data packet reception signal sequence of the estimator and the eavesdropper Constructing five-tuples The update process for sensor scheduling decisions in this scenario is modeled as a Markov Decision Process (MDP): Action space: the action is defined as the decision of selecting sensors, i.e. state space is the set of all possible combinations; wherein and represents the duration of the data packet loss for sensor i at the estimator side or the eavesdropper side at time k; and represents receiving an indication signal; if the estimator or the eavesdropper successfully receives a data packet from sensor i at time k, otherwise ; State transition probability: the state transition probability is obtained according to the failure rate, recovery rate and related symbol representation: Define action as , to indicate that sensor i is scheduled to transmit data; The state transition probability of the estimator end is as follows: The state transition probability of the eavesdropper end is as follows: Discount factor: the discount factor γ is a parameter for balancing the current reward and future reward, and its value range is between 0 and 1. wherein represents whether the transmitted data packet successfully arrived at the remote estimator or eavesdropper at time k; Cost function: In an MDP process, a certain state is taken action and the corresponding immediate cost function is fed back, as follows: , where, , the function expression is ; the next-time immediate cost is iteratively computed as the system transitions to the next time step's state; the immediate reward function is defined as the negative immediate cost: The optimization problem translates into a minimization of the expected sum of immediate cost function and MDP optimal policy solving problem, where represents the policy taken at the time of obtaining the minimum cost, i.e. the optimal transmission scheduling policy: The optimal value function is obtained by training the Markov decision process (MDP) model using a double deep Q network (DDQN) method.
6. The eavesdropping-oriented multi-sensor information transmission scheduling method according to claim 1, characterized in that, Based on the MDP modeling, a state-action value function that maximizes the discounted average immediate reward expectation is designed based on the reinforcement learning method: According to the definition of the optimal value function, it satisfies the Bellman optimality equation: The initialization module is used to input the sensor system quantity and process parameters of each system, set the failure rate and recovery rate of the normal transmission channel and the eavesdropper channel, and set the trade-off parameter for balancing the legal estimation performance and privacy protection weight. where represents the immediate reward under the current s and a combination, and represent the next state and action, respectively, and γ is the discount factor; the optimal value function is extracted, and when the state is the optimal action satisfies the following relationship: Using D3QN reinforcement learning method to obtain optimal Q function, using deep neural network Approximate approximation estimation And improve the network structure into two independent neural networks of target network and training network, which are used for selecting action and evaluating the value of action respectively; wherein the learning weight parameters of neural network are updated using gradient descent method Corresponding The loss function of the network is constructed as: where the optimization objective in the loss function is calculated by the following equation The state value function and advantage function are output separately, and the output is summed to obtain the output value: value: where represents the advantage function , defined as the difference between the state-action value function and the state value function , i.e. .
7. The eavesdropping-oriented multi-sensor information transmission scheduling method according to claim 6, characterized in that, During the network training process, an experience replay mechanism is adopted, the state transition is stored to an experience replay buffer , and is randomly sampled during the training process for model updating to better evaluate the training effect; the value of is calculated in each training round, the value of is updated through gradient descent of the loss function, and after a fixed number of training steps, the value of is synchronized to the target network , the exploration rate is decreased , and the training of the next round is started until the training is completed.
8. A transmission scheduling system for eavesdropping-oriented multi-sensor information, characterized by, At the same time, the reinforcement learning parameter initialization setting is completed. The model construction module is used to construct a Markov decision process (MDP) model for the multi-sensor information transmission scenario under eavesdropping attack. The training module is configured to train a Markov decision process (MDP) model by using a double deep Q network (DDQN) method to obtain an optimal value function. The output module is configured to extract an optimal transmission scheduling strategy based on the optimal value function obtained after the training, and apply the strategy to a multi-sensor information transmission process to implement transmission scheduling of the multi-sensor information.
9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the steps of the transmission scheduling method for multi-sensor information facing eavesdropping according to any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the steps of the transmission scheduling method for multi-sensor information facing eavesdropping according to any one of claims 1 to 7.