Safe charging method and device for resisting attack in wireless charging network

Abstract

The invention discloses a safe charging method and device for resisting attacks in a wireless charging network. The method comprises the following steps: constructing a charging and electromagnetic radiation model considering wave interference to accurately calculate charging power and electromagnetic radiation distribution; a piecewise constant function is utilized to approximate nonlinear electromagnetic radiation, a two-dimensional region is divided into a plurality of equivalent sub-regions, and electromagnetic radiation safety constraints of all points in the region are simplified into finite constraints of a maximum value point by utilizing a maximum radiation point calculation method based on interference characteristics; converting a safe charging problem against an attack point in the wireless charging network into a double-layer optimization problem of finite constraints; a dual learning method is combined with a reinforcement learning method to realize a defense charging strategy against an optimal attack, and the strategy can realize Stackelberg equalization. According to the method, the attack on electromagnetic radiation safety is effectively defended, and the charging effectiveness is averagely improved by 40.48% compared with that of a comparison algorithm.

Description

Technical Field

[0001] The present invention relates to the field of power transmission in wireless charging networks, and more specifically, to a method for ensuring high efficiency and safety of power transmission in wireless charging networks with adversarial scenarios. Background Art

[0002] Wireless power transmission (WPT) technology in wireless charging networks refers to the wireless charger radiating electromagnetic waves to transfer energy to the rechargeable device through the air gap. With its reliability, contactless and low maintenance, WPT technology has become a commercially viable charging method. However, during the power transmission process, the surrounding environment will inevitably be exposed to a certain degree of electromagnetic radiation. High electromagnetic radiation exposure is considered to be a threat to human health and may lead to risks such as tissue damage, cardiovascular disease and brain tumors. Therefore, qualified wireless charging methods must follow electromagnetic radiation safety standards to ensure that the electromagnetic radiation intensity at any point does not exceed the safety threshold.

[0003] At present, in order to provide a stable energy supply, achieve full coverage of devices, and ensure charging continuity when charging devices fail, the deployment density of wireless chargers in wireless charging networks is usually higher than the theoretical minimum. Although this redundant design enhances the robustness of the network, it also inadvertently provides opportunities for attackers. Since wireless chargers lack physical anti-tampering measures and are usually unattended, remotely placed, and easily accessible, malicious attackers can capture and manipulate these devices, and by activating additional chargers, they can undermine the electromagnetic radiation safety constraints of the network and endanger biosafety. Existing wireless charging methods do not consider this adversarial charging scenario, and it is difficult to ensure safety after an attack. Summary of the invention

[0004] The purpose of the present invention is to provide a safe charging method and device for resisting attacks in a wireless charging network, so as to ensure that the network can still ensure the safety and practical effectiveness of electromagnetic radiation after being attacked.

[0005] In order to achieve the above invention object, the technical solution of the present invention is as follows:

[0006] In a first aspect, a secure charging method for resisting attacks in a wireless charging network comprises the following steps:

[0007] (1) establishing a charging model and an electromagnetic radiation model that consider wave interference according to the distribution of wireless charging network devices, wherein the charging model considers the phase change caused by the electromagnetic waves radiated by different chargers when reaching the rechargeable device due to the difference in path lengths, and calculates the cumulative charging power provided by multiple chargers;

[0008] (2) constructing a mathematical model for the safe charging problem against attacks in a wireless charging network based on the established charging model and electromagnetic radiation model, wherein the optimization goal of the safe charging problem against attacks in the wireless charging network is to maximize the sum of the charging utilities of all rechargeable sensors while ensuring electromagnetic radiation safety constraints, wherein the electromagnetic radiation safety constraints refer to the total amount of electromagnetic radiation at any point in the network not exceeding a predetermined safety threshold after being attacked against electromagnetic radiation safety;

[0009] (3) Using piecewise constant functions to approximate nonlinear electromagnetic radiation, the two-dimensional plane area of ​​the wireless charging network is divided into multiple equivalent electromagnetic radiation sub-areas, and the maximum radiation value point of each equivalent electromagnetic radiation sub-area is calculated according to the interference characteristics. Based on the radiation constraint of the maximum radiation value point, the safe charging problem against attacks in the wireless charging network is transformed into a two-level optimization problem with finite constraints;

[0010] (4) For the transformed two-layer optimization problem, a constrained reinforcement learning algorithm is proposed to generate defense strategies and achieve Stackelberg equilibrium to counter the optimal attack by combining primal-dual optimization with a multi-agent deep reinforcement learning framework. In the constrained reinforcement learning algorithm, the defender and attacker are regarded as agents that execute deep deterministic policy gradients, and are equipped with an execution network and a dual evaluation network. The Lagrangian relaxation technique is used to transform the constrained optimization problem into an unconstrained problem, and the primal-dual optimization method is used to iteratively update the strategy and dual variables.

[0011] In a second aspect, a secure charging strategy decision device for countering attacks in a wireless charging network includes:

[0012] A charging model and an electromagnetic radiation model construction module considering wave interference, which is used to establish a charging model and an electromagnetic radiation model considering wave interference according to the distribution of wireless charging network devices. The charging model considers the phase change caused by the electromagnetic waves radiated by different chargers when they reach the chargeable device due to the difference in path lengths to calculate the cumulative charging power provided by multiple chargers;

[0013] A mathematical model building module for safe charging problem, used to build a mathematical model for safe charging problem against attacks in wireless charging network based on the established charging model and electromagnetic radiation model, wherein the optimization goal of the safe charging problem against attacks in wireless charging network is to maximize the sum of charging utilities of all rechargeable sensors while ensuring electromagnetic radiation safety constraints, wherein the electromagnetic radiation safety constraints refer to that after suffering an attack on electromagnetic radiation safety, the total amount of electromagnetic radiation at any point in the network does not exceed a predetermined safety threshold;

[0014] A mathematical model conversion module for the safe charging problem, which is used to use a piecewise constant function to approximate nonlinear electromagnetic radiation, divide the two-dimensional plane area of ​​the wireless charging network into multiple equivalent electromagnetic radiation sub-areas, calculate the maximum radiation value point of each equivalent electromagnetic radiation sub-area according to the interference characteristics, and based on the radiation constraint of the maximum radiation value point, convert the safe charging problem against attacks in the wireless charging network into a two-level optimization problem with finite constraints;

[0015] A defense strategy generation module based on constrained reinforcement learning is used for the transformed two-layer optimization problem. It combines the primal-dual optimization with the multi-agent deep reinforcement learning framework, and proposes a constrained reinforcement learning algorithm for generating defense strategies to achieve Stackelberg equilibrium to counter the optimal attack. In the constrained reinforcement learning algorithm, the defender and the attacker are regarded as agents that execute deep deterministic policy gradients, and have an execution network and a dual evaluation network. The Lagrangian relaxation technique is used to transform the constrained optimization problem into an unconstrained problem, and the primal-dual optimization method is used to iteratively update the strategy and dual variables.

[0016] In a third aspect, the present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the method for secure charging against attacks in a wireless charging network as described above are implemented.

[0017] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method for secure charging against attacks in a wireless charging network as described above are implemented.

[0018] Beneficial effects:

[0019] (1) The present invention proposes a safe charging method against attacks in a wireless charging network. First, a charging and electromagnetic radiation model considering wave interference is constructed to accurately calculate the charging power and electromagnetic radiation distribution. Then, a piecewise constant function is used to approximate nonlinear electromagnetic radiation, and the two-dimensional region is divided into multiple equivalent sub-regions. The electromagnetic radiation safety constraints of all points in the region are simplified to finite constraints of the maximum point using the maximum radiation point calculation method based on interference characteristics, and the problem of safe charging against attack points in the wireless charging network is transformed into a two-level optimization problem with finite constraints. Finally, a dual learning method is combined with a reinforcement learning method to realize a defensive charging strategy against optimal attacks, which can achieve Stackelberg equilibrium. This method maximizes the overall charging utility and ensures electromagnetic radiation safety while responding to attacks by determining the switch state of the wireless charger.

[0020] (2) The present invention combines primal-dual optimization with a multi-agent deep reinforcement learning framework to propose a constrained reinforcement learning algorithm. The primal-dual training method can effectively manage key constraints, optimize charging utility while ensuring electromagnetic radiation safety; the multi-agent deep reinforcement learning framework can capture the complex interactions between defenders and attackers and formulate robust defense strategies; this integration enables the algorithm to strive to achieve Stackelberg equilibrium while continuously optimizing performance, and ultimately determine the optimal defense strategy in the adversarial charging scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Schematic diagram of the counter-charging scenario considering wave interference.

[0022] Figure 2 The electromagnetic radiation distribution diagram of three wireless chargers.

[0023] Figure 3 Schematic diagram for calculating the maximum radiation value point.

[0024] Figure 4 This is a framework diagram of the constrained reinforcement learning algorithm.

[0025] Figure 5 This is a comparison chart of charging effectiveness between the method of the present invention and Bi-AC, PDO, Greedy, and Random methods under different attack times k and electromagnetic radiation thresholds Rt. DETAILED DESCRIPTION

[0026] Due to the interference of electromagnetic waves, in the overlapping charging area of ​​multiple wireless chargers, the electromagnetic radiation received by the chargeable device not only weakens with increasing distance, but also fluctuates significantly due to wave interference. Figure 1 A wireless charging network in an adversarial charging scenario considering wave interference is shown. It can be seen that due to the wave interference of the two chargers, constructive interference and destructive interference appear alternately, among which constructive interference will significantly enhance the electromagnetic radiation intensity, which is easy to lead to high electromagnetic radiation exposure. The existing empirical calculation formula for electromagnetic radiation has a significant deviation from the actual value, and the error can be as high as twice the actual value. Therefore, if the interference effect is not considered, the calculated electromagnetic radiation intensity may be far lower than the actual exposure level, and safety cannot be ensured. In order to ensure the safety and actual effectiveness of electromagnetic radiation after being attacked, the present invention studies the problem of safe charging in adversarial charging scenarios, with the goal of satisfying electromagnetic radiation safety constraints in adversarial charging scenarios and maximizing the overall charging utility, activating wireless chargers, and providing efficient and safe charging services for rechargeable devices in the network. By determining the switching state of the wireless charger, the overall charging utility can be maximized and the electromagnetic radiation safety can be guaranteed while responding to attacks.

[0027] There are three main technical challenges for safe charging under adversarial attacks. The first challenge is that the safe charging problem is essentially an NP-hard problem. Since electromagnetic radiation constraints are imposed on each point on the plane, the number of constraints is infinite, and the non-convex nature of the objective function hinders the direct application of classical optimization methods. Even if these constraints are simplified to a finite set, the problem is still a multidimensional 0 / 1 knapsack problem. The second challenge is that wave interference makes the safe charging problem more complicated. While optimizing the charging utility and electromagnetic radiation safety constraints, the impact of wave interference on both must be considered. The cumulative charging utility and electromagnetic radiation of multiple chargers are not simply added, but are affected by the complex interaction of their spatial arrangement. The third challenge is to introduce adversarial attacks to transform the safe charging problem into a two-level optimization problem, which is also an NP-hard problem. In addition, adversarial attacks and wave interference problems are tightly coupled and cannot be solved independently. Therefore, solving the problem of safe charging under adversarial attacks requires dealing with multiple complex and interrelated NP-hard problems.

[0028] In view of this, in terms of problem modeling and formalization, the present invention establishes a charging and electromagnetic radiation model that takes into account wave interference, accurately calculates the charging power and electromagnetic radiation distribution of the network, considers attacks on electromagnetic radiation security, and formalizes the problem into a two-layer optimization problem. In terms of the infinite number of electromagnetic radiation security constraints, the present invention calculates the maximum radiation value points corresponding to each attack and defense strategy through regional discretization and wave interference characteristics, reducing the infinite electromagnetic radiation security constraints to a finite number. In terms of defense strategy determination, the present invention adopts the primal dual optimization and multi-agent deep reinforcement learning framework, and proposes a constrained reinforcement learning algorithm for generating defense strategies and achieving Stackelberg equilibrium to resist optimal attacks.

[0029] According to the safe charging method for dealing with attacks in a wireless charging network of the present invention, the scenario of the wireless charging network is: there are several rechargeable devices for monitoring the network in a plane, and there are some wireless chargers whose positions have been determined but whose switch states have not been determined. There are also malicious attackers in the network who can capture and manipulate a certain number of wireless chargers and change their switch states to destroy the electromagnetic radiation safety constraints. The problem is: how to determine the switch state of the wireless charger so that after being attacked, the electromagnetic radiation safety constraints are still met and the overall charging utility is maximized.

[0030] The attack-resistant safe charging method of the present invention aims to meet the electromagnetic radiation safety constraints in the attack-resistant charging scenario and maximize the overall charging utility, activate the wireless charger, and provide efficient and safe charging services for the rechargeable devices in the network, including the following steps:

[0031] (1) Establish a charging model and an electromagnetic radiation model that take into account wave interference, and then construct a mathematical model for the problem of secure charging against attacks in a wireless charging network based on the established charging model and electromagnetic radiation model;

[0032] (2) The two-dimensional region is discretized into equivalent sub-regions through electromagnetic radiation approximation, and then the maximum radiation value point is calculated according to the interference characteristics, which transforms the charging problem into a two-level optimization problem with finite constraints.

[0033] (3) Combining the primal-dual optimization with the multi-agent deep reinforcement learning framework, a constrained reinforcement learning algorithm is proposed to generate defense strategies and achieve Stackelberg equilibrium to counter the optimal attack.

[0034] The present invention performs problem modeling and formalization in step (1). The adversarial charging scenario considering wave interference in the embodiment of the present invention is specifically as follows: Assume that there are n identical wireless chargers and m identical rechargeable sensors Located in the two-dimensional plane Ω, the wireless charger radiates electromagnetic waves uniformly in all directions to wirelessly charge the rechargeable sensor. i and j Indicates wireless chargers i and rechargeable sensors o j There is also a malicious attacker in the network who attempts to cause high electromagnetic radiation exposure in some areas by turning on additional k wireless chargers.

[0035] The interference of the waves causes the distribution of electromagnetic radiation in the network to show alternating areas of increased intensity (constructive interference) and decreased intensity (destructive interference). Each area varies in shape and size, see the attached Figure 2 , which is a simulation result considering wave interference disclosed in an embodiment of the present invention, shows the complex electromagnetic radiation distribution radiated by three wireless chargers. The specific model construction method is as follows:

[0036] The charging model of a single wireless charger is established based on the widely accepted empirical charging model. i With rechargeable sensor o j The distance from the wireless charger is d. i To rechargeable sensor o j The charging power is given by:

[0037]

[0038] Among them, A0 is the wireless charger i The amplitude of the radiated electromagnetic wave, is the path loss, a and b are two predetermined constants determined by the hardware of the charger or device and the surrounding environment, and D is the wireless charger s i Maximum charging distance.

[0039] When a device is charged by multiple wireless chargers, the cumulative charging power received by the chargeable device is not the sum of the received powers from multiple chargers, but is affected by the amplitude and phase of all electromagnetic waves reaching the device. The phase difference is the phase change caused by the electromagnetic waves radiated by different chargers when they reach the chargeable device due to the difference in path length: Whenever the path lengths differ by one wavelength λ w , the phase change undergoes a cycle of 2π. i To rechargeable sensor o j The cumulative charging power is expressed as:

[0040]

[0041] For devices located in the charging area of ​​multiple chargers, the charging power they receive is not only affected by path loss, but also by the interference effect of multiple chargers. As the distance increases, the charging power gradually decreases, but due to the existence of phase difference, the power may fluctuate up and down.

[0042] Since both the charging utility and the electromagnetic radiation intensity are proportional to the power received by the device, the charging utility model is established as follows:

[0043]

[0044] Among them, x is the charger activation strategy, x i Indicates wireless chargers i Whether it is activated, C1 is a constant predetermined in the charging utility model, P ij and P kj Respectively represent chargers i Transmit to sensor j and k The charging power of the charger is U(x), and U(x) is the charging utility obtained by the charger activation strategy x.

[0045] Similarly, the electromagnetic radiation model at any position in the plane is established as follows:

[0046]

[0047] Where C2 is a predetermined constant in the electromagnetic radiation model, p is any position in the two-dimensional plane, and P ip and P kp Respectively represent chargers i and k The charging power transmitted to point p, e p (x) is the electromagnetic radiation intensity at point p.

[0048] The problem is formalized as follows. First, based on the established charging model and electromagnetic radiation model, the problem of safe charging against attacks in wireless charging networks is defined. The present invention models the problem of safe charging against attacks as a Stackelberg game process, in which the defender first determines the switch state of the charger (Stage I), and then the attacker activates additional chargers (Stage II) after observing the defense strategy to destroy the electromagnetic radiation safety:

[0049] (P1) Stage I (Defense):

[0050] Stage II (Attack):

[0051]

[0052] Among them, x and y represent the n binary variables x i and i Composed of defense and attack strategies, x i and i Respectively represent wireless chargers i Whether it is activated / attacked. Therefore, the set of activated chargers S′={s i ∈S|x i ∨y i ,i=1,2,...,n} are jointly controlled by the attack and defense strategies x and y. In the wireless charging network, the attack and defense state can be described as a two-stage Stackelberg game process, in which the defender first determines the switch state of the wireless charger. After observing the strategy, the attacker intelligently turns on additional k wireless chargers, aiming to maximize the peak value of electromagnetic radiation to destroy the electromagnetic radiation safety constraint of the network. k is the maximum number of wireless chargers that the attacker can capture, R t is a given electromagnetic radiation safety threshold. The task of the present invention is to determine the safety threshold of each wireless charger s through this two-stage Stackelberg game. i The switch state ensures that after the network is attacked, the electromagnetic radiation intensity at any position in the two-dimensional plane still does not exceed the given safety threshold R t That is, the defense strategy needs to ensure that after being attacked, the network still meets the electromagnetic radiation safety constraints while maximizing the overall charging utility of the device.

[0053] In step (2), the present invention uses regional discretization technology and interference characteristics to obtain the maximum radiation value point corresponding to each attack and defense strategy. Figure 3 , is a schematic diagram of a maximum radiation value point calculation disclosed in an embodiment of the present invention. A maximum radiation value point calculation method disclosed in an embodiment of the present invention comprises the following specific steps:

[0054] The first step is to approximate the electromagnetic radiation intensity. Let e(d) represent the electromagnetic radiation when the charging distance is d, and use the piecewise constant function Approximate electromagnetic radiation:

[0055]

[0056] in is the approximate value of e(d). l(0)=0,l(J)=D,D is the maximum charging distance,l(q)=β((1+∈) q / 2 -1), (q=1,2,...,Q-1), β and ∈ are constants, with each wireless charger s i With q as the center, concentric circles with radii l(1), l(2), …, l(Q) are drawn, where l(q) represents the radius of the qth concentric circle of the charger and Q represents the total number of concentric circles. Thus, the two-dimensional plane is divided into several equivalent electromagnetic radiation sub-areas by these circles, such as Figure 3 As shown in (a) in .

[0057] The second step is to use the interference characteristics to draw constructive interference curves corresponding to each attack and defense strategy. Specifically, the electromagnetic waves radiated by the wireless charger will affect the electromagnetic radiation intensity in the network due to the interference effect. When the phase difference between two or more chargers is an even multiple of half a wavelength, constructive interference will be formed; when the phase difference is an odd multiple of half a wavelength, destructive interference will be formed. Determine the switch state of the charger according to the attack and defense strategy, and draw constructive interference curves of all turned-on chargers determined by the attack and defense strategy to show the points in the network where electromagnetic radiation is significantly enhanced due to the interference effect, such as Figure 3 As shown in (b) in .

[0058] The third step is to calculate the intersection points of the constructive interference curves in each equivalent electromagnetic radiation sub-area to identify several points where the path attenuation is significantly enhanced due to the interference effect, and calculate the electromagnetic radiation intensity of these points, and finally determine the point with the highest intensity as the maximum radiation value point, such as Figure 3 As shown by the red star in the figure.

[0059] Therefore, the electromagnetic radiation safety constraints of all points in the two-dimensional plane of problem P1 are transformed into electromagnetic radiation of the maximum radiation value point The safety constraint is This converts P1 with an infinite number of constraints into P2 with a finite number of constraints:

[0060]

[0061] In step (3), the present invention adopts a defense strategy based on primal-dual optimization and reinforcement learning algorithm to determine the optimal attack. Figure 4, which is a framework diagram of a constrained reinforcement learning algorithm for resisting attacks disclosed in an embodiment of the present invention. A constrained reinforcement learning algorithm for resisting attacks disclosed in an embodiment of the present invention has the following specific steps:

[0062] In wireless charging networks, the decision-making of defender and attacker strategies is constructed as a Markov game model. in, is the state space, is the discrete action space of agent i, is the joint action space of the agents, is a probability transfer mechanism. Agent i receives rewards and incurs costs according to its state and behavior (respectively represented by the reward function and the cost function Definition). The goal of agent i is to maximize its total expected reward and minimize its total expected cost where γ∈[0,1] is a discount factor used to balance long-term and short-term returns, and T is the time horizon.

[0063] In the established constrained reinforcement learning algorithm, the defender and attacker are respectively agent 1 and agent 2, each with an execution network and a dual evaluation network. i Approximate deterministic policy π i The dual evaluation network includes a reward evaluation network Q i (s,a|φ i ) and cost review network C i (s,a|ξ i ). Where a is the joint action of the defender and attacker agents.

[0064] According to the characteristics of the Stackelberg game, the agent makes decisions and collects experience in the following order:

[0065]

[0066] stC1(s′,a1,a2)≤b1.

[0067] a′2←π2(s′,a1′;θ2),

[0068] stC2(s′,a1′,a2)≤b2,

[0069] Among them, b1 and b2 are the maximum thresholds of the defender and attacker's respective constraints. Specifically, the defender's strategy aims to select actions that maximize the charging utility increment while ensuring that after execution, the electromagnetic radiation intensity at the maximum radiation value point does not exceed the specified threshold R tThe attacker's goal is to maximize the peak value of network electromagnetic radiation under the constraint of a limited number of attacks k. To ensure that the subsequent attack and defense strategies reach the Tuckerberg equilibrium, the defender considers all possible defense-attack combinations when making decisions.

[0070] The Lagrangian relaxation technique is used to transform the constrained optimization problem into the following unconstrained problem:

[0071]

[0072] Among them, L i (π,λ) is the Lagrangian penalty function that combines the cost term with the original long-term reward. The Lagrangian multiplier λ dynamically balances the trade-off between reward and cost, ensuring that the constraints are met without affecting the overall goal.

[0073] The primal-dual optimization method is used to iteratively update the agent strategy π and the dual variable λ:

[0074]

[0075] Among them, α i and β i are the learning rates of the policy and dual variables, θ i are policy network parameters, is the objective function L i (π,λ) for the policy parameters θ i The gradient of , which guides the optimization direction of the strategy. In addition, b is the threshold of the cost constraint, [x] + =max{0,x} is the projection onto the dual space.

[0076] The agent replays the experience buffer A batch of experience samples is randomly drawn from Based on these samples, the policy gradient can be estimated by calculating the expected value. The specific calculation formula is as follows:

[0077]

[0078] At the same time, the agent uses the reward r obtained from the experience sample i and cost c i Update the evaluation network parameters, evaluate each action, and update the parameters of the reward evaluation network by minimizing the temporal difference error:

[0079]

[0080] The target value y i for:

[0081] The parameters of the cost evaluation network are also updated by minimizing the temporal difference error:

[0082]

[0083] The target value g i for:

[0084] To improve learning stability, the parameters of the target network (including the policy network π i ′、Reward value network Q i ′ and cost value network C i ′) Use the delay parameter for soft update: θ′←τθ+(1-τ)θ i ′,φ′←τφ+(1-τ)φ i ′,ξ′←τξ+(1-τ)ξ i ′, where τ is a small constant used to control the update rate of the target network parameters.

[0085] The constrained reinforcement learning algorithm uses a centralized training and decentralized execution framework. During execution, the defender agent maximizes the Lagrangian penalty function based on local observations, and then the attacker agent makes the best response after observing the defender's strategy. The agent collects experience through interaction with the environment and stores it in the experience replay buffer. When the environment reaches a terminal state (i.e., the defense and attack strategies are determined), it is reset. During training, the agent optimizes the strategy by sharing experience. The evaluation network calculates the temporal difference error and updates the parameters based on the reward, cost, and observations. The agent updates the policy parameters using the sampled policy gradient and adjusts the Lagrange multiplier to balance reward maximization with cost minimization. The algorithm stabilizes the training process by iteratively updating the target network parameters. These steps enable the agent to learn the optimal charger activation strategy, effectively defend against attacks, and maximize charging utility. The pseudo code of the algorithm is as follows.

[0086] Algorithm 1: Constrained reinforcement learning algorithm for solving the problem of safe charging against attacks

[0087] Initialization: Policy network parameters θ = [θ i ] i∈{1,2} , reward and cost evaluation network parameters φ = [φ i ] i∈{1,2} and ξ=[ξ i ] i∈{1,2} ;

[0088] The target network parameters are initialized to be equal to the main network parameters: θ′←θ, φ′←φ, ξ′←ξ;

[0089] Initialize the Lagrange multiplier: λ = [λ i ];

[0090] 1 For each round e=1,...,M: % Experience collection phase

[0091] 2 Each agent i∈{1,2} observes state s and takes action a i , and receive a reward i and cost c i ;

[0092] 3. Execute action a and observe the next state s′;

[0093] 4(s,a i ,r i ,c i ,s′) is stored in the playback buffer

[0094] 5 If s′ is a terminal state, reset the environment state;

[0095] 6 For each step t=1,...,H: % parameter update phase

[0096] 7 from Randomly sample a batch of transformation tuples from

[0097] 8 Calculate target reward and cost: y i =r i +γQ i ′(s′,a1′,a′2),g i =c i +γC i ′(s′,a1′,a′2);

[0098] 9 Update the reward-evaluation network by minimizing the loss:

[0099] 10 Update the cost evaluation network by minimizing the loss:

[0100] 11 Update the policy using the sampled policy gradient:

[0101] 12 Update the Lagrange multiplier:

[0102] 13 Update target network: θ′ i ←τθ i +(1-τ)θ′ i ,φ′ i ←τφ i +(1-τ)φ′ i ,ξ′ i ←τξ i+(1-τ)ξ′ i .

[0103] In order to verify the performance of the method of the present invention, a comparative experiment was conducted. The experimental results are shown in Figure 5 , which are the differences between the proposed method (CARL) and the bi-layer actor-critic method (Bi-AC), the primal dual optimization method (PDO), the greedy method (Greedy) and the random method (Random) in the attack number k ( Figure 5 (a)) and the electromagnetic radiation safety threshold R t ( Figure 5 In addition to the changed parameters, the other default parameters are set to: n = 20, m = 50, D = 4m, a = 10, b = 10, C1 = 0.01, C2 = 1, H = 1000, γ = 0.9. The experimental results show that the anti-attack safe charging method of the present invention is significantly better than the Bi-AC, PDO, Greedy, and Random methods. On average, the number of attacks k and the electromagnetic radiation safety threshold R t The present invention is superior to the other four comparison methods by 34.66% and 46.3% respectively. The present invention can maintain a high charging utility in confrontation scenarios and shows excellent robustness.

[0104] Based on the same technical concept as the method embodiment, another embodiment of the present invention further provides a safe charging strategy decision device for resisting attacks in a wireless charging network, comprising:

[0105] A charging model and an electromagnetic radiation model construction module considering wave interference, which is used to establish a charging model and an electromagnetic radiation model considering wave interference according to the distribution of wireless charging network devices. The charging model considers the phase change caused by the electromagnetic waves radiated by different chargers when they reach the chargeable device due to the difference in path lengths to calculate the cumulative charging power provided by multiple chargers;

[0106] A mathematical model building module for safe charging problem, used to build a mathematical model for safe charging problem against attacks in wireless charging network based on the established charging model and electromagnetic radiation model, wherein the optimization goal of the safe charging problem against attacks in wireless charging network is to maximize the sum of charging utilities of all rechargeable sensors while ensuring electromagnetic radiation safety constraints, wherein the electromagnetic radiation safety constraints refer to that after suffering an attack on electromagnetic radiation safety, the total amount of electromagnetic radiation at any point in the network does not exceed a predetermined safety threshold;

[0107] A mathematical model conversion module for the safe charging problem, which is used to use a piecewise constant function to approximate nonlinear electromagnetic radiation, divide the two-dimensional plane area of ​​the wireless charging network into multiple equivalent electromagnetic radiation sub-areas, calculate the maximum radiation value point of each equivalent electromagnetic radiation sub-area according to the interference characteristics, and based on the radiation constraint of the maximum radiation value point, convert the safe charging problem against attacks in the wireless charging network into a two-level optimization problem with finite constraints;

[0108] A defense strategy generation module based on constrained reinforcement learning is used for the transformed two-layer optimization problem. It combines the primal-dual optimization with the multi-agent deep reinforcement learning framework, and proposes a constrained reinforcement learning algorithm for generating defense strategies to achieve Stackelberg equilibrium to counter the optimal attack. In the constrained reinforcement learning algorithm, the defender and the attacker are regarded as agents that execute deep deterministic policy gradients, and have an execution network and a dual evaluation network. The Lagrangian relaxation technique is used to transform the constrained optimization problem into an unconstrained problem, and the primal-dual optimization method is used to iteratively update the strategy and dual variables.

[0109] It should be understood that the safe charging strategy decision device for resisting attacks in the wireless charging network in the embodiment of the present invention can implement all the technical solutions in the above method embodiment, and the functions of its various functional modules can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can refer to the relevant description in the above embodiment, which will not be repeated here.

[0110] The present invention also provides a computer device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the safe charging method for resisting attacks in a wireless charging network as described above are implemented.

[0111] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method for secure charging against attacks in a wireless charging network as described above are implemented.

[0112] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices (systems), computer equipment or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0113] The present invention is described with reference to a flowchart of a method according to an embodiment of the present invention. It should be understood that each process in the flowchart and a combination of processes in the flowchart can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A device that specifies functions in a process or multiple processes.

[0114] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A function specified in a process or multiple processes.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 The steps of a specified function in a process or multiple processes.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A safe charging method against attacks in a wireless charging network, characterized in that: The following steps are involved: (1) establishing a charging model and an electromagnetic radiation model that consider wave interference according to the distribution of wireless charging network devices. The charging model considers the phase change caused by the electromagnetic waves radiated by different chargers when they reach the rechargeable device due to the difference in path lengths, and calculates the cumulative charging power provided by multiple chargers; (2) constructing a mathematical model for the safe charging problem against attacks in a wireless charging network based on the established charging model and electromagnetic radiation model, wherein the optimization goal of the safe charging problem against attacks in the wireless charging network is to maximize the sum of the charging utilities of all rechargeable sensors while ensuring electromagnetic radiation safety constraints, wherein the electromagnetic radiation safety constraints refer to the total amount of electromagnetic radiation at any point in the network not exceeding a predetermined safety threshold after being attacked against electromagnetic radiation safety; (3) Using piecewise constant functions to approximate nonlinear electromagnetic radiation, the two-dimensional plane area of ​​the wireless charging network is divided into multiple equivalent electromagnetic radiation sub-areas, and the maximum radiation value point of each equivalent electromagnetic radiation sub-area is determined according to the interference characteristics. Based on the radiation constraint of the maximum radiation value point, the safe charging problem against attacks in the wireless charging network is converted into a two-level optimization problem with finite constraints; (4) For the transformed two-layer optimization problem, a constrained reinforcement learning algorithm is proposed to generate defense strategies and achieve Stackelberg equilibrium to counter the optimal attack by combining primal-dual optimization with a multi-agent deep reinforcement learning framework. In the constrained reinforcement learning algorithm, the defender and attacker are regarded as agents that execute deep deterministic policy gradients, and are equipped with an execution network and a dual evaluation network. The Lagrangian relaxation technique is used to transform the constrained optimization problem into an unconstrained problem, and the primal-dual optimization method is used to iteratively update the strategy and dual variables.

2. The method according to claim 1, characterized in that Assume there are n identical wireless chargers and m identical rechargeable sensors Located in the two-dimensional plane Ω, the charging model is expressed as follows: Among them, x is the charger activation strategy, x i Indicates wireless chargers i Whether it is activated, C1 is a constant predetermined in the charging model; P ij and P kj Respectively represent chargers i Transmit to sensor j and k The charging power of the charger is U(x), U(x) is the charging utility obtained by the charger activation strategy x; Δφ is the phase change caused by the electromagnetic waves radiated by different chargers when they reach the rechargeable device due to the difference in path length; The electromagnetic radiation model at any position p in the plane is expressed as follows: Where C2 is a constant predetermined in the electromagnetic radiation model, P ip and P kp Respectively represent chargers i and k The charging power transmitted to point p, e p (x) is the electromagnetic radiation intensity at point p.

3. The method according to claim 1, characterized in that The problem of secure charging against attacks in wireless charging networks is defined as follows: (P1)StageI(Defense): StageII(Attack): Among them, x and y are the charger activation strategy and attack strategy. The charger activation strategy is used as the defense strategy. x and y are respectively composed of n binary variables x i and i Composition, indicating wireless chargers i Whether it is activated / attacked, the activated charger set S′={s i ∈S|x i ∨y i , i=1,2,...,n} is controlled by x and y; U(x,y) is the charging utility under x and y strategies, e p (x, y) is the electromagnetic radiation intensity at point p under the x and y strategies; Problem P1 describes the attack and defense state in the wireless charging network as a Stackelberg game process. The defender Defense determines the charger activation strategy in the first stage StageI, and the attacker Attack observes the strategy in the second stage StageII and intelligently turns on additional k wireless chargers, with the goal of maximizing the electromagnetic radiation peak value to destroy the electromagnetic radiation safety constraints of the network. t is a given electromagnetic radiation safety threshold.

4. The method according to claim 1, characterized in that: The method for determining the maximum radiation value point includes: The first step is to approximate the electromagnetic radiation intensity. Use e(d) to represent the electromagnetic radiation when the charging distance is d, and use a piecewise constant function to approximate the electromagnetic radiation intensity: in is the approximate value of e(d), with each wireless charger s i With q as the center, draw concentric circles with radii l(1), l(2), …, l(Q) respectively, where l(q) represents the radius of the qth concentric circle of the charger, Q represents the total number of concentric circles, l(q) = β((1+ε) q / 2 -1), β and ∈ are constants; The second step is to use the interference characteristics to draw the constructive interference curve corresponding to each attack and defense strategy. The switch state of the charger is determined according to the attack and defense strategy, and the constructive interference curve of all turned-on chargers is drawn to show the points in the network where electromagnetic radiation is significantly enhanced due to the interference effect. The third step is to calculate the intersection points of the constructive interference curves in each equivalent electromagnetic radiation sub-area to identify several points where the path attenuation is significantly enhanced due to the interference effect, and calculate the electromagnetic radiation intensity of these points, and determine the point with the highest intensity as the maximum radiation value point.

5. The method according to claim 3, characterized in that: The problem of secure charging against attacks in the wireless charging network is transformed into a two-level optimization problem with finite constraints, which can be expressed as: (P2) s.t.e p# (x,y)≤R t , in, Indicates electromagnetic radiation at the point of maximum radiation value.

6. The method according to claim 1, characterized in that The constrained reinforcement learning algorithm adopts a centralized training and decentralized execution framework. During the execution process, the defender agent maximizes the Lagrangian penalty function based on local observations, and the attacker agent responds after observation; the agent collects experience by interacting with the environment and stores it in the experience playback buffer; it is reset when the environment reaches the terminal state, that is, the strategy is determined; During the training process, the agents share experience to optimize the strategy, and the evaluation network updates the parameters through the temporal difference error. The agents use the policy gradient to update the policy parameters and adjust the Lagrange multiplier to balance the reward and cost. The training process iteratively updates the target network parameters, allowing the agent to learn the optimal charger activation strategy, effectively defend against optimal attacks, and maximize the overall charging utility.

7. The method according to claim 1, characterized in that The constraint reinforcement learning algorithm includes: the attack and defense strategy decision is constructed as a Markov game model in, is the state space, is the discrete action space of agent i, is the joint action space of the agents, It is a probabilistic transfer mechanism. Agent i receives rewards and incurs costs according to its state and behavior. The goal of agent i is to maximize its total expected return. and minimize its total expected cost where r i : c i : γ∈[0,1] is the discount factor, T is the time horizon; The agent adopts the neural network parameters θ i Approximate deterministic policy π i ; The dual evaluation network includes the reward evaluation network Q i (s,a|φ i ) and cost review network C i (s,a|ξ i ), where a is the joint action of the agent; the agent's goal is to i To maximize the expected return, the decision process is as follows: stC1(s′,a1,a2)≤b1. a′2←π2(s′,a1′;θ2), stC2(s′,a1′,a2)≤b2. The Lagrangian relaxation technique is used to transform the constrained optimization problem into the following unconstrained problem: Among them, L i (π,λ) is the Lagrangian penalty function, which combines the cost term with the original long-term reward. The Lagrangian multiplier λ dynamically balances the trade-off between reward and cost to ensure that the constraints are met without affecting the overall goal. The primal-dual optimization method is used to iteratively update the strategy π and the dual variable λ: Among them, α i and β i are the learning rates of the policy and dual variables, respectively, is the objective function L i (π,λ) for the policy parameters θ i The gradient of , b is the threshold of the cost constraint, [x] + =max{0,x} is the projection on the dual space; The agent’s policy gradient is defined as: in, Represents the playback buffer An empirical data extracted from Record experience samples of all agents; The parameters of the reward-criteria network are updated by minimizing the temporal difference error: The target value y i for: The parameters of the cost evaluation network are updated as follows: The target value g i for: The parameters of the target network are soft-updated using the delay parameter: θ′←τθ+(1-τ)θ i ′,φ′←τφ+(1-τ)φ i ′,ξ′←τξ+(1-τ)ξ i ′, where τ is a constant used to control the update rate of the target network parameters.

8. A safe charging strategy decision device for countering attacks in a wireless charging network, characterized in that: include: A charging model and an electromagnetic radiation model construction module considering wave interference, which is used to establish a charging model and an electromagnetic radiation model considering wave interference according to the distribution of wireless charging network devices. The charging model considers the phase change caused by the electromagnetic waves radiated by different chargers when they reach the chargeable device due to the difference in path lengths to calculate the cumulative charging power provided by multiple chargers; A mathematical model building module for safe charging problem, used to build a mathematical model for safe charging problem against attacks in wireless charging network based on the established charging model and electromagnetic radiation model, wherein the optimization goal of the safe charging problem against attacks in wireless charging network is to maximize the sum of charging utilities of all rechargeable sensors while ensuring electromagnetic radiation safety constraints, wherein the electromagnetic radiation safety constraints refer to that after suffering an attack on electromagnetic radiation safety, the total amount of electromagnetic radiation at any point in the network does not exceed a predetermined safety threshold; A mathematical model conversion module for the safe charging problem, which is used to use a piecewise constant function to approximate nonlinear electromagnetic radiation, divide the two-dimensional plane area of ​​the wireless charging network into multiple equivalent electromagnetic radiation sub-areas, calculate the maximum radiation value point of each equivalent electromagnetic radiation sub-area according to the interference characteristics, and based on the radiation constraint of the maximum radiation value point, convert the safe charging problem against attacks in the wireless charging network into a two-level optimization problem with finite constraints; A defense strategy generation module based on constrained reinforcement learning is used for the transformed two-layer optimization problem. It combines the primal-dual optimization with the multi-agent deep reinforcement learning framework, and proposes a constrained reinforcement learning algorithm for generating defense strategies to achieve Stackelberg equilibrium to counter the optimal attack. In the constrained reinforcement learning algorithm, the defender and the attacker are regarded as agents that execute deep deterministic policy gradients, and have an execution network and a dual evaluation network. The Lagrangian relaxation technique is used to transform the constrained optimization problem into an unconstrained problem, and the primal-dual optimization method is used to iteratively update the strategy and dual variables.

9. A computer device, characterized in that: include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and when the programs are executed by the processors, the steps of the method for secure charging against attacks in a wireless charging network as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for secure charging against attacks in a wireless charging network according to any one of claims 1 to 7 are implemented.

Images