Vehicle signal energy power distribution method and device based on deep deterministic policy gradient

Abstract

The present application relates to the technical field of Internet of Vehicles, in particular to a vehicle information and energy power distribution method and device based on deep deterministic policy gradient. The method comprises: in view of the problem that the information transmission rate and energy consumption are difficult to balance in the energy-carrying communication of electric buses, a Markov decision process model of urban Internet of Vehicles environment is established, and a vehicle information and energy power distribution method based on deep deterministic policy gradient is proposed. By using the Actor-Critic architecture of DDPG algorithm, combining with the experience replay and soft update mechanism, the power distribution is adaptively optimized, the problem that the traditional distribution strategy is difficult to adapt to the real-time change of traffic environment is solved, and the dynamic balance of communication efficiency and energy consumption is realized. Under the premise of meeting the electric energy required for the normal operation of the bus, the information transmission rate is maximized, and a feasible scheme is provided for the efficient and energy-saving communication of urban electric buses.

Description

Technical Field

[0001] This invention relates to the field of vehicle networking technology, and in particular to a method and apparatus for vehicle information and power allocation based on deep deterministic policy gradient. Background Technology

[0002] The rise of Vehicle-to-Infrastructure (V2I) and Vehicle-to-Vehicle (V2V) communication technologies has provided vehicle users with a more convenient driving experience and richer entertainment services. However, with the increasing number of vehicles and the continuous growth of communication demands, existing vehicle communication systems face significant challenges in terms of communication efficiency and energy consumption. This not only affects vehicle operating efficiency but may also lead to energy waste and communication interruptions, seriously impacting traffic safety and user experience.

[0003] Simultaneous Wireless Information and Power Transfer (SWIPT) technology offers a viable solution for vehicle communication, allowing vehicles to acquire energy while receiving information, reducing reliance on traditional charging infrastructure. This technology is suitable for vehicles with fixed routes, such as buses in urban transportation, enabling them to acquire energy from roadside units (RSUs) while simultaneously conducting data communication, thus improving energy efficiency and communication continuity.

[0004] While SWIPT technology offers advantages such as reducing reliance on fixed charging stations and improving energy efficiency, existing vehicle-to-everything (V2X) systems still suffer from insufficient dynamic power allocation capabilities and limited adaptability to real-time changes. To achieve a balance between vehicle communication and energy acquisition, it's necessary to rationally allocate vehicle information and energy reception resources. However, current power allocation strategies are mostly static, making it difficult to dynamically adjust based on real-time communication needs and energy status. This static power allocation method can lead to low communication efficiency or insufficient energy supply, severely impacting normal vehicle operation.

[0005] In urban traffic environments, vehicle communication needs and energy consumption are influenced by various factors, such as traffic flow, traffic lights, and road conditions. These highly dynamic and uncertain factors limit vehicle operation during extended periods or in high-demand scenarios. Therefore, it is necessary to design a dynamic power allocation method that can adapt to these changes in real time to optimize vehicle communication efficiency and energy consumption. Summary of the Invention

[0006] To address the technical problem of highly dynamic and uncertain factors in existing technologies that limit vehicle operation under long-term or high-demand scenarios, this invention provides a vehicle information and power allocation method and apparatus based on a deep deterministic strategy gradient. The technical solution is as follows:

[0007] On the one hand, a vehicle information energy power allocation method based on deep deterministic policy gradient is provided, characterized in that the method includes:

[0008] S1. Initialize parameter information;

[0009] S2. Establish a vehicle-based energy-carrying communication model;

[0010] S3. Transform the vehicle energy-carrying communication model into a vehicle power distribution MDP model.

[0011] S4. Based on the initialized parameter information, a bus power allocation method based on DDPG is designed, and the vehicle energy-carrying communication model is solved to obtain an approximately optimal bus energy-carrying power allocation decision.

[0012] Optionally, in S1, the initialization parameter information includes:

[0013] Initialize road environment information, obtain the total number of buses B and the distance d from the RSU. b,r Road environment information; initialization of θ, ω and replay memory pool D neural network parameters, MDP model state.

[0014] Optionally, in S2, a vehicle-based energy-carrying communication model is established, including:

[0015] With κ as the energy charging efficiency coefficient, the power components of the radio frequency energy received and the information transmitted by the receiver of bus b from RSUr can be expressed as follows:

[0016] Ep r,b (t)=κp r,b (t)g b,r (t)δ b

[0017] Ip r,b (t)=p r,b (t)-Ep r,b (t)

[0018] Where, δ b ∈[0,1] represents the power allocation coefficient of the receiver on bus b, i.e., the proportion of radio frequency signal used for information reception and energy charging; p r,b (t) represents the total power received by bus b from RSUr within time slot t; Ep r,b (t) The receiver on bus b in time slot t receives the RF energy fractional power from RSUr; Ipr,b This indicates the power division of information received by the receiver of time slot bus b from RSUr;

[0019] Then the SINR of the information power allocation received by bus b from signal source RSUr is:

[0020]

[0021] Where, σ 2 and These represent the power of Gaussian white noise and signal processing noise, respectively; γ r,b (t) represents the SINR of the information power allocation received by bus b from signal source RSUr; C represents the total number of other vehicles communicating simultaneously with RSUr; g r,b (t) represents the channel gain between RSUr and bus b; c represents another vehicle communicating simultaneously with RSUr; g c,r (t) represents the channel gain between other vehicles and RSUr;

[0022] The transmission rate R between bus b and RSUr b,r (t) is:

[0023] R b,r (t)=W b,r log2(1+γ b,r (t)(1-δ b ))

[0024] Among them, W b,r This represents the bandwidth between bus b and RSUr.

[0025] Optionally, in S3, the vehicle energy-carrying communication model is transformed into a vehicle power distribution MDP model, including:

[0026] When the battery level drops to 0, the bus stops running, terminating the DDPG training loop; at this point, the state-space expression S is:

[0027] S = {batt1,...,batt} b ,...,batt B};

[0028] Among them, the battery level of the first bus in batt1, batt B This indicates the battery level of the B-th bus;

[0029] In each time slot, a certain proportion of the wireless charging power is allocated to the bus, and the remaining power is used for communication. The action space expression is as follows. for:

[0030]

[0031] When the bus's battery level drops to negative, a penalty is applied to reduce the frequency with which the agent explores in that direction; then the reward function... for:

[0032]

[0033] Optionally, in S4, a bus power allocation method based on DDPG is designed, including:

[0034] Obtain initial road environment information, including the number of buses (B) and the distance to the RSU (Roadside Unit). b,r Road environment information, initialization of θ, ω, and replay memory pool D via network parameters, and MDP model state.

[0035] In each iteration, based on the current state and selected action Calculate the bus throughput RT over a period of time and obtain the reward value. and the next state

[0036] Will Stored in replay memory pool D;

[0037] When D is full, randomly draw a small batch of samples from it to update the policy gradient of the Actor network. The Q-value y of the Critic network t The target network parameters θ′ and ω′ are updated using the loss function Loss and a soft update mechanism; then, the state is updated to the next state S. t ←S t+1 ;

[0038] Repeat the above process until the bus's battery is depleted or the preset number of iterations N is reached, to obtain an approximately optimal bus power allocation decision Ep*={Ep1*,...,Ep b *,...,Ep B *}

[0039] Optionally, the action space expression A t for:

[0040] A t =μ(S) t |θ)+ε.

[0041] On the other hand, a vehicle information and energy power allocation device based on deep deterministic policy gradient is provided. This device implements the aforementioned vehicle information and energy power allocation method based on deep deterministic policy gradient. The device includes:

[0042] The parameter initialization module is used to initialize parameter information;

[0043] The communication model establishment module is used to establish a vehicle energy-carrying communication model;

[0044] The MDP model conversion module is used to convert the vehicle energy-carrying communication model into the vehicle power distribution MDP model.

[0045] The allocation decision module is used to design a bus power allocation method based on DDPG based on the initialized parameter information, solve the vehicle energy-carrying communication model, and obtain an approximately optimal bus energy-carrying power allocation decision.

[0046] Optionally, a parameter initialization module is used to initialize road environment information, such as the total number of buses B and the distance d from the RSU. b,r Road environment information; initialization of θ, ω and replay memory pool D neural network parameters, MDP model state.

[0047] On the other hand, a vehicle information and power allocation device based on deep deterministic policy gradient is provided. The vehicle information and power allocation device based on deep deterministic policy gradient includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, they implement any of the methods described above for vehicle information and power allocation based on deep deterministic policy gradient.

[0048] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods of vehicle energy allocation based on deep deterministic policy gradient.

[0049] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0050] This invention proposes a vehicle energy allocation method based on Deep Deterministic Policy Gradient (DDPG). By constructing a Markov Decision Process (MDP) model for urban vehicle-powered communication, a city bus is used as the agent, with its remaining battery power as the state space and the wireless charging power allocated to the bus as the action space. The bus's transmission rate is set as the reward function. The optimal power allocation strategy is solved using a DDPG-based algorithm to achieve a balance between bus communication efficiency and energy consumption. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart illustrating the vehicle information power allocation method based on deep deterministic policy gradient provided in an embodiment of the present invention;

[0053] Figure 2 A block diagram of a vehicle information and power allocation device based on deep deterministic policy gradient provided in an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0055] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0056] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0057] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0058] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0059] This invention provides a vehicle information and power allocation method based on deep deterministic policy gradient. This method can be implemented by a vehicle information and power allocation device based on deep deterministic policy gradient, which can be a terminal or a server. Figure 1 The flowchart shown is for a vehicle information energy power allocation method based on deep deterministic policy gradient. Figure 1 As shown, the vehicle information power allocation method based on deep deterministic policy gradient proposed in this invention may include the following steps in its processing flow:

[0060] S1. Initialize parameter information;

[0061] In one feasible implementation, in S1, initializing parameter information includes:

[0062] Initialize road environment information, obtain the total number of buses B and the distance d from the RSU. b,r Road environment information; initialization of θ, ω and replay memory pool D neural network parameters, MDP model state.

[0063] S2. Establish a vehicle-based energy-carrying communication model;

[0064] In one feasible implementation, in S2, a vehicle-powered communication model is established, including:

[0065] Suppose that the number of buses within a city's vehicle area in time slot t follows a Poisson distribution, then the probability density function of the buses is:

[0066]

[0067] Where λ(t) represents the average arrival value of the sampled vehicles. λ represents the total number of vehicles in the time slot; Pb represents the probability density function of the bus; X represents the number of buses sampled in time slot t, which follows a Poisson distribution with parameter λ(t); e is a mathematical constant appearing in the formula for the Poisson distribution, and is the base of the natural logarithm; e -λ(t) This represents the normalization factor of the Poisson distribution, ensuring that the sum of the probabilities of all possible events is 1.

[0068] Suppose there are A RSUs and B buses in this area. Establish a rectangular coordinate system on the traffic map, and within time slot t, the coordinates of bus b and RSU r are respectively (X... b ,Y b ) and (X r ,Y r Therefore, their Euclidean distance d b,r (t) is:

[0069]

[0070] Let cg0 be the channel gain at the initial reference distance d0. Let α represent an exponentially distributed random variable. pl Let represent the channel path loss exponent. Then the channel gain between them is:

[0071]

[0072] in, This represents the path loss index between bus b and RSUr. Let g represent an exponentially distributed random variable; g0 represents the channel gain at the initial reference distance.

[0073] Assuming that C other vehicles are simultaneously communicating with RSUr, the expression for the signal-to-interference-plus-noise ratio (SINR) between bus b and RSUr is:

[0074]

[0075] Where, p b and p c Let g represent the transmission power of bus b and interfering vehicle c, respectively. b,r and g c,r Let σ represent the channel gain from bus b and interfering vehicle c to RSUr, respectively. 2 This represents Gaussian white noise.

[0076] Let W b,r Let the communication bandwidth between bus b and RSUr be denoted as , then the transmission rate between them is:

[0077] R b,r (t)=W b,r log2(1+γ b,r (t)) (5)

[0078] In urban transportation, buses that provide communication services to vehicles often face energy challenges such as rapid battery depletion, sometimes even running out of power before reaching a charging station. To address this issue, SWIPT technology allows buses to wirelessly charge using the magnetic resonance between the RSU transmitter and the bus receiver. By employing a power distribution mode to rationally allocate the radio frequency signal, the bus can charge while receiving information, effectively improving energy efficiency, reducing reliance on traditional charging stations, and providing a more reliable energy guarantee for the continuous operation of buses.

[0079] With κ as the energy charging efficiency coefficient, the power components of the radio frequency energy received and the information transmitted by the receiver of bus b from RSUr can be expressed as follows:

[0080] Ep r,b (t)=κp r,b (t)g b,r (t)δ b (6)

[0081] Ip r,b (t)=p r,b (t)-Ep r,b (t) (7)

[0082] Where, δ b∈[0,1] represents the power allocation coefficient of the receiver on bus b, i.e., the proportion of radio frequency signal used for information reception and energy charging; p r,b (t) represents the total power received by bus b from RSUr within time slot t; Ep r,b (t) represents the RF energy fractional power received by the receiver of bus b in time slot t from RSUr; r represents a specific RSU; Ip r,b This indicates the power division of information received by the receiver of time slot bus b from RSUr;

[0083] Then the SINR of the information power allocation received by bus b from signal source RSUr is:

[0084]

[0085] Where, σ 2 and These represent the power of Gaussian white noise and signal processing noise, respectively; γ r,b (t) represents the SINR of the information power allocation received by bus b from signal source RSUr; C represents the total number of other vehicles communicating simultaneously with RSUr; g r,b (t) represents the channel gain between RSUr and bus b; c represents another vehicle communicating simultaneously with RSUr; g c,r (t) represents the channel gain between other vehicles and RSUr;

[0086] The transmission rate R between bus b and RSUr b,r (t) is:

[0087] R b,r (t)=W b,r log2(1+γ b,r (t)(1-δ b (9)

[0088] Among them, W b,r This represents the bandwidth between bus b and RSUr.

[0089] To maintain the normal operation of the bus, its battery level needs to be constantly checked. A power allocation model is used to maximize information transmission rate while ensuring the necessary electrical energy for normal bus operation. The optimization problem expression is:

[0090]

[0091] Where B represents the total number of buses in the system; constraint C1 represents the communication power Ip in the bus SWIPT. b and charging power Ep b The total amount is fixed; constraint C2 ensures that the battery power is sufficient for the operation of the bus.

[0092] S3. Transform the vehicle energy-carrying communication model into a vehicle power distribution MDP model.

[0093] In one feasible implementation, in S3, the vehicle energy-carrying communication model is transformed into a vehicle power distribution MDP model, including:

[0094] Before applying the DDPG algorithm to solve the power allocation problem for buses, it is necessary to construct an MDP model that includes an agent, state space, action space, and reward function to formally describe the decision problem.

[0095] Intelligent Agent: All buses in this vehicle-to-everything (V2X) system are considered as an intelligent agent.

[0096] State Space: In each time slot, the system state is defined as the remaining battery power of the bus. If the battery power drops to 0, the bus is considered to have stopped running, and the DDPG training loop terminates. The state space expression at this time is... for:

[0097]

[0098] Where batt1 represents the battery level of the first bus, and batt... B The table shows the battery level of bus B.

[0099] Action Space: In each time slot, a certain proportion of the wireless charging power is allocated to the bus, and the remaining power is used for communication to maximize the communication rate. The action space expression is as follows: for:

[0100]

[0101] When the bus's battery level drops to negative, a penalty is applied to reduce the frequency with which the agent explores in that direction; then the reward function... for:

[0102]

[0103] S4. Based on the initialized parameter information, a bus power allocation method based on DDPG is designed, and the vehicle energy-carrying communication model is solved to obtain an approximately optimal bus energy-carrying power allocation decision.

[0104] In one feasible implementation, in step S4, a bus power allocation method based on DDPG is designed, including:

[0105] Obtain initial road environment information, including the number of buses (B) and the distance to the RSU (Roadside Unit). b,r Road environment information, initialization of θ, ω, and replay memory pool D via network parameters, and MDP model state.

[0106] In each iteration, based on the current state and selected action Calculate the bus throughput RT over a period of time and obtain the reward value. and the next state

[0107] Will Stored in replay memory pool D;

[0108] When D is full, randomly draw a small batch of samples from it to update the policy gradient of the Actor network. The Q-value y of the Critic network t The target network parameters θ′ and ω′ are updated using the loss function Loss and a soft update mechanism; then, the state is updated to the next state.

[0109] Repeat the above process until the bus's battery is depleted or the preset number of iterations N is reached, to obtain an approximately optimal bus power allocation decision Ep*={Ep1*,...,Ep b *,...,Ep B *}

[0110] In one feasible implementation, an action is selected based on the current state in each iteration. Action exploration noise ε is introduced, and the action space expression is... for:

[0111]

[0112] The bus throughput RT over a time period T is calculated using equations (9) and (13), and the reward value is obtained. and the next state The expression for throughput RT is:

[0113]

[0114] State, action, reward, and next state The samples are stored in the replay memory pool D. When the replay memory pool is full, a small batch of samples is randomly drawn to update the parameters of the Actor and Critic networks. The sampling policy gradient is then utilized. The rules for updating the Actor network are as follows:

[0115]

[0116] Critic's current network objective Q-value y t for:

[0117]

[0118] Its loss function is:

[0119]

[0120] The weights θ and ω of the target network are adjusted using a soft update mechanism. Let η∈[0,1] be the network update parameters. The update rules for the Actor target network parameters and the Critic target network parameters are as follows:

[0121] θ′←ηθ+(1-η)θ′(18)

[0122] w′←ηw+(1-η)w′(19) Then update the next state, the expression is:

[0123]

[0124] Finally, repeat the above process until the bus's battery is depleted or the preset number of iterations N is reached, to obtain the approximate optimal bus power allocation decision Ep*={Ep1*,...,Ep b *,...,Ep B *}, ensuring the electrical energy required for the normal operation of buses while maximizing the information transmission rate.

[0125] This invention addresses the challenge of balancing information transmission rate and energy consumption in Simultaneous Wireless Information and Power Transfer (SWIPT) communication for electric buses. A Markov Decision Process (MDP) model for urban vehicle-to-everything (V2X) environments is established, and a vehicle information and energy power allocation method based on Deep Deterministic Policy Gradient (DDPG) is proposed. Utilizing the Actor-Critic architecture of the DDPG algorithm, combined with experience replay and soft update mechanisms, power allocation is adaptively optimized to solve the problem of traditional allocation strategies failing to adapt to real-time changes in the traffic environment, achieving a dynamic balance between communication efficiency and energy consumption. This approach maximizes information transmission rate while meeting the electrical energy requirements for normal bus operation, providing a feasible solution for efficient and energy-saving communication for urban electric buses.

[0126] Figure 2 This is a block diagram of a vehicle signal power allocation device 300 based on a deep deterministic policy gradient, according to an exemplary embodiment. The device 300 is used in a vehicle signal power allocation method based on a deep deterministic policy gradient. (Refer to...) Figure 2The device includes a parameter initialization module 310, a communication model establishment module 320, an MDP model conversion module 330, and an allocation decision module 340. Wherein:

[0127] The parameter initialization module 310 is used to initialize parameter information;

[0128] The communication model establishment module 320 is used to establish a vehicle energy-carrying communication model;

[0129] MDP model conversion module 330 is used to convert the vehicle energy-carrying communication model into the vehicle power distribution MDP model.

[0130] The allocation decision module 340 is used to design a bus power allocation method based on DDPG based on the initialized parameter information, solve the vehicle energy-carrying communication model, and obtain an approximately optimal bus energy-carrying power allocation decision.

[0131] Optionally, the parameter initialization module 310 is used to initialize road environment information, such as the total number of buses B and the distance d from the RSU. b,r Road environment information; initialization of θ, ω and replay memory pool D neural network parameters, MDP model state.

[0132] Optionally, a vehicle-based energy-carrying communication model is established, including:

[0133] With κ as the energy charging efficiency coefficient, the power components of the radio frequency energy received and the information transmitted by the receiver of bus b from RSUr can be expressed as follows:

[0134] Ep r,b (t)=κp r,b (t)g b,r (t)δ b

[0135] Ip r,b (t)=p r,b (t)-Ep r,b (t)

[0136] Where, δ b ∈[0,1] represents the power allocation coefficient of the receiver on bus b, i.e., the proportion of radio frequency signal used for information reception and energy charging; p r,b (t) represents the total power received by bus b from RSUr within time slot t; Ep r,b (t) represents the RF energy fractional power received by the receiver of bus b in time slot t from RSUr; r represents a specific RSU; Ip r,b This indicates the power division of information received by the receiver of time slot bus b from RSUr;

[0137] Then the SINR of the information power allocation received by bus b from signal source RSUr is:

[0138]

[0139] Where, σ 2 and These represent the power of Gaussian white noise and signal processing noise, respectively; γ r,b (t) represents the SINR of the information power allocation received by bus b from signal source RSUr; C represents the total number of other vehicles communicating simultaneously with RSUr; g r,b (t) represents the channel gain between RSUr and bus b; c represents another vehicle communicating simultaneously with RSUr; g c,r (t) represents the channel gain between other vehicles and RSUr;

[0140] The transmission rate R between bus b and RSUr b,r (t) is:

[0141] R b,r (t)=W b,r log2(1+γ b,r (t)(1-δ b ))

[0142] Among them, W b,r This represents the bandwidth between bus b and RSUr.

[0143] Optionally, the vehicle energy-carrying communication model is transformed into a vehicle power distribution MDP model, including:

[0144] When the battery level drops to 0, the bus stops running, terminating the DDPG training loop; at this point, the state-space expression... for:

[0145]

[0146] Where batt1 represents the battery level of the first bus, and batt... B This indicates the battery level of the B-th bus;

[0147] In each time slot, a certain proportion of the wireless charging power is allocated to the bus, and the remaining power is used for communication. The action space expression is as follows. for:

[0148]

[0149] When the bus's battery level drops to negative, a penalty is applied to reduce the frequency with which the agent explores in that direction; then the reward function... for:

[0150]

[0151] Optionally, a DDPG-based bus power allocation method is designed, including:

[0152] Obtain initial road environment information, including the number of buses (B) and the distance to the RSU (Roadside Unit). b,r Road environment information, initialization of θ, ω, and replay memory pool D via network parameters, and MDP model state.

[0153] In each iteration, based on the current state and selected action Calculate the bus throughput RT over a period of time and obtain the reward value. and the next state

[0154] Will Stored in replay memory pool D;

[0155] When D is full, randomly draw a small batch of samples from it to update the policy gradient of the Actor network. The Q-value y of the Critic network t The target network parameters θ′ and ω′ are updated using the loss function Loss and a soft update mechanism; then, the state is updated to the next state.

[0156] Repeat the above process until the bus's battery is depleted or the preset number of iterations N is reached, to obtain an approximately optimal bus power allocation decision Ep*={Ep1*,...,Ep b *,...,Ep B *}

[0157] Optionally, action space expression for:

[0158] A t =μ(S) t |θ)+ε.

[0159] This invention addresses the challenge of balancing information transmission rate and energy consumption in Simultaneous Wireless Information and Power Transfer (SWIPT) communication for electric buses. A Markov Decision Process (MDP) model for urban vehicle-to-everything (V2X) environments is established, and a vehicle information and energy power allocation method based on Deep Deterministic Policy Gradient (DDPG) is proposed. Utilizing the Actor-Critic architecture of the DDPG algorithm, combined with experience replay and soft update mechanisms, power allocation is adaptively optimized to solve the problem of traditional allocation strategies failing to adapt to real-time changes in the traffic environment, achieving a dynamic balance between communication efficiency and energy consumption. This approach maximizes information transmission rate while meeting the electrical energy requirements for normal bus operation, providing a feasible solution for efficient and energy-saving communication for urban electric buses.

[0160] Figure 3 This is a schematic diagram of a vehicle information and power allocation device based on a deep deterministic policy gradient, provided in an embodiment of the present invention. Figure 3 As shown, the vehicle energy distribution device based on deep deterministic policy gradient can include the above-mentioned... Figure 2 The illustrated vehicle information and power allocation device is based on a deep deterministic policy gradient. Optionally, the vehicle information and power allocation device 410 based on a deep deterministic policy gradient may include a first processor 2001.

[0161] Optionally, the vehicle information and power allocation device 410 based on deep deterministic policy gradient may also include a memory 2002 and a transceiver 2003.

[0162] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0163] The following is combined with Figure 3 The components of the vehicle information and power distribution device 410 based on deep deterministic policy gradient are described in detail below:

[0164] The first processor 2001 is the control center of the vehicle information and power distribution device 410 based on deep deterministic policy gradient. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0165] Optionally, the first processor 2001 can execute various functions of the vehicle energy distribution device 410 based on deep deterministic policy gradient by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0166] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0167] In a specific implementation, as one example, the vehicle information and power allocation device 410 based on deep deterministic policy gradients may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0168] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0169] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the vehicle information and power allocation device 410 based on a deep deterministic policy gradient. Figure 3 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0170] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0171] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0172] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the vehicle signal power distribution device 410 based on deep deterministic policy gradient. Figure 3 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0173] It should be noted that, Figure 3 The structure of the vehicle information and power allocation device 410 based on deep deterministic policy gradient shown in the figure does not constitute a limitation on the router. The actual knowledge structure identification device may include more or fewer components than shown, or combine some components, or have different component arrangements.

[0174] Furthermore, the technical effects of the vehicle information and power allocation device 410 based on deep deterministic policy gradient can be referred to the technical effects of the vehicle information and power allocation method based on deep deterministic policy gradient described in the above method embodiments, and will not be repeated here.

[0175] It should be understood that the first processor 2001 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0176] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0177] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable sensors. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0178] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0179] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0180] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0181] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0182] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0183] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0184] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A vehicle information energy power allocation method based on deep deterministic policy gradient, characterized in that, The method includes: S1. Initialize parameter information; S2. Establish a vehicle-based energy-carrying communication model; by For the energy charging efficiency coefficient, buses The receiver from RSU The power components for receiving radio frequency energy and transmitting information can be expressed as follows: in, Indicates bus The power allocation factor of the receiver, i.e., the proportion of radio frequency signal used for information reception and energy charging; Indicates time slot Inner bus From RSU Total power received; RSU r Channel gain relative to bus b; The receiver for bus b in time slot t is from the RSU. r Receive radio frequency energy divided by power; This indicates the power division of information received by the receiver of time slot bus b from RSUr; Then the bus Receive signal from RSU The SINR of information power allocation is: in, and These represent the power of Gaussian white noise and signal processing noise, respectively. Indicates bus Receive signal from RSU SINR for information power allocation; C indicates other than RSU r There are C vehicles communicating simultaneously; c represents another vehicle that is communicating simultaneously with RSU r. This represents the transmission power of the interfering vehicle c; This represents the channel gain of other vehicles relative to RSU r; Bus B and RSU Transmission rate between for: in, This indicates the bandwidth between bus b and RSUr; S3. Transform the vehicle energy-carrying communication model into a vehicle power distribution MDP model. When the battery level drops to 0, the bus stops running, terminating the DDPG training loop; at this point, the state-space expression... for: ； in, Indicates the battery level of the first bus. This indicates the battery level of the B-th bus; In each time slot, a certain proportion of the wireless charging power is allocated to the bus, and the remaining power is used for communication. The action space expression is as follows. for: Specifically, when the bus's battery level becomes insufficient and turns negative, a penalty is applied to reduce the frequency with which the agent explores in that direction; then the reward function... for: S4. Based on the initialized parameter information, a bus power allocation method based on DDPG is designed, and the vehicle energy-carrying communication model is solved to obtain an approximately optimal bus energy-carrying power allocation decision.

2. The method according to claim 1, characterized in that, In S1, the initialization parameter information includes: Initialize road environment information and obtain the total number of buses. Distance from RSU Road environment information; initialization , and replay memory pool Neural network parameters and MDP model status.

3. The method according to claim 2, characterized in that, In S4, a bus power allocation method based on DDPG is designed, including: Obtain initial road environment information and the number of buses. Distance from RSU Wait for road environment information and initialize. , and replay memory pool Neural network parameters, MDP model state ; In each iteration, based on the current state and selected action Calculate the throughput of buses over a period of time. Get reward points and the next state ; Will Stored in the replay memory pool middle; when When the pool is full, randomly select a small batch of samples to update the policy gradient of the Actor network. Q-value of the Critic network and loss function Update the target network parameters through a soft update mechanism. , Next, update the state to the next state. ; Repeat the above process until the bus's battery is depleted or the preset number of iterations is reached. The approximate optimal bus power allocation decision was obtained. .

4. The method according to claim 3, characterized in that, Action space expression for: in, This indicates exploratory noise.

5. A vehicle information power allocation device based on deep deterministic policy gradient, wherein the vehicle information power allocation device based on deep deterministic policy gradient is used to implement the vehicle information power allocation method based on deep deterministic policy gradient as described in any one of claims 1-4, characterized in that, The device includes: The parameter initialization module is used to initialize parameter information; The communication model establishment module is used to establish a vehicle energy-carrying communication model; The MDP model conversion module is used to convert the vehicle energy-carrying communication model into the vehicle power distribution MDP model. The allocation decision module is used to design a bus power allocation method based on DDPG based on the initialized parameter information, solve the vehicle energy-carrying communication model, and obtain an approximately optimal bus energy-carrying power allocation decision.

6. The apparatus according to claim 5, characterized in that, The parameter initialization module is used to initialize road environment information and obtain the total number of buses. Distance from RSU Road environment information; initialization , and replay memory pool Neural network parameters and MDP model status.

7. A vehicle information and energy power allocation device based on deep deterministic policy gradient, the vehicle information and energy power allocation device based on deep deterministic policy gradient comprising: processor; The memory stores computer-readable instructions that, when executed by the processor, implement any one of the methods in the vehicle energy power allocation method based on deep deterministic policy gradient as described in any one of claims 1-4.

8. A computer-readable storage medium storing at least one instruction, the at least one instruction being loaded and executed by a processor to implement any one of the methods in the vehicle energy power allocation method based on deep deterministic policy gradient as described in any one of claims 1-4.

Images