Vehicle-to-vehicle communication spectrum resource allocation method based on deep reinforcement learning

Abstract

The invention discloses a vehicle-to-vehicle communication spectrum resource allocation method based on deep reinforcement learning, and the method comprises the steps: constructing a memory enhanced deep reinforcement learning neural network model, setting the capacity of a scene memory, and enabling a neural network of the memory enhanced deep reinforcement learning neural network model to comprise an Actor network and a Critic network; initializing parameters of the memory enhanced deep reinforcement learning neural network model; setting environment state information of the vehicle-to-vehicle communication intelligent agent; setting a reward form of the vehicle-to-vehicle communication agent; starting an interactive training process according to the environment state information and the reward form; after training convergence, storing model parameters of the memory enhanced deep reinforcement learning neural network model; and in an actual vehicle-to-vehicle communication environment, loading model parameters of the memory enhanced deep reinforcement learning neural network model, and outputting a distribution mode of vehicle-to-vehicle communication spectrum resources. According to the invention, cooperative control and adjustment of the train are realized, and the operation efficiency and safety of the train are improved.

Description

Technical Field

[0001] This invention belongs to the field of rail transit communication, specifically relating to a method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning. Background Technology

[0002] Train-to-Train (T2T) communication allows trains to exchange information such as location and speed in real time, enabling coordinated control and adjustments, and improving operational efficiency and safety. However, in T2T communication, each communication terminal typically requires an independent physical link to the 5G network. This process necessitates numerous wireless interfaces, leading to significant spectrum waste and severely limiting the development of diverse network services. To address this issue and fully utilize limited spectrum resources, T2T communication links need to reuse idle or low-load spectrum resources from Train-to-Ground (T2G) communication links without additional spectrum allocation. This spectrum reuse method further improves spectrum utilization, maximizing the communication needs of high-speed trains. However, reusing T2G spectrum resources in T2T communication links presents challenges such as high inter-link interference and low system throughput.

[0003] Train-to-train communication refers to the direct exchange of data between trains via wireless communication technology. This data includes information such as the train's position, speed, braking status, and direction of travel. Through this communication, trains can be aware of the status of other trains in the vicinity in real time, allowing them to react in advance, avoid collisions, optimize train operation scheduling, reduce delays, and improve operational efficiency.

[0004] Deep Reinforcement Learning (DRL) agents learn by interacting with their environment. The environment transmits state information to the agent, which then performs corresponding actions, receiving rewards in return. The agent's goal is to maximize cumulative rewards, learning the optimal decision-making outcome through repeated interactions. The advantage of DRL lies in its superior decision-making ability, similar to biological learning mechanisms. The agent learns through repeated interactions with the environment, maximizing cumulative rewards to determine the best decisions. Using a vehicle-to-vehicle (V2V) communication transmitter as the DRL agent, iterative optimization of deep neural network parameters enables the agent to learn optimal transmission power and spectrum resources, maximizing system throughput.

[0005] Memory is a crucial pathway to achieving higher-level behaviors in intelligent agents; complex human behaviors are all based on the ability to remember. Traditional memory structures, such as LSTM and GRU, integrate the network's training parameters with the memory capacity. As the memory capacity increases, training the neural network becomes difficult. Memory-augmented Deep Reinforcement Learning (M-DRL) models separate the network's training parameters from the memory capacity, thus increasing the memory capacity does not lead to an increase in training parameters.

[0006] Patent CN112367638A discloses an intelligent spectrum selection method for vehicle-to-vehicle communication in urban rail transit, which introduces a deep reinforcement learning model with multi-agent low-dimensional fingerprints to optimize spectrum resource allocation through training and testing phases. However, existing related methods all suffer from problems such as insufficient dynamic adaptability of spectrum resource allocation, insufficient global optimization of spectrum resource allocation, low throughput of communication links, insufficient ability to handle sudden interference, and limited generalization ability of the model. Summary of the Invention

[0007] The purpose of this invention is to provide a method for allocating spectrum resources for vehicle-to-vehicle communication based on deep reinforcement learning. This method aims to solve the problems of high inter-link interference and low system throughput in vehicle-to-vehicle communication, as well as the deficiencies in the dynamic adaptability of spectrum resource allocation, global optimization of spectrum resource allocation, and ability to handle sudden interference. This will enable coordinated control and adjustment of trains, thereby improving train operation efficiency and safety.

[0008] In view of the above problems, this application provides a method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning.

[0009] The first aspect disclosed in this application provides a method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning, the method comprising the following steps: Step 100: Construct a memory-enhanced deep reinforcement learning neural network model, and set the capacity of the context memory. The memory-enhanced deep reinforcement learning neural network model includes two Actor networks and one Critic network. Step 200: Initialize the parameters of the memory-enhanced deep reinforcement learning neural network model; Step 300: Set the environmental status information of the vehicle-to-vehicle communication agent; Step 400: Set the reward format for the vehicle-to-vehicle communication agent; Step 500: Based on the environmental status information and reward format, begin the interactive training process; Step 600: After training converges, save the model parameters of the memory-enhanced deep reinforcement learning neural network model; Step 700: In a real vehicle-to-vehicle communication environment, load the model parameters of the memory-enhanced deep reinforcement learning neural network model and output the allocation method of vehicle-to-vehicle communication spectrum resources.

[0010] Preferably, step 100 specifically includes the following steps: Step 101: Construct a memory-enhanced deep reinforcement learning neural network model using an Actor-Critic structure, including two Actor networks and one Critic network. The two Actor networks and one Critic network all adopt the Transformer network structure. The updated Actor network and Critic network share parameters. The Actor network receives environmental state information from vehicle-to-vehicle communication and outputs action values. The Critic network receives reward and punishment signals from the environment for the action and outputs time difference error. Step 102: Based on the complexity of the environment in which the vehicle-to-vehicle communication agent is located, set the capacity of the context memory. The context memory stores key-value pairs, where the key in the key-value pair is the environment state and the value is the state function value for evaluating the environment state. The capacity of the context memory is the number of key-value pairs stored.

[0011] Preferably, step 200 specifically includes the following steps: Step 201: Randomly initialize the parameters of the memory-enhanced deep reinforcement learning neural network model; Step 202: Set the hyperparameters for model training, including learning rate, batch size, random dropout rate, and number of iterations.

[0012] Preferably, step 300 specifically includes the following steps: The environmental state at time t will be represented as ,in, , This indicates that the i-th user participating in vehicle-to-vehicle communication is... t throughput at any given moment The number of users participating in vehicle-to-vehicle communication.

[0013] Preferably, step 400 specifically includes the following steps: Step 401: Calculate the vehicle-to-vehicle communication signal-to-noise ratio using equations (1) and (2) respectively. Vehicle-to-ground communication signal-to-noise ratio ,in, and They represent in t Time of the first mVehicle-to-ground communication users and the first n Transmit power of individual vehicle-to-vehicle communication users L represents the discrete number of transmit power levels. Indicates in t Time of the first m Each vehicle-to-ground communication user resource block has corresponding communication bandwidth B Channel gain below, Indicates in t Time of the first n Individual vehicle communication users in the corresponding communication bandwidth B Channel gain below, Indicates in t Time of the first n Channel gain for individual vehicle-to-vehicle communication users Indicates in t Time of the first n The vehicle-to-vehicle communication user and the first m Channel gain between vehicle-to-ground communication user resource blocks Indicates noise power. The Boolean value represents the first n Does the m-th vehicle-to-vehicle communication user and the m-th vehicle-to-ground communication user resource block have a reuse relationship? Number of users participating in vehicle-to-vehicle communication:

[0014]

[0015] Step 402: Calculate using equation (3) t Time of the first n The vehicle-to-vehicle communication user agent selects and reuses the first m Rewards for communication between vehicle-to-ground user resource blocks :

[0016] in, M represents the bandwidth of the communication, and M represents the number of user resource blocks for vehicle-to-ground communication.

[0017] Preferably, step 500 specifically includes the following steps: Step 501: Based on environmental conditions Using contextual memory, the state value function estimate is calculated using equation (4). , among which, among which This is the weight coefficient vector. The first one stored in the context memory i Each state function value, Q Capacity of contextual memory:

[0018] Step 502: Update the backpropagation parameters of the Actor network of the memory-enhanced deep reinforcement learning neural network model using Equation (5):

[0019] in, The parameters of the memory-enhanced deep reinforcement learning neural network model. Indicates about gradient, Let be the policy function, and , indicating the parameter at time t status Selecting Actions in the Actor Network The probability, and , and Select the reused vehicle-to-ground communication user resource blocks and transmit power for the vehicle-to-vehicle communication intelligent agent. The differential error provided by the Critic network to the Actor network based on real-time feedback from the environment. As a discount factor, Represents the state at time t. Selecting Actions in the Actor Network The reward is indicated by the sign "=".

[0020] Preferably, step 600 specifically includes the following steps: The cumulative reward is maximized using Equation (6) until it converges, and the parameters of the memory-enhanced deep reinforcement learning neural network model at this point are saved:

[0021] in, The total throughput of vehicle-to-vehicle communication users and vehicle-to-ground communication users. For vehicle-to-vehicle communication signal-to-noise ratio, For vehicle-to-ground communication signal-to-noise ratio, Indicates the bandwidth of communication. M represents the number of user resource blocks for vehicle-to-vehicle communication, which is the number of users participating in vehicle-to-vehicle communication.

[0022] The second aspect disclosed in this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning.

[0023] The third aspect disclosed in this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning.

[0024] The fourth aspect disclosed in this application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of the above-described method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning.

[0025] The beneficial effects of this invention are as follows: (1) It improves the throughput of the vehicle-to-vehicle communication system, improves the network carrying capacity of the vehicle-to-vehicle communication system, and improves the anti-interference capability of the vehicle-to-vehicle communication system.

[0026] (2) It improves the dynamic adaptability of spectrum resource allocation in vehicle-to-vehicle communication system and enhances the global optimization capability of spectrum resource allocation in vehicle-to-vehicle communication system.

[0027] (3) It realizes the coordinated control and adjustment of trains, and improves the efficiency and safety of train operation. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart of an overall method for allocating spectrum resources for vehicle-to-vehicle communication based on deep reinforcement learning.

[0030] Figure 2 This is a diagram showing the overall structure of a memory-enhanced deep reinforcement learning neural network model.

[0031] Figure 3 This is a diagram of the context memory storage structure.

[0032] Figure 4 This is a diagram showing the relationship between the agent, the environment state, the reward, and the action. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example 1: like Figure 1 As shown in the figure, this application provides a method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning, including the following steps: Step 100: Construct a memory-enhanced deep reinforcement learning neural network model and set the capacity of the context memory. The memory-enhanced deep reinforcement learning neural network model includes two Actor networks and one Critic network.

[0035] Step 100 specifically includes the following steps: Step 101: Construct a memory-enhanced deep reinforcement learning neural network model using an Actor-Critic structure, including two Actor networks and one Critic network. The two Actor networks and one Critic network all adopt the Transformer network structure. The updated Actor network and Critic network share parameters. The Actor network receives environmental state information from vehicle-to-vehicle communication and outputs action values. The Critic network receives reward and punishment signals from the environment for the action and outputs time difference error. Step 102: Based on the complexity of the environment in which the vehicle-to-vehicle communication agent is located, set the capacity of the context memory. The context memory stores key-value pairs, where the key in the key-value pair is the environment state and the value is the state function value for evaluating the environment state. The capacity of the context memory is the number of key-value pairs stored.

[0036] Step 200: Initialize the parameters of the memory-enhanced deep reinforcement learning neural network model.

[0037] Step 200 specifically includes the following steps: Step 201: Randomly initialize the parameters of the memory-enhanced deep reinforcement learning neural network model; Step 202: Set the hyperparameters for model training, including learning rate, batch size, random dropout rate, and number of iterations.

[0038] Step 300: Set the environmental status information of the vehicle-to-vehicle communication agent, specifically including: The environmental state at time t will be represented as ,in, , This indicates that the i-th user participating in vehicle-to-vehicle communication is... t throughput at any given moment The number of users participating in vehicle-to-vehicle communication.

[0039] Step 400: Set the reward format for the vehicle-to-vehicle communication agent.

[0040] Step 400 specifically includes the following steps: Step 401: Calculate the vehicle-to-vehicle communication signal-to-noise ratio using equations (1) and (2) respectively. Vehicle-to-ground communication signal-to-noise ratio ,in, and They represent in t Time of the first m Vehicle-to-ground communication users and the first n Transmit power of individual vehicle-to-vehicle communication users L represents the discrete number of transmit power levels. Indicates in t Time of the first m Each vehicle-to-ground communication user resource block has corresponding communication bandwidth B Channel gain below, Indicates in t Time of the first n Individual vehicle communication users in the corresponding communication bandwidth B Channel gain below, Indicates in t Time of the first n Channel gain for individual vehicle-to-vehicle communication users Indicates in t Time of the first n The vehicle-to-vehicle communication user and the first m Channel gain between vehicle-to-ground communication user resource blocks Indicates noise power. The Boolean value represents the first n Does the m-th vehicle-to-vehicle communication user and the m-th vehicle-to-ground communication user resource block have a reuse relationship? Number of users participating in vehicle-to-vehicle communication:

[0041]

[0042] Step 402: Calculate using equation (3) t Time of the first n The vehicle-to-vehicle communication user agent selects and reuses the first m Rewards for communication between vehicle-to-ground user resource blocks :

[0043] in, M represents the bandwidth of the communication, and M represents the number of user resource blocks for vehicle-to-ground communication.

[0044] Step 500: Start the interactive training process based on the environmental status information and reward format.

[0045] Step 500 specifically includes the following steps: Step 501: Based on environmental conditions Using contextual memory, the state value function estimate is calculated using equation (4). , among which, among which This is the weight coefficient vector. The first one stored in the context memory i Each state function value, Q Capacity of contextual memory:

[0046] Step 502: Update the backpropagation parameters of the Actor network of the memory-enhanced deep reinforcement learning neural network model using Equation (5):

[0047] in, The parameters of the memory-enhanced deep reinforcement learning neural network model. Indicates about gradient, Let be the policy function, and , indicating the parameter at time t status Selecting Actions in the Actor Network The probability, and , and Select the reused vehicle-to-ground communication user resource blocks and transmit power for the vehicle-to-vehicle communication intelligent agent. The differential error provided by the Critic network to the Actor network based on real-time feedback from the environment. As a discount factor, Represents the state at time t. Selecting Actions in the Actor Network The reward is indicated by the sign "=".

[0048] Step 600: After training converges, save the model parameters of the memory-enhanced deep reinforcement learning neural network model.

[0049] Step 600 specifically includes the following steps: The cumulative reward is maximized using Equation (6) until it converges, and the parameters of the memory-enhanced deep reinforcement learning neural network model at this point are saved:

[0050] in, The total throughput of vehicle-to-vehicle communication users and vehicle-to-ground communication users. For vehicle-to-vehicle communication signal-to-noise ratio, For vehicle-to-ground communication signal-to-noise ratio, Indicates the bandwidth of communication. M represents the number of user resource blocks for vehicle-to-vehicle communication, which is the number of users participating in vehicle-to-vehicle communication.

[0051] Step 700: In a real vehicle-to-vehicle communication environment, load the model parameters of the memory-enhanced deep reinforcement learning neural network model and output the allocation method of vehicle-to-vehicle communication spectrum resources.

[0052] In summary, the vehicle-to-vehicle communication spectrum resource allocation method based on deep reinforcement learning provided in this application has the following technical effects: (1) It improves the throughput of the vehicle-to-vehicle communication system, improves the network carrying capacity of the vehicle-to-vehicle communication system, and improves the anti-interference capability of the vehicle-to-vehicle communication system.

[0053] (2) It improves the dynamic adaptability of spectrum resource allocation in vehicle-to-vehicle communication system and enhances the global optimization capability of spectrum resource allocation in vehicle-to-vehicle communication system.

[0054] (3) It realizes the coordinated control and adjustment of trains, and improves the efficiency and safety of train operation.

[0055] Example 2: In Embodiment 2, a computer device is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the above-described method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning.

[0056] Example 3: In Embodiment 3, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning.

[0057] Example 4: In Embodiment 4, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implement the steps of the above-described method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning.

[0058] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0059] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning, characterized in that, The method includes the following steps: Step 100: Construct a memory-enhanced deep reinforcement learning neural network model, and set the capacity of the context memory. The memory-enhanced deep reinforcement learning neural network model includes two Actor networks and one Critic network. Step 200: Initialize the parameters of the memory-enhanced deep reinforcement learning neural network model; Step 300: Set the environmental status information of the vehicle-to-vehicle communication agent; Step 400: Set the reward format for the vehicle-to-vehicle communication agent; Step 500: Based on the environmental status information and reward format, begin the interactive training process; Step 600: After training converges, save the model parameters of the memory-enhanced deep reinforcement learning neural network model; Step 700: In a real vehicle-to-vehicle communication environment, load the model parameters of the memory-enhanced deep reinforcement learning neural network model and output the allocation method of vehicle-to-vehicle communication spectrum resources.

2. The method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning as described in claim 1, characterized in that, Step 100 specifically includes the following steps: Step 101: Construct a memory-enhanced deep reinforcement learning neural network model using an Actor-Critic structure, including two Actor networks and one Critic network. The two Actor networks and one Critic network all adopt the Transformer network structure. The updated Actor network and Critic network share parameters. The Actor network receives environmental state information from vehicle-to-vehicle communication and outputs action values. The Critic network receives reward and punishment signals from the environment for the action and outputs time difference error. Step 102: Based on the complexity of the environment in which the vehicle-to-vehicle communication agent is located, set the capacity of the context memory. The context memory stores key-value pairs, where the key in the key-value pair is the environment state and the value is the state function value for evaluating the environment state. The capacity of the context memory is the number of key-value pairs stored.

3. The method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning as described in claim 1, characterized in that, Step 200 specifically includes the following steps: Step 201: Randomly initialize the parameters of the memory-enhanced deep reinforcement learning neural network model; Step 202: Set the hyperparameters for model training, including learning rate, batch size, random dropout rate, and number of iterations.

4. The method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning as described in claim 1, characterized in that, Step 300 specifically includes the following steps: The environmental state at time t will be represented as ,in, , This indicates that the i-th user participating in vehicle-to-vehicle communication is... t throughput at any given moment The number of users participating in vehicle-to-vehicle communication.

5. The method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning as described in claim 1, characterized in that, Step 400 specifically includes the following steps: Step 401: Calculate the vehicle-to-vehicle communication signal-to-noise ratio using equations (1) and (2) respectively. Vehicle-to-ground communication signal-to-noise ratio ,in, and They represent in t Time of the first m The vehicle-to-ground communication user and the first n Transmit power of individual vehicle-to-vehicle communication users L represents the discrete number of transmit power levels. Indicates in t Time of the first m Each vehicle-to-ground communication user resource block has corresponding communication bandwidth B Channel gain below, Indicates in t Time of the first n Individual vehicle communication users in the corresponding communication bandwidth B Channel gain below, Indicates in t Time of the first n Channel gain for individual vehicle-to-vehicle communication users Indicates in t Time of the first n The vehicle-to-vehicle communication user and the first m Channel gain between vehicle-to-ground communication user resource blocks Indicates noise power. The Boolean value represents the first n Does the m-th vehicle-to-vehicle communication user and the m-th vehicle-to-ground communication user resource block have a reuse relationship? Number of users participating in vehicle-to-vehicle communication: Step 402: Calculate using equation (3) t Time of the first n The vehicle-to-vehicle communication user agent selects and reuses the first m Rewards for communication between vehicle-to-ground user resource blocks : in, M represents the bandwidth of the communication, and M represents the number of user resource blocks for vehicle-to-ground communication.

6. The method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning as described in claim 1, characterized in that, Step 500 specifically includes the following steps: Step 501: Based on environmental conditions Using contextual memory, the state value function estimate is calculated using equation (4). , among which, among which This is the weight coefficient vector. The first one stored in the context memory i Each state function value, Q Capacity of contextual memory: Step 502: Update the backpropagation parameters of the Actor network of the memory-enhanced deep reinforcement learning neural network model using Equation (5): in, The parameters of the memory-enhanced deep reinforcement learning neural network model. Indicates about gradient, Let be the policy function, and , indicating the parameter at time t status Selecting Actions in the Actor Network The probability, and , and Select the reused vehicle-to-ground communication user resource blocks and transmit power for the vehicle-to-vehicle communication intelligent agent. The differential error provided by the Critic network to the Actor network based on real-time feedback from the environment. As a discount factor, Represents the state at time t. Selecting Actions in the Actor Network The reward is indicated by the sign "=".

7. The method for allocating vehicle-to-vehicle communication spectrum resources based on deep reinforcement learning as described in claim 1, characterized in that, Step 600 specifically includes the following steps: The cumulative reward is maximized using Equation (6) until it converges, and the parameters of the memory-enhanced deep reinforcement learning neural network model at this point are saved: in, The total throughput of vehicle-to-vehicle communication users and vehicle-to-ground communication users. For vehicle-to-vehicle communication signal-to-noise ratio, For vehicle-to-ground communication signal-to-noise ratio, Indicates the bandwidth of communication. M represents the number of user resource blocks for vehicle-to-vehicle communication, which is the number of users participating in vehicle-to-vehicle communication.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the vehicle-to-vehicle communication spectrum resource allocation method based on deep reinforcement learning as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the vehicle-to-vehicle communication spectrum resource allocation method based on deep reinforcement learning as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the steps of the vehicle-to-vehicle communication spectrum resource allocation method based on deep reinforcement learning as described in any one of claims 1 to 7.

Images