Inter-vehicle communication resource selection method and device, equipment, medium and product

Abstract

The invention discloses an inter-vehicle communication resource selection method and device, equipment, a medium and a product, and relates to the field of Internet of Vehicles communication, and the method comprises the steps: constructing an energy characteristic spectrum of a candidate resource library based on RB energy characteristics; based on the energy characteristic spectrum, constructing and training a DNN network, and modeling V2V communication link quality related to a vehicle motion state into environment state information; based on the DNN, establishing a DQN application model; when the VUE triggers and selects the resource block in any time slot, the DQN application model is utilized to select the resource block set matched with the quality of the current V2V communication link from the candidate resource library, V2V communication resources in a high-density traffic state can be reasonably allocated, and the reliability and time delay performance of message propagation can meet the technical requirements of automatic driving.

Description

Technical Field

[0001] The present application relates to the field of vehicle networking communications, and in particular to a method, device, equipment, medium and product for selecting vehicle communication resources. Background Art

[0002] Cellular Vehicle to Everything (C-V2X) uses the cellular network to achieve interconnection between moving vehicles through vehicle user equipment (VUE) and transmit traffic status messages and traffic safety messages, providing information communication services for the intelligent transportation network. In the Internet of Vehicles, communication connections mainly include vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), vehicle-to-pedestrian (V2P) and vehicle-to-network (V2N). The Third Generation Partnership (3GPP) has formulated distributed Mode 4 and Mode 2 communication resource allocation modes for V2V communication in LTE-V2X (Long Term Evolution-V2X) and NR-V2X (New Radio-V2X) networks, respectively, and both use the Sensing Based Semi-Persistent Scheduling (SB-SPS) mechanism. The SB-SPS mechanism includes three processes: resource perception, candidate resource selection, and resource selection (or resource reselection). In the resource selection (or resource reselection) process, the selection method of communication resources directly affects the propagation quality of V2V communication messages and the utilization rate of limited communication resources.

[0003] In C-V2X, communication resources are two-dimensional resource blocks (ResourceBlock, RB) consisting of communication time slots and communication frequencies. The resource granularity on the time slot is the subframe, and the resource granularity on the frequency is the subchannel. At present, the standard SB-SPS mechanism uses a random allocation method to select available communication resources for V2V communication, without considering the dynamic time-varying nature of the vehicle state, which increases the collision probability of resource selection and reduces the propagation performance of V2V communication messages. The resource allocation method based on dynamic spectrum dynamically allocates available resources according to V2V communication needs, but under high-density traffic conditions, it is necessary to frequently propagate and update vehicle motion state information, the VUE has high computational complexity, and the message communication delay is large. It can be seen that the existing technology divides the strong correlation between resource selection and vehicle motion state, resulting in the reliability and delay performance of message propagation failing to meet the technical requirements of autonomous driving when the VUE uses the allocated resources for V2V communication under high-density traffic conditions. Summary of the invention

[0004] The purpose of this application is to provide a vehicle-to-vehicle communication resource selection method, device, equipment, medium and product to solve the problem of unreasonable allocation of V2V communication resources under high-density traffic conditions, resulting in the reliability and delay performance of message propagation failing to meet the technical requirements of autonomous driving.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for selecting a workshop communication resource, comprising:

[0007] Based on the RB energy characteristics, an energy characteristic map of the candidate resource library is constructed;

[0008] Based on the energy characteristic map, a DNN network is constructed and trained to model the V2V communication link quality related to the vehicle motion state as environmental state information; the V2V communication link quality is a V2V communication resource; the V2V communication resource is a resource block sequence consisting of 10 time domain subframes and 12 frequency domain subchannels;

[0009] Based on the DNN network, a DQN application model is established; the DQN application model is stored in the VUE storage space; the DQN application model includes an agent, an environment, a state, an action, and an immediate reward; the agent includes a main neural network and a target neural network; the main neural network and the target neural network have the same structure but different weight parameter vectors; the main neural network is used to calculate the approximate Q value of the main neural network under the current input state; the target neural network is used to calculate the approximate Q value of the target neural network under the transfer state; the transfer state is the state of the current input state at the next moment; the main neural network is a trained DNN network;

[0010] When the VUE triggers the selection of a resource block in any time slot, the DQN application model is used to select a set of resource blocks matching the current V2V communication link quality from the candidate resource pool.

[0011] In a second aspect, the present application provides a vehicle communication resource selection device, including:

[0012] An energy feature map construction module is used to construct an energy feature map of a candidate resource library based on RB energy features;

[0013] A DNN network construction module is used to construct and train a DNN network based on the energy feature map, and model the V2V communication link quality related to the vehicle motion state as environmental state information; the V2V communication link quality is a V2V communication resource; the V2V communication resource is a resource block sequence consisting of 10 time domain subframes and 12 frequency domain subchannels;

[0014] A DQN application model building module is used to build a DQN application model based on the DNN network; the DQN application model is stored in the VUE storage space; the DQN application model includes an agent, an environment, a state, an action, and an immediate reward; the agent includes a main neural network and a target neural network; the main neural network and the target neural network have the same structure but different weight parameter vectors; the main neural network is used to calculate the approximate Q value of the main neural network under the current input state; the target neural network is used to calculate the approximate Q value of the target neural network under the transfer state; the transfer state is the state of the current input state at the next moment; the main neural network is a trained DNN network;

[0015] The resource block set selection module is used to select a resource block set matching the current V2V communication link quality from the candidate resource library using the DQN application model when the VUE triggers the selection of a resource block in any time slot.

[0016] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described methods for selecting workshop communication resources.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for selecting workshop communication resources.

[0018] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for selecting workshop communication resources.

[0019] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0020] In order to solve the problem of V2V communication resource selection under high-density traffic conditions, a deep neural network (DNN) is constructed and trained through pre-calculation, and the quality of the vehicle-to-vehicle communication link related to the vehicle motion state is modeled as environmental state information. A deep reinforcement learning (DRL) algorithm based on a deep Q-network (DQN) is used to select a set of resource blocks that match the current V2V communication link quality from a candidate resource library. This application achieves a quasi-real-time association between the V2V communication link quality (i.e., resource quality) selected by the VUE and the vehicle motion state through pre-calculation training to obtain the best quality resource relative to the current motion position, thereby improving the message propagation performance and maximizing the throughput of limited communication resources, so that the reliability and latency performance of message propagation meet the technical requirements of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A flow chart of a method for selecting workshop communication resources provided in an embodiment of the present application;

[0023] Figure 2 It is the energy characteristic spectrum of the candidate resource library;

[0024] Figure 3 It is a DNN network diagram based on resource energy feature map;

[0025] Figure 4 This is a DQN model diagram based on a dual DNN network;

[0026] Figure 5 A system model diagram for the vehicle communication resource selection method provided in this application;

[0027] Figure 6 for Figure 5 Resource selection flow chart in ;

[0028] Figure 7 is a curve diagram showing the relationship between the packet received ratio (PRR) and the vehicle density;

[0029] Figure 8It is a curve diagram showing the relationship between resource utilization (RU) and vehicle density. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0032] The embodiment of the present application provides a method for selecting a workshop communication resource. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, Figure 1 As shown, the method includes the following steps.

[0033] S1: Based on the RB energy characteristics, an energy characteristic map of the candidate resource library is constructed.

[0034] S2: Based on the energy feature map, a DNN network is constructed and trained to model the V2V communication link quality related to the vehicle motion state as environmental state information; the V2V communication link quality is a V2V communication resource; the V2V communication resource is a resource block sequence consisting of 10 time domain subframes and 12 frequency domain subchannels.

[0035] S3: Based on the DNN network, a DQN application model is established; the DQN application model is stored in the VUE storage space; the DQN application model includes an agent, an environment, a state, an action and an immediate reward; the agent includes a main neural network and a target neural network; the main neural network and the target neural network have the same structure but different weight parameter vectors; the main neural network is used to calculate the approximate Q value of the main neural network under the current input state; the target neural network is used to calculate the approximate Q value of the target neural network under the transfer state; the transfer state is the state of the current input state at the previous moment; the main neural network is a trained DNN network.

[0036] S4: When the VUE triggers resource block selection in any time slot, a resource block set matching the current V2V communication link quality is selected from the candidate resource pool using the DQN application model.

[0037] In an exemplary embodiment, step S1 may be replaced by the following steps.

[0038] S11: Calculate the RSSI average of the received signal in any time domain subframe; the RSSI average is: in, is the RSSI mean of the received signal in the i-th time domain subframe; i is the time domain subframe number, i∈[1,10]; h is a measurement cycle in the perception window, h∈[1,H], H is the total number of measurement cycles in the perception window, T is the duration of the perception window, and RRI is the resource reservation interval.

[0039] S12: Calculate the SINR mean of the RB set corresponding to any frequency domain subchannel in any time domain subframe; the SINR mean is in, is the SINR mean of the RB set corresponding to the j-th subchannel in the i-th subframe, j∈[1,12]; P s is the transmission power per resource block; G sr is the channel gain between the transmitter s and the receiver r; N0 is the noise power per resource block; K is the set of VUEs competing for the same resource block within the measurement period; G sk is the interference channel gain between the transmitting end s and the competing end k; the RB energy characteristics include all RSSI averages and all SINR averages.

[0040] S13: Construct an energy characteristic spectrum of a candidate resource library according to the RSSI average value and the SINR average value.

[0041] Furthermore, the Release sequence standard released by 3GPP describes the V2V communication resources of the cellular vehicle network as a resource block sequence consisting of 10 time domain subframes (Subframe) and 12 frequency domain subchannels (Subchannel).

[0042] By calculating the received signal strength index (RSSI) value in the time domain subframe, the idle state of RBs in the subframe can be estimated. Assume that the RSSI average value of the i-th subframe in the perception window is As shown in formula (1).

[0043]

[0044] Among them, i∈[1,10], h∈[1,H] is a measurement cycle within the perception window, is the total number of measurement cycles within the sensing window, T is the duration of the sensing window, and RRI is the resource reservation interval.

[0045] The quality state of a sub-channel in a sub-frame is estimated by calculating the mean value of the signal to interference plus noise ratio (SINR) of the sub-channel.

[0046] Assume that the average SINR value of the RB set corresponding to the jth subchannel in the i-th subframe is As shown in formula (2).

[0047]

[0048] Among them, j∈[1,12], P s is the transmission power per resource block, G sr is the channel gain between the transmitter s and the receiver r, N0 is the noise power per resource block, K is the set of VUEs competing for the same resource block within the measurement period, G sk is the interference channel gain between the transmitter s and the contender k.

[0049] Based on the RSSI mean within the subframe calculated by formula (1) and the SINR mean of different subchannels within the subframe calculated by formula (2), the energy characteristic spectrum of the candidate resource library is constructed, such as Figure 2 shown.

[0050] In an exemplary embodiment, S2 may be replaced by the following steps.

[0051] S21: Establishing an experience sample library based on the SINR mean value.

[0052] S22: Randomly select multiple experience samples from the experience sample library, train the DNN network, and model the V2V communication link quality related to the vehicle motion state as environmental state information; the experience samples include the main neural network input state, action, immediate reward and the transfer state at the next moment; wherein the main neural network input state and the transfer state represent an energy feature spectrum, the action is a set of time-frequency resource blocks, and the immediate reward is the total throughput obtained under the premise of the selected resources; the DNN network includes an input layer, a subframe iteration layer, a subchannel iteration layer and an output layer connected in sequence.

[0053] Further, construct a DNN network: Assume that at a certain moment, the perception window measures all subframes of the candidate resource library Collection All sub-channels Collection The number of resource blocks allocated for VUE business communication requests is N. The two-layer iterative DNN network constructed based on the energy characteristic parameters of the candidate resource pool as the input layer state parameters is as follows: Figure 3As shown, it includes an input layer for the distribution state of the energy characteristics of the resource library, an iteration layer for selecting subframes, an iteration layer for selecting subchannels, and an output layer for selecting resource block sets.

[0054] Establish an experience sample library: Let the immediate reward r be the total throughput of the resource block set selected by the current action a Where i is the subframe number, j is the subchannel number, N is the total number of resource blocks required by the VUE, and B RB The bandwidth of a single resource block is generally 15 kHz. If the main neural network inputs state s at a certain moment, and obtains an immediate reward r after executing action a, it outputs an approximate Q value Q(s,a;θ), and the input state is transferred from s to s'. Subsequently, state s' is input to the target neural network, and after executing action a', its approximate Q value Q(s',a';θ) is output. - ), thereby generating an "experience sample" e = (s, a, r, s') and putting it into the experience pool. Each time the agent interacts with the environment, an experience sample is put into the experience pool and accumulated into a historical experience sample set E. The first-in-first-out (FIFO) method is used to update and retain the last M experience samples.

[0055] The ε-greedy algorithm is used to select action a, which selects a random action with probability ε and selects the action that maximizes the current Q value with probability (1-ε).

[0056] In an exemplary embodiment, the training process of the DNN network specifically includes:

[0057] S31: Input the transfer state into the target neural network, execute the action corresponding to the transfer state, and output an approximate Q value; the approximate Q value includes the transfer state, the action corresponding to the transfer state, and the weight parameter vector of each edge of the target neural network.

[0058] S32: Determine a target output value of the target neural network according to the approximate Q value.

[0059] S33: According to the target output value and the approximate Q value of the main neural network, construct the loss function of the weight parameter vector of each edge of the DNN network, and determine the mean of the loss function.

[0060] S34: Use the stochastic gradient descent algorithm to minimize the mean of the loss function and update the weight parameter vectors of each edge of the DNN network.

[0061] S35: During the training process, the number of iterations is set at each interval, the weight parameter vectors of each edge of the DNN network are assigned to the target neural network, the weight parameter vectors of each edge of the target neural network are updated, and the training of the DNN network is completed.

[0062] In an exemplary embodiment, the loss function is: L(θ)=[yQ(s,a;θ)] 2 ; Where y is the target output value, y=r+γmaxQ(s',a';θ - ), r is the immediate reward, γ is the discount factor, Q(s',a';θ - ) is the weight parameter vector of each edge of the target neural network, s' is the transition state, a' is the action corresponding to the transition state, θ - is the weight parameter vector of each edge of the target neural network; Q(s,a;θ) is the approximate Q value of the main neural network, s is the current input state of the main neural network, a is the action, and θ is the weight parameter vector of each edge of the DNN network.

[0063] Further, train the DNN network: when the DNN weight parameter vector θ needs to be updated, randomly select N samples from the experience pool sample set E E “Experience samples” are used to train the DNN network. Assume that the target output value y of the target neural network is as shown in formula (3).

[0064] y=r+γmax Q(s',a';θ - ) (3)

[0065] Among them, r is the immediate reward and γ is the discount factor.

[0066] The loss function L(θ) of the DNN weight parameter vector θ constructed based on the target output value y and the main neural network approximate Q value Q(s,a;θ) is L(θ)=[yQ(s,a;θ)] 2 , then the loss function mean L E (θ) is as shown in formula (4).

[0067]

[0068] Use the stochastic gradient descent algorithm to minimize L E (θ), that is Then update the parameter vector θ as shown in formula (5).

[0069] θ←θ+E[yQ(s,a;θ)]▽ θ Q(s,a;θ)(5)

[0070] in,

[0071] During the training process, the weight parameter vector θ of the main neural network is copied to the target neural network to update the weight parameter vector θ at a certain number of iterations C. - , in order to improve the approximation between the target neural network and the main neural network. The number of iterations C is set according to the actual calculation accuracy requirements.

[0072] Furthermore, a DQN application model is established: The DQN application model based on the two-layer iterative DNN network is as follows Figure 4 As shown in the figure, it includes five basic elements: agent, environment, state s, action a and immediate reward r. The agent contains two DNN networks with exactly the same structure but different weight parameters, called the main neural network (Main DNN) and the target neural network (TargetDNN). The main neural network is used to calculate the approximate Q value Q(s,a;θ) under the current input state, and θ is the parameter vector of the weights of each edge of the main neural network. The target neural network is used to calculate the target Q value Q(s',a';θ - ) to improve the stability of the main neural network during iteration, θ - is the parameter vector of the weights of each edge of the target neural network. The environment is the candidate resource library, the state s is the energy characteristic map of the resource library, the action a is the time-frequency resource block set selected by the ε-greedy algorithm, and the immediate reward r is the total throughput obtained under the premise of the selected resources.

[0073] Storing DQN application model: After the DNN network training is completed, the DQN application model built based on the main neural network is placed in the VUE storage space for V2V communication to select a resource block set that matches the vehicle's motion state. When the DNN network is retrained, the VUE updates the DQN application model.

[0074] In an exemplary embodiment, S4 may be replaced by the following steps.

[0075] S41: When the VUE triggers the selection of a resource block in any time slot, the energy characteristic spectrum of the candidate resource pool measured in the perception period is input into the DQN application model.

[0076] S42: Based on the energy feature map, the main neural network is used to iteratively output a set of resource blocks in each time slot time domain subframe, and the intelligent agent is instructed to use an ε-greedy algorithm to select the maximum SINR mean in the time domain subframe corresponding to the minimum RSSI mean.

[0077] S43: Using the maximum SINR mean as an output action, generating experience samples, using the FIFO method to update the retained experience samples, and determining a resource block set that matches the current V2V communication link quality.

[0078] Furthermore, the DQN application model is used to select a resource block set.

[0079] Assuming that VUE triggers the selection of resource blocks in a certain time slot, the DQN application model inputs the distribution state map of the resource pool energy characteristics measured during the perception period. Its built-in main neural network DNN iteratively outputs the resource block set in each time slot subframe The agent uses the ε-greedy algorithm to select Maximum in subframe The resource block set is used as the output action a. The agent generates experience samples e = (s, a, r, s') and puts them into the experience pool E, and uses the FIFO method to update and retain the last M experience samples. At the same time, the resource scheduling mechanism of the VUE allocates the selected resource block set to the V2V communication connection for transmitting V2V message data.

[0080] like Figure 5 As shown, the system model of this embodiment assumes that in a high-density intersection scenario, the vehicle-mounted VUE set V = {V1, V2, ..., V U}, where U is the total number of VUEs. Multiple pairs of V2V communications need to reuse cellular network spectrum resources to establish direct near-field communication connections. VUE uses the SB-SPS scheduling mechanism to select RB sets for vehicles to transmit message data. The number of RBs required is determined by the message data size, coding, and modulation schemes.

[0081] The process of this embodiment is as follows Figure 6 As shown, the steps are as follows:

[0082] 1) The system model includes multiple pairs of vehicle-to-vehicle communications that need to transmit near-field information data simultaneously. Vehicle-to-vehicle communications establish vehicle-to-vehicle communication links by reusing the idle authorized spectrum in the cellular network, while not interfering with the communications of legally authorized users of the cellular network. Therefore, spectrum with lower RSSI values ​​is selected for V2V communication. When a V2V communication needs to transmit message data, the SB-SPS scheduling mechanism is triggered to allocate spectrum resources for it.

[0083] 2) The scheduling mechanism of VUE calculates the RSSI mean of the candidate resource subframe using formula (1) according to the candidate resource parameter values ​​measured in the sensing period, and calculates the SINR mean of the candidate resource subchannel using formula (2), and constructs the energy characteristic spectrum of the candidate resource library.

[0084] 3) The VUE calculation unit calls the stored DQN application model to iteratively calculate the SINR mean of N candidate resource blocks in each time slot, and outputs the set of available resource blocks.

[0085] 4) The agent of the DQN application model uses the ε-greedy algorithm to select from all available resource blocks N subframes have The resource block set is used as the output resource selection result.

[0086] 5) The VUE scheduling mechanism assigns a set of resource blocks to the V2V requesting resources according to the DQN output action a for transmitting V2V communication message data.

[0087] The following is an example of simulating the present application using the WiLabV2Xsim simulation system based on the MATLAB platform on a computer. In the simulation, the standard SB-SPS resource scheduling mechanism released by 3GPP Release 17 is used, the communication channel bandwidth is 10 MHz, the perception window duration is 1000 ms, the resource reservation interval is 100 ms, the resource reservation probability is 0.4, the modulation and coding scheme is 11, and the transmission power per resource block is P s =2.3dBm, the noise power per resource block N0 = 0.9dBm. The channel propagation model is WINNER+B1 model, and the data packet size is 300Byte. The average vehicle speed is set to 60km / h, the DNN learning rate α = 0.01, the experience pool size M = 10000 samples, the discount factor γ = 0.9, and the number of small batch experience samples N E =64, the initial experience sample set E is generated by the WiLabV2Xsim simulation system in the urban traffic scenario Urban3GPP.cfg.

[0088] Figure 7 Figure 2 is a curve showing the relationship between the packet received ratio (PRR) and the vehicle density. Figure 7 It can be seen that the PRR of the proposed method (Deep Q Network Energy Mapping, DQN-EM) is higher than that of the Standard SB-SPS method and the Resource Alternation Selection (RAS) method, which shows that a high-quality vehicle communication channel can be established by selecting a set of resource blocks that are suitable for the vehicle's motion state. For example, when the vehicle density is 200veh / km, the PRR of the proposed method is 92.7%, which is 4.6% and 2.9% higher than the 88.1% packet reception rate of the Standard SB-SPS method and the 89.8% packet reception rate of the RAS method, respectively. The key is that the minimum requirement for the reliability index of message data transmission in the Internet of Vehicles is a PRR greater than 90%.

[0089] Figure 8 is a curve diagram showing the relationship between resource utilization (RU) and vehicle density. Figure 8It can be seen that the resource utilization of the DQN-EM method is better than that of the Standard SB-SPS method and the RAS method, which shows that by selecting a set of high-quality resource blocks to establish a vehicle communication channel, the error probability of channel transmission is reduced, and the number of packet retransmissions is reduced. For example, when the vehicle density is 200veh / km, the resource utilization of the proposed method is 49.3%, which is 15.7% and 12.2% higher than the 33.6% of the Standard SB-SPS method and the 15.7% of the RAS method, respectively. Under the condition of limited spectrum resources, it is particularly important for the transmission of traffic emergency messages in high-density traffic scenarios.

[0090] Compared with the prior art, this application has the following advantages:

[0091] 1. The energy parameter values ​​of the candidate resource library subframes and subchannels are constructed as energy feature maps, which improves the adaptability between the energy state of the resource block and the vehicle motion state.

[0092] 2. The DQN application model based on the dual DNN network improves the stability of the resource selection algorithm in highly dynamic vehicle networks.

[0093] 3. The DQN application model stored in VUE can respond to V2V communication resource allocation requests in real time, improving V2V communication performance and spectrum resource utilization under high-density communication conditions.

[0094] Based on the same inventive concept, the embodiment of the present application also provides a workshop communication resource selection device for implementing the above-mentioned workshop communication resource selection method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more workshop communication resource selection device embodiments provided below can refer to the limitations of the workshop communication resource selection method above, and will not be repeated here.

[0095] In an exemplary embodiment, a vehicle communication resource selection device is provided, comprising:

[0096] An energy feature map construction module is used to construct an energy feature map of a candidate resource library based on RB energy features;

[0097] A DNN network construction module is used to construct and train a DNN network based on the energy feature map, and model the V2V communication link quality related to the vehicle motion state as environmental state information; the V2V communication link quality is a V2V communication resource; the V2V communication resource is a resource block sequence consisting of 10 time domain subframes and 12 frequency domain subchannels;

[0098] A DQN application model building module is used to build a DQN application model based on the DNN network; the DQN application model is stored in the VUE storage space; the DQN application model includes an agent, an environment, a state, an action, and an immediate reward; the agent includes a main neural network and a target neural network; the main neural network and the target neural network have the same structure but different weight parameter vectors; the main neural network is used to calculate the approximate Q value of the main neural network under the current input state; the target neural network is used to calculate the approximate Q value of the target neural network under the transfer state; the transfer state is the state of the current input state at the next moment; the main neural network is a trained DNN network;

[0099] The resource block set selection module is used to select a resource block set matching the current V2V communication link quality from the candidate resource library using the DQN application model when the VUE triggers the selection of a resource block in any time slot.

[0100] In an exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store workshop communication resource selection data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a workshop communication resource selection method is implemented.

[0101] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the above method is implemented when the processor executes the computer program.

[0102] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the above method when executed by a processor.

[0103] In an exemplary embodiment, a computer program product is provided, including a computer program, which implements the above method when executed by a processor.

[0104] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ReadOnlyMemory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (Magnetoresistive RandomAccess Memory, MRAM), ferroelectric random access memory (Ferroelectric RandomAccess Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (RandomAccess Memory, RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0105] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0106] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0107] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for selecting workshop communication resources, characterized in that: The workshop communication resource selection method comprises: Based on the RB energy characteristics, an energy characteristic map of the candidate resource library is constructed; Based on the energy characteristic map, a DNN network is constructed and trained to model the V2V communication link quality related to the vehicle motion state as environmental state information; the V2V communication link quality is a V2V communication resource; the V2V communication resource is a resource block sequence consisting of 10 time domain subframes and 12 frequency domain subchannels; Based on the DNN network, a DQN application model is established; the DQN application model is stored in the VUE storage space; the DQN application model includes an agent, an environment, a state, an action, and an immediate reward; the agent includes a main neural network and a target neural network; the main neural network and the target neural network have the same structure but different weight parameter vectors; the main neural network is used to calculate the approximate Q value of the main neural network under the current input state; the target neural network is used to calculate the approximate Q value of the target neural network under the transfer state; the transfer state is the state of the current input state at the next moment; the main neural network is a trained DNN network; When the VUE triggers the selection of a resource block in any time slot, the DQN application model is used to select a set of resource blocks matching the current V2V communication link quality from the candidate resource pool.

2. The method for selecting workshop communication resources according to claim 1, characterized in that: Based on the RB energy characteristics, an energy characteristic map of the candidate resource library is constructed, including: Calculate the RSSI mean of the received signal in any time domain subframe; the RSSI mean is: in, is the RSSI mean of the received signal in the i-th time domain subframe; i is the time domain subframe number, i∈[1,10]; h is a measurement cycle in the perception window, h∈[1,H], H is the total number of measurement cycles in the perception window, T is the duration of the perception window, RRI is the resource reservation interval; Calculate the SINR mean of the RB set corresponding to any frequency domain subchannel in any time domain subframe; the SINR mean is in, is the SINR mean of the RB set corresponding to the j-th subchannel in the i-th subframe, j∈[1,12]; P s is the transmission power per resource block; G sr is the channel gain between the transmitter s and the receiver r; N0 is the noise power per resource block; K is the set of VUEs competing for the same resource block within the measurement period; G sk is the interference channel gain between the transmitting end s and the competing end k; the RB energy characteristics include all RSSI means and all SINR means; An energy characteristic spectrum of a candidate resource library is constructed according to the RSSI mean value and the SINR mean value.

3. The method for selecting workshop communication resources according to claim 2, characterized in that: Based on the energy feature map, a DNN network is constructed and trained to model the V2V communication link quality related to the vehicle motion state as environmental state information, specifically including: Based on the SINR mean, establishing an experience sample library; A plurality of experience samples are randomly selected from the experience sample library, and the DNN network is trained to model the V2V communication link quality related to the vehicle motion state as environmental state information; the experience samples include the main neural network input state, action, immediate reward and the transfer state at the next moment; wherein the main neural network input state and the transfer state represent an energy feature spectrum, the action is a set of time-frequency resource blocks, and the immediate reward is the total throughput obtained under the premise of the selected resources; the DNN network includes an input layer, a subframe iteration layer, a subchannel iteration layer and an output layer connected in sequence.

4. The method for selecting workshop communication resources according to claim 3, characterized in that: The training process of the DNN network specifically includes: Input the transfer state into the target neural network, execute the action corresponding to the transfer state, and output an approximate Q value; the approximate Q value includes the transfer state, the action corresponding to the transfer state, and the weight parameter vector of each edge of the target neural network; Determine a target output value of the target neural network according to the approximate Q value; According to the target output value and the approximate Q value of the main neural network, a loss function of the weight parameter vector of each edge of the DNN network is constructed, and a mean value of the loss function is determined; Using a stochastic gradient descent algorithm to minimize the mean of the loss function and update the weight parameter vectors of each edge of the DNN network; During the training process, the number of iterations is set at each interval, the weight parameter vectors of each edge of the DNN network are assigned to the target neural network, the weight parameter vectors of each edge of the target neural network are updated, and the training of the DNN network is completed.

5. The method for selecting workshop communication resources according to claim 4, characterized in that: The loss function is: L(θ)=[yQ(s,a;θ)] 2 ; Where y is the target output value, y=r+γmaxQ(s',a';θ - ), r is the immediate reward, γ is the discount factor, Q(s',a';θ - ) is the weight parameter vector of each edge of the target neural network, s' is the transition state, a' is the action corresponding to the transition state, θ - is the weight parameter vector of each edge of the target neural network; Q(s,a;θ) is the approximate Q value of the main neural network, s is the current input state of the main neural network, a is the action, and θ is the weight parameter vector of each edge of the DNN network.

6. The method for selecting workshop communication resources according to claim 1, characterized in that: When the VUE triggers the selection of a resource block in any time slot, the DQN application model is used to select a resource block set matching the current V2V communication link quality from the candidate resource library, specifically including: When the VUE triggers the selection of a resource block in any time slot, an energy characteristic spectrum of the candidate resource pool measured in the perception period is input into the DQN application model; Based on the energy feature map, the main neural network is used to iteratively output a set of resource blocks in each time slot time domain subframe, and the intelligent agent is instructed to select the maximum SINR mean value in the time domain subframe corresponding to the minimum RSSI mean value by using an ε-greedy algorithm; The maximum SINR mean is used as an output action to generate experience samples, and the FIFO method is used to update the retained experience samples to determine a resource block set that matches the current V2V communication link quality.

7. A vehicle communication resource selection device, characterized in that: The vehicle communication resource selection device comprises: An energy feature map construction module is used to construct an energy feature map of a candidate resource library based on RB energy features; A DNN network construction module is used to construct and train a DNN network based on the energy feature map, and model the V2V communication link quality related to the vehicle motion state as environmental state information; the V2V communication link quality is a V2V communication resource; the V2V communication resource is a resource block sequence consisting of 10 time domain subframes and 12 frequency domain subchannels; A DQN application model building module is used to build a DQN application model based on the DNN network; the DQN application model is stored in the VUE storage space; the DQN application model includes an agent, an environment, a state, an action, and an immediate reward; the agent includes a main neural network and a target neural network; the main neural network and the target neural network have the same structure but different weight parameter vectors; the main neural network is used to calculate the approximate Q value of the main neural network under the current input state; the target neural network is used to calculate the approximate Q value of the target neural network under the transfer state; the transfer state is the state of the current input state at the next moment; the main neural network is a trained DNN network; The resource block set selection module is used to select a resource block set matching the current V2V communication link quality from the candidate resource library using the DQN application model when the VUE triggers the selection of a resource block in any time slot.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the vehicle communication resource selection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the inter-vehicle communication resource selection method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the inter-vehicle communication resource selection method according to any one of claims 1 to 6 is implemented.

Images