Vehicle ad hoc network system and method based on deep reinforcement learning

Abstract

The invention discloses a vehicle ad hoc network system and method based on deep reinforcement learning, and relates to the technical field of vehicle networking communication. The system comprises a dynamic routing module which adjusts a relay path in real time by using a deep reinforcement learning algorithm and optimizes a network load; the relay signal transmission module is used for constructing a multi-hop relay link by means of network signals of other vehicles when the signals are weak; the multimode communication module is used for intelligently switching a V2X communication protocol and a Wi-Fi 6 communication protocol according to the signal strength and the network environment; the positioning cooperation module fuses UWB and IMU sensor data to realize high-precision inter-vehicle positioning; the security verification module is used for ensuring data security and identity credibility by adopting a national cipher SM4 encryption algorithm and a block chain smart contract; and the communication control module is used for coordinating data transmission, path selection and protocol switching. According to the invention, the problems of signal attenuation, protocol switching and safety in the dynamic environment of the Internet of Vehicles are effectively solved, and the communication capability and the system stability of the intelligent networked vehicle are improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle networking communication technology, and in particular to a vehicle self-organizing network system and method based on deep reinforcement learning. Background Art

[0002] With the rapid development of intelligent connected vehicle (ICV) technology, vehicle-to-vehicle communication (V2X) has become a key technology to improve traffic safety, optimize road utilization and enhance the performance of intelligent transportation systems (ITS). V2X communication enables information exchange between vehicles and other traffic participants (including other vehicles, infrastructure, pedestrians, etc.) to support applications such as vehicle platooning, autonomous driving, and collision warning. However, the existing V2X communication system still faces many technical challenges in actual deployment, especially in a dynamic network environment. How to ensure efficient data transmission, stable connection and security is still the focus of current research.

[0003] Although the traditional V2X communication system has improved the information interaction capability between vehicles to a certain extent, there are still many problems. First, the dynamic changes in network topology lead to unstable routing. Due to the mobility of vehicles, traditional static or semi-dynamic routing protocols are difficult to quickly adapt to topology changes, which can easily cause communication links to break and affect the reliability of data transmission. Secondly, multi-hop transmission is inefficient and network congestion is serious. When the single-hop communication signal attenuates, the data needs to be transmitted through multiple relay nodes, but the existing relay mechanism lacks intelligent optimization, which may lead to too many hops, increase latency and cause network congestion. In addition, the existing heterogeneous network switching method is not intelligent enough. Although the V2X system and communication technologies such as Wi-Fi 6 are complementary, they still rely on fixed thresholds when switching protocols, making it difficult to adapt to complex network environments, resulting in reduced communication quality. At the same time, the high-precision positioning capability of the existing V2X system is limited. Traditional GPS positioning is susceptible to interference in complex environments such as high-rise buildings and tunnels, and it is difficult to provide high-precision relative positioning information, which affects the effect of vehicle formation and collaborative perception. Finally, due to the openness of V2X communications, the system has risks in terms of data security and credibility. Malicious nodes may interfere with communications by forging identities or man-in-the-middle attacks. Existing authentication and encryption mechanisms are difficult to provide sufficient security while ensuring low latency. Summary of the invention

[0004] In response to the above problems, the present invention proposes a vehicle self-organizing network system and method based on deep reinforcement learning. Through deep reinforcement learning, network topology management, intelligent relay transmission, dynamic protocol switching, high-precision positioning fusion and blockchain security verification are optimized, thereby realizing a more efficient, safe and stable vehicle self-organizing network system, further improving the communication capabilities of intelligent connected vehicles and the overall operating efficiency of the transportation system.

[0005] In order to achieve the above object, the present invention is implemented by the following technical solutions: A vehicle self-organizing network system based on deep reinforcement learning, comprising: A dynamic routing module is used to collect vehicle density, signal strength and network load data in real time through a deep reinforcement learning algorithm, select relay nodes based on the data and adaptively adjust multi-hop paths, where the upper limit of the number of hops for path adjustment is dynamically optimized based on the network load; The relay signal transmission module is used to build a multi-hop relay link through the path selection result provided by the dynamic routing module when the signal strength is lower than the preset threshold, and borrow the network signal of other vehicles through wireless transmission; A multi-mode communication module, which is used to intelligently switch between the V2X communication protocol and the Wi-Fi 6 communication protocol according to the network environment and signal strength of the vehicle; The positioning coordination module is used to integrate the positioning data between vehicles, including using UWB technology and IMU sensor data to generate high-precision relative positioning information between vehicles, and perform positioning coordination based on this information; The security verification module is used to protect the security of data transmitted in the system. It uses encryption algorithms to encrypt data, verify the identity of relay nodes, and use blockchain smart contract mechanisms to ensure the validity and immutability of identity verification. A communication control module is used to coordinate data transmission, path selection and communication protocols in a vehicle ad hoc network system based on deep reinforcement learning.

[0006] As a preferred solution of the present invention, the dynamic routing module includes: The signal acquisition unit is used to collect vehicle density, signal strength RSSI and network load data in real time. The sampling period is 100 milliseconds. The RSSI threshold range is adjusted according to the vehicle density. When the vehicle density is high, the RSSI threshold range is set in the low range; when the vehicle density is low, the RSSI threshold range is set in the high range; The strategy decision unit adopts the deep Q network DQN algorithm. The input parameters include the RSSI threshold range and the bandwidth occupancy rate threshold range. The relay node selection and hop allocation strategy are output according to the input parameters. The instruction execution unit constructs a relay link according to the relay node selection and hop allocation strategy output by the policy decision unit, and adjusts the upper limit of the hop count according to the network load rate. When the network load rate exceeds the preset value for three consecutive sampling periods, the upper limit of the hop count is the preset value; when the network load rate does not exceed the preset value for three consecutive sampling periods, the upper limit of the hop count is another preset value.

[0007] As a preferred solution of the present invention, the strategy decision unit constructs an adaptive path optimization model based on an improved deep Q network (DQN) algorithm to solve the problems of fixed path selection, insufficient dynamic adjustment capability, and high computational complexity in the prior art, optimizes channel resource allocation and transmission paths, and adopts the following reward function: ; in: :Based on the vehicle-to-vehicle channel quality indicator value of V2X communication, dynamically weigh the communication quality of relay node selection; : Adaptively adjust the minimum channel quality threshold to ensure network stability in different scenarios; : Current channel load, dynamically optimize bandwidth resource allocation through reinforcement learning to reduce channel congestion; : The maximum available bandwidth of the channel, adaptively adjust the load balancing strategy in complex vehicle network environments; : The relative speed change between vehicles uses a nonlinear exponential function to adjust the path stability weight to reduce communication interruption caused by high-speed movement; : The average speed of the vehicle group to optimize the stability of data transmission and make route selection more intelligent; :The maximum communication distance is adjusted adaptively based on the channel status to improve network coverage; : The communication delay of the current link, using the delay optimization strategy to reduce the cumulative delay of multi-hop transmission; : The number of hops of the data packet is automatically optimized through reinforcement learning method to reduce the cumulative packet loss rate; , , , , : The adaptive optimization coefficient used to adjust the weights of each parameter, so that the network can dynamically adjust the optimization strategy under different vehicle environments.

[0008] Based on the reward function, the strategy decision unit uses reinforcement learning to dynamically optimize path selection, so that the system can perceive the communication environment in real time, intelligently adjust the channel load distribution, and optimize the relay node selection and hop count to reduce latency, improve link stability and increase data throughput. Compared with the existing path optimization methods based on static thresholds or fixed optimization strategies, the adaptive path optimization model of the present invention can dynamically optimize the communication path according to different vehicle network conditions, while ensuring network stability and reducing computational complexity. It is suitable for vehicle edge computing environments and improves the real-time performance and computational efficiency of the system.

[0009] As a preferred solution of the present invention, the relay signal transmission module includes: A signal strength detection unit is used to detect the signal strength of the current vehicle in real time. When the signal strength is lower than a preset signal strength threshold, the relay signal transmission function is triggered; The relay path selection unit selects a suitable relay node to build a relay link according to the path selection result provided by the dynamic routing module; The wireless communication unit is used to borrow the network signal of other vehicles through wireless communication to realize a multi-hop relay link, and transmit the network signal back to the current vehicle or other vehicles that need network resources through the relay link.

[0010] As a preferred solution of the present invention, the multi-mode communication module includes: A protocol switching unit, configured to select one of a V2X communication protocol and a Wi-Fi 6 communication protocol for switching according to a network environment and available signal strength, wherein the V2X communication protocol operates at a frequency band of 5.9 GHz, the Wi-Fi 6 communication protocol operates at a frequency band of 5 GHz or 6 GHz, and the frequency band interval between the two is not less than 200 MHz; A signal strength monitoring unit, used to monitor the current signal strength in real time, and when the signal strength is lower than a preset threshold, switch to other communication protocols for data transmission; The protocol adaptation strategy unit selects the communication protocol for data transmission according to the distance between the vehicle and other vehicles, bandwidth requirements and signal stability.

[0011] As a preferred solution of the present invention, the positioning collaboration module includes: A positioning information receiving unit, used to receive positioning information from other vehicles and transmit the received positioning data to the positioning processing unit; A positioning processing unit, used to fuse positioning data from UWB technology and IMU sensors to generate high-precision relative positioning information between vehicles, wherein the operating frequency band of the UWB technology is 6.5 GHz to 7.5 GHz; The collaborative positioning unit is used to coordinate positioning based on the relative positioning information between vehicles to ensure high-precision positioning results in complex environments.

[0012] As a preferred solution of the present invention, the security verification module includes: Data encryption unit, used to encrypt data transmitted in the network using the national secret SM4 algorithm to ensure the confidentiality and integrity of data during transmission; The identity verification unit is used to verify the identity of the relay node through the blockchain smart contract, ensure the credibility of the relay node, and prevent malicious nodes from interfering with the system; The secure communication unit is used to ensure the security of the communication link during data transmission and use digital certificates to authenticate both communicating parties to prevent data from being tampered with during transmission.

[0013] As a preferred solution of the present invention, the blockchain smart contract includes: The contract creation unit is used to generate smart contracts and sign the contract content through public key encryption technology to ensure the legality and non-tamperability of the contract; The contract execution unit is used to execute the verification logic in the smart contract and determine the legitimacy of the relay node based on the identity information of the relay node and the preset verification rules; The contract recording unit is used to record the execution results of each smart contract on the blockchain, including the verification records and verification results of the relay nodes, to ensure that all operations are traceable and cannot be tampered with.

[0014] A vehicle self-organizing network method based on deep reinforcement learning includes the following steps: Step 1: Collect vehicle density, signal strength RSSI and network load data in real time; Step 2: Select relay nodes through deep reinforcement learning algorithm; Step 3: Adjust the multi-hop path according to the network load and optimize the upper limit of the number of hops of the path; Step 4: When the signal strength is lower than the preset threshold, the relay signal transmission module selects a path and builds a multi-hop relay link to borrow the network signal of other vehicles through wireless transmission; Step 5: Switch between the V2X communication protocol and the Wi-Fi 6 communication protocol according to the network environment and signal strength. The V2X communication protocol operates at a frequency band of 5.9 GHz, the Wi-Fi 6 communication protocol operates at a frequency band of 5 GHz or 6 GHz, and the frequency band interval is greater than or equal to 200 MHz. Step 6: Fusion of data from UWB technology and IMU sensors generates relative positioning information between vehicles, and performs positioning coordination based on this information; Step 7: Encrypt the data transmitted on the network, verify the identity of the relay node, and use the blockchain smart contract mechanism to ensure the validity and immutability of identity authentication; Step 8: Coordinate data transmission, path selection, and communication protocols to ensure the normal operation of the vehicle self-organizing network system based on deep reinforcement learning.

[0015] As a preferred solution of the present invention, based on the real-time relative position data between vehicles, a weighted factor is used to calculate the priority of the candidate relay node and adjust the path selection strategy; Based on traffic flow monitoring data, combined with the time series prediction model, the path hop allocation is optimized, and the maximum hop limit is dynamically adjusted to adapt to network load conditions with different traffic densities.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention collects vehicle density, signal strength and network load data in real time through a deep reinforcement learning algorithm, adaptively adjusts the multi-hop path and relay node selection, and improves the stability and reliability of data transmission. Compared with traditional static or semi-dynamic routing protocols, the system is more adaptable to the high dynamic change environment of the vehicle network and ensures the stability of the communication link. When the vehicle signal strength is lower than the preset threshold, the system automatically triggers the multi-hop relay link, supplements the communication with the help of the network signal of other vehicles, avoids network interruption caused by signal attenuation, thereby expanding the communication coverage and improving the overall connection quality. In addition, the system can intelligently switch V2X and Wi-Fi 6 protocols according to different network environments and signal strengths, optimize bandwidth utilization while reducing latency, and ensure the communication quality of the Internet of Vehicles in various application scenarios. In terms of positioning, the system integrates UWB technology and IMU sensor data to achieve high-precision relative positioning between vehicles, and can perform positioning coordination in complex environments, improving the safety and efficiency of autonomous driving and intelligent transportation systems. At the same time, the present invention adopts the national secret SM4 encryption algorithm and blockchain smart contract technology to ensure the security of data during transmission, and to verify the identity of relay nodes, prevent malicious node attacks, and improve the security and anti-tampering capabilities of the system. The present invention optimizes the path selection, communication protocol, signal transmission and security mechanism of the vehicle-mounted ad hoc network through deep reinforcement learning, significantly improving the communication efficiency, reliability and security of the Internet of Vehicles system, and can effectively cope with the dynamics and complexity of Internet of Vehicles applications, providing more efficient and stable technical support for intelligent networked vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them: Figure 1 A system modular structure diagram of an embodiment of the present invention; Figure 2 The figure is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0019] like Figure 1 As shown in FIG. 1 , an embodiment of the present invention is provided, which provides a vehicle self-organizing network system based on deep reinforcement learning, including: (1) Dynamic routing module It is used to collect vehicle density, signal strength and network load data in real time through deep reinforcement learning algorithms, select relay nodes based on the data and adaptively adjust multi-hop paths, where the upper limit of the number of hops for path adjustment is dynamically optimized based on the network load conditions; This embodiment aims at the dynamic routing problem in vehicle-to-everything (V2X) communications. The method can collect vehicle density, signal strength, and network load data in real time, and adaptively select the optimal relay node through a reinforcement learning algorithm to optimize multi-hop paths, thereby reducing communication delays and improving network stability.

[0020] The dynamic routing module of this embodiment mainly includes a signal acquisition unit, a strategy decision unit and an instruction execution unit, and each unit works together to ensure the stability, real-time and high efficiency of data transmission.

[0021] The dynamic routing module includes: Signal acquisition unit The signal acquisition unit is mainly used to obtain key parameters of the vehicle network environment in real time, including: Vehicle density detection: Based on the number of wireless communication connections between vehicles, the number of neighboring vehicles per unit time is calculated to determine the dynamic nature of the network topology. When the density is high, channel competition is fierce and data conflicts are likely to occur; when the density is low, there may be signal coverage blind spots.

[0022] Signal strength (RSSI) detection: When the vehicle density is high, the RSSI threshold range is dynamically adjusted to a lower level to reduce interference in long-distance communications and improve the stability of short-distance communications; When the vehicle density is low, the RSSI threshold range is increased to ensure that long-distance communications can still establish an effective connection.

[0023] Network load monitoring: Determine the current network load by measuring bandwidth occupancy, channel utilization and congestion, and perform sampling every 100ms to ensure real-time data.

[0024] This unit ensures that the dynamic routing module is always optimized based on the latest network environment data, improving the stability of data transmission.

[0025] The strategy decision unit adopts the deep Q network DQN algorithm. The input parameters include the RSSI threshold range and the bandwidth occupancy rate threshold range. The relay node selection and hop allocation strategy are output according to the input parameters. The instruction execution unit constructs a relay link according to the relay node selection and hop allocation strategy output by the policy decision unit, and adjusts the upper limit of the hop count according to the network load rate. When the network load rate exceeds the preset value for three consecutive sampling periods, the upper limit of the hop count is the preset value; when the network load rate does not exceed the preset value for three consecutive sampling periods, the upper limit of the hop count is another preset value.

[0026] In one embodiment, the policy decision unit constructs an adaptive path optimization model based on an improved deep Q network DQN algorithm to optimize channel resource allocation and transmission paths, and adopts the following reward function: ; in: : Vehicle-to-vehicle channel quality indicator value based on V2X communication; : The lowest available threshold of channel quality; : Current channel load, indicating bandwidth occupancy; : The maximum available bandwidth of the channel; : Relative speed change between vehicles; : The average speed of the vehicle group; : Maximum communication distance; : Communication delay of the current link; : Number of hops of the data packet; , , , , : Optimization coefficient used to adjust the weight of each parameter; Based on the reward function, the strategy decision unit uses the reinforcement learning method to calculate the communication environment parameters in real time, dynamically adjust the channel load distribution, optimize the relay node selection and the number of hops, determine the data transmission path based on the path optimization model, and adjust the maximum communication distance and delay control strategy according to the channel status.

[0027] The dynamic routing module of this embodiment has the following innovative features: Intelligent routing optimization based on reinforcement learning: Compared with traditional static or historical data-based routing methods, the DQN algorithm can adaptively adjust the path and improve data transmission efficiency; Real-time signal acquisition and network adaptability: Dynamically adjust the communication strategy based on RSSI thresholds, so that the system can adapt to network environments with different vehicle densities; Adaptive hop optimization: Reinforcement learning algorithm optimizes multi-hop path selection strategy to ensure end-to-end delay is minimized; Path prediction mechanism: Combines vehicle motion trajectory and signal strength prediction to adjust the data transmission path in advance and reduce the occurrence of link interruption.

[0028] This embodiment is applicable to the following typical application scenarios: Intelligent transportation system: used for optimizing V2X communication in the Internet of Vehicles to improve the efficiency of road information sharing and collaborative control; Highway fleet communication: supports intelligent fleet management and ensures low-latency and high-stability information transmission between vehicles; Disaster emergency response network: When fixed communication facilities are damaged, vehicle ad hoc networks are used for emergency data transmission to ensure uninterrupted communications.

[0029] This embodiment introduces in detail a dynamic routing optimization method based on deep reinforcement learning, including a collaborative working mechanism of a signal acquisition unit, a strategy decision unit, and an instruction execution unit, and ensures that the system can adapt to a complex Internet of Vehicles environment through reinforcement learning algorithms, adaptive path optimization models, and dynamic hop count adjustment, thereby improving the reliability of data transmission and communication efficiency. The innovation of the present invention is to combine deep reinforcement learning and real-time path optimization technology, so that the system can continuously optimize the data transmission path in a dynamic network environment, thereby improving the overall performance of the vehicle self-organizing network.

[0030] (2) Relay signal transmission module When the signal strength is lower than the preset threshold, the dynamic routing module is used to build a multi-hop relay link and borrow the network signal of other vehicles through wireless transmission; In this embodiment, the relay signal transmission module is designed to build a multi-hop relay link through the path selection results provided by the dynamic routing module when the signal strength is lower than the preset threshold, so as to ensure the continuous transmission of data and the stability of communication. The module consists of a signal strength detection unit, a relay path selection unit and a wireless propagation unit, ensuring that the system can continue to transmit data with the help of network resources of other vehicles in an environment with unstable signals or insufficient coverage.

[0031] Signal strength detection unit The signal strength detection unit is responsible for real-time monitoring of the vehicle's network signal strength and evaluating the signal quality. When the vehicle's signal strength is lower than the preset threshold, the relay signal transmission function is triggered. In order to ensure the accuracy and timeliness of the detection, the signal strength detection unit uses a high-frequency (e.g. 100ms) sampling cycle to provide real-time feedback on the signal status of the vehicle's environment. This unit uses the RSSI (received signal strength indication) value to determine whether the current signal meets the communication requirements. When the signal strength is insufficient, the system automatically triggers the relay link construction process. Through accurate signal strength monitoring, network transmission interruptions can be effectively avoided, thereby ensuring stable data transmission.

[0032] Relay Path Selection Unit The relay path selection unit selects appropriate relay nodes to build relay links based on the path selection results provided by the dynamic routing module. The core of this unit is to optimize path selection in combination with dynamic routing algorithms. The dynamic routing module uses a deep reinforcement learning algorithm to adaptively adjust the number of path hops based on data such as vehicle density, signal strength, and network load. When the signal strength is lower than the threshold, the relay path selection unit selects the vehicle closest to the current vehicle, with better signal quality and lower load as the relay node based on these real-time environmental parameters. This can effectively avoid the degradation of path quality caused by improper relay node selection and ensure smooth and low-latency data transmission.

[0033] In addition, the relay path selection unit also needs to consider the selection of multi-hop paths. By calculating the relative positions and signal strengths between vehicles in real time, the number of hops in the multi-hop relay path is optimized to minimize transmission delay and maximize transmission bandwidth. This path selection is not only based on a single shortest path or lowest delay, but also comprehensively considers path stability, channel load, and available bandwidth to form an efficient relay link.

[0034] Wireless communication unit The role of the wireless communication unit is to use wireless communication to borrow the network signals of other vehicles to build a multi-hop relay link when the signal strength is insufficient. In the traditional network architecture, when the signal is weak, the communication link often cannot maintain stability. In this system, the wireless communication unit uses the network signals of surrounding vehicles to build a relay link to transmit the network signal back to the current vehicle or other vehicles that need network resources.

[0035] The wireless propagation unit selects the most suitable path for data transmission based on the signal quality of multiple candidate relay nodes. When the vehicle receives a signal forwarded through a relay node, the data is gradually transmitted through the link until it reaches the final destination node. In order to further improve the efficiency of the relay link, the wireless propagation unit can introduce signal enhancement and beamforming technology to improve the signal quality and transmission stability during multi-hop transmission. In specific implementation, the wireless propagation unit must not only ensure reliable data transmission, but also optimize the multi-path propagation strategy to avoid packet loss due to signal interference or path loss.

[0036] Through this wireless propagation mechanism, the system can fully utilize the wireless communication capabilities between vehicles, expand the signal coverage, and enhance the connectivity of the vehicle ad hoc network. In areas with low signal strength, such as busy urban blocks and tunnels, it can ensure continuous network connection between vehicles.

[0037] The relay signal transmission module of this embodiment has the following advantages: Flexible dynamic routing adjustment: The dynamic routing module based on deep reinforcement learning can dynamically adjust the relay path according to the real-time network conditions, avoiding network bottlenecks caused by static path planning.

[0038] Optimized multi-hop relay mechanism: By intelligently selecting relay nodes and optimizing the number of path hops, transmission delays are reduced and efficient use of the network is ensured.

[0039] Wireless communication capability: By borrowing network signals from other vehicles, the relay link can continuously transmit data in weak signal environments, greatly improving communication reliability.

[0040] Adaptive path selection: The relay path selection unit maximizes the efficiency of data transmission by calculating the best path in real time, reducing the performance degradation caused by improper path selection.

[0041] The relay signal transmission module in this embodiment can effectively solve the problems of signal attenuation and network load through precise signal strength monitoring, intelligent relay path selection and efficient wireless propagation mechanism, and improve the stability and reliability of the vehicle self-organizing network system in complex dynamic environments.

[0042] (3) Multi-mode communication module Used to intelligently switch between the V2X communication protocol and the Wi-Fi 6 communication protocol based on the network environment and signal strength of the vehicle; This embodiment provides a multi-mode communication module that intelligently switches between the V2X communication protocol and the Wi-Fi 6 communication protocol. The module can intelligently switch between the V2X communication protocol and the Wi-Fi 6 communication protocol according to the network environment and signal strength of the vehicle to ensure the stability and efficiency of communication. The working principle and structure of the multi-mode communication module include a protocol switching unit, a signal strength detection unit, and a protocol adaptation strategy unit, which work together to automatically select the optimal communication protocol according to different network conditions and communication requirements.

[0043] Protocol switching unit The protocol switching unit is responsible for selecting one of the V2X communication protocol and Wi-Fi 6 communication protocol for switching based on the real-time network environment and signal strength. The specific working method is as follows: V2X communication protocol: This protocol operates in the 5.9GHz frequency band and is mainly used for efficient communication between vehicles (V2V), vehicles and infrastructure (V2I), and vehicles and pedestrians (V2P). This protocol has low latency and high reliability in the Internet of Vehicles, and is particularly suitable for real-time, low-latency application scenarios.

[0044] Wi-Fi 6 communication protocol: The Wi-Fi 6 protocol operates in the 5GHz or 6GHz frequency bands, providing higher bandwidth and larger network capacity, and is suitable for data-intensive applications such as high-definition video transmission, real-time map updates, etc.

[0045] The protocol switching unit selects the appropriate communication protocol based on the following factors: Signal strength: When the signal strength of a communication protocol is lower than the set threshold, switch to another protocol to ensure communication quality; Network environment: According to the bandwidth requirements and stability of the current network environment, select the appropriate communication protocol to achieve the best data transmission effect; Frequency band spacing: The frequency band spacing between V2X and Wi-Fi 6 protocols should be no less than 200MHz to avoid interference between communication protocols.

[0046] Signal strength detection unit The signal strength detection unit is responsible for real-time monitoring of the current signal strength. When the unit is working, it will detect the following information: Signal strength assessment: Real-time assessment of the signal strength of the current V2X communication protocol and Wi-Fi 6 communication protocol, and calculation of the signal quality (RSSI) of each protocol. When the signal strength of a protocol is lower than the set threshold, the system will trigger the protocol switching function.

[0047] Switching trigger condition: When the signal strength is detected to be lower than the preset threshold, the signal strength detection unit will send a signal switching request to the protocol switching unit to switch to another communication protocol. The signal strength detection unit can respond quickly to ensure timely switching when the signal is weak to avoid communication interruption.

[0048] The design of this unit ensures that the vehicle can maintain a stable communication connection in different environments, whether in high-density urban areas or places far away from base stations with weak signals.

[0049] Protocol Adaptation Policy Unit The protocol adaptation strategy unit selects the most suitable communication protocol for data transmission based on the following factors: Distance between vehicles: When the distance between vehicles is close, the Wi-Fi 6 communication protocol provides higher bandwidth and is suitable for data-intensive applications; when the distance between vehicles is far, the V2X communication protocol can maintain lower latency and higher reliability.

[0050] Bandwidth requirements: If the bandwidth requirements of the current communication task are large (such as video streaming, high-definition map updates, etc.), the Wi-Fi 6 protocol can provide a higher transmission rate; if the requirements are lower, the V2X protocol can ensure real-time performance and low latency.

[0051] Signal stability: If the signal in the current communication environment is unstable, the protocol adaptation strategy unit will select a communication protocol with higher stability to ensure that data transmission will not experience packet loss.

[0052] Through these strategies, the protocol adaptation strategy unit flexibly selects the communication protocol according to the real-time status of the vehicle, network environment and data requirements, and optimizes the efficiency and stability of data transmission.

[0053] The multi-mode communication module in this embodiment has the following advantages: Intelligent protocol switching: The system adaptively selects the optimal protocol based on real-time signal strength, network environment and bandwidth requirements to ensure stable communication quality; Improve network efficiency: By selecting appropriate protocols for data transmission, improve network bandwidth utilization and transmission rate; Flexible response to changing environments: Whether in densely populated urban areas or suburban areas, or in environments far away from base stations, the system can flexibly respond to different signal strengths and maintain stable communications; Optimize resource allocation: Through intelligent switching and bandwidth demand judgment, the system avoids frequent switching and unnecessary waste of resources, and improves the overall efficiency of communication.

[0054] This embodiment intelligently switches the V2X communication protocol and the Wi-Fi 6 communication protocol through the coordinated work of the protocol switching unit, the signal strength monitoring unit, and the protocol adaptation strategy unit, ensuring that the vehicle can maintain a stable and efficient communication connection in different network environments. This technology can improve the flexibility, stability, and data transmission efficiency of the Internet of Vehicles system, and is an effective solution to meet the changing needs of intelligent transportation systems.

[0055] (4) Positioning collaboration module Used to fuse the positioning data between vehicles, including using UWB technology and IMU sensor data to generate high-precision relative positioning information between vehicles, and perform positioning coordination based on this information; This embodiment provides a data fusion technology based on ultra-wideband (UWB) technology and inertial measurement unit (IMU) sensors to achieve high-precision relative positioning between vehicles and ensure accurate positioning of intelligent connected vehicles (ICVs) in complex environments. The module mainly includes a positioning information receiving unit, a positioning processing unit, and a collaborative positioning unit, which work together to achieve efficient positioning coordination.

[0056] The positioning collaboration module includes: A positioning information receiving unit, used to receive positioning information from other vehicles and transmit the received positioning data to the positioning processing unit; In this embodiment, the main function of the positioning information receiving unit is to receive positioning information from other vehicles, including: UWB ranging data: provides high-precision absolute distance information; IMU sensor data: provides the vehicle's acceleration and angular velocity information for short-term position estimation; GNSS data (optional): Provides additional positioning data when the signal is good.

[0057] This unit ensures that accurate positioning information can be shared between vehicles, providing reliable input for subsequent data fusion.

[0058] A positioning processing unit, used to fuse positioning data from UWB technology and IMU sensors to generate high-precision relative positioning information between vehicles, wherein the operating frequency band of the UWB technology is 6.5 GHz to 7.5 GHz; In this embodiment, the positioning processing unit fuses UWB and IMU data and generates high-precision relative positioning information through the extended Kalman filter (EKF) algorithm. Specific working method: UWB data: provides centimeter-level accuracy and is suitable for stable signal environments; IMU data: Provides short-term relative motion estimates when UWB signals are obstructed.

[0059] The unit can dynamically weight and fuse data, optimize positioning accuracy, and correct errors in real time.

[0060] The collaborative positioning unit is used to coordinate positioning based on the relative positioning information between vehicles to ensure high-precision positioning results in complex environments.

[0061] In this embodiment, the collaborative positioning unit performs positioning coordination based on the relative positioning information between vehicles to ensure high-precision positioning in complex environments. The main functions include: Fleet formation coordination: ensure that vehicles in the fleet maintain precise spacing to improve driving efficiency; Complex environment positioning: In tunnels or high-density urban areas, high-precision positioning is maintained using UWB and IMU data; Emergency avoidance: Based on real-time positioning information, quickly predict and avoid collision risks to ensure driving safety.

[0062] The unit ensures that the vehicle maintains a stable position even when the GPS signal is blocked.

[0063] This embodiment provides high-precision, low-latency relative positioning between vehicles through the fusion of UWB and IMU data, ensuring efficient and reliable positioning coordination even in complex environments, and improving the driving safety and autonomous driving capabilities of intelligent connected vehicles.

[0064] (5) Security Verification Module It is used to protect the security of data transmitted in the system, encrypt data using encryption algorithms, verify the identity of relay nodes, and use blockchain smart contract mechanisms to ensure the validity and immutability of identity authentication; This embodiment is used to protect the security of data in the vehicle ad hoc network system and prevent malicious attacks and identity forgery. The module mainly includes a data encryption unit, an identity authentication unit and a secure communication unit, and ensures communication security through end-to-end encryption and distributed identity authentication.

[0065] The security verification module includes: Data encryption unit, used to encrypt data transmitted in the network using the national secret SM4 algorithm to ensure the confidentiality and integrity of data during transmission; The data encryption unit uses the national SM4 symmetric encryption algorithm to encrypt the data in the Internet of Vehicles to ensure the confidentiality and integrity of the data. Specific encryption methods include: Data transmission encryption: All data packets for inter-vehicle communication are encrypted using SM4 before being sent, and the receiver uses the corresponding key to decrypt them to prevent data from being stolen by man-in-the-middle attacks (MITM).

[0066] Dynamic key update mechanism: trigger key replacement regularly or based on security events to prevent security risks caused by key leakage.

[0067] Integrity check: Combined with hash check (SHA-256), it ensures that the data has not been tampered with and improves data integrity.

[0068] The identity verification unit is used to verify the identity of the relay node through the blockchain smart contract, ensure the credibility of the relay node, and prevent malicious nodes from interfering with the system; In this embodiment, the identity verification unit verifies the identity of the relay node through the blockchain smart contract to prevent malicious nodes from interfering with system communications. Its main functions include: Distributed identity authentication: All legitimate relay nodes register unique identity information on the blockchain to prevent identity forgery.

[0069] Real-time verification mechanism: Before data transmission, the system verifies the identity of the relay node through smart contracts and refuses unauthorized nodes from participating in communication.

[0070] Tamper-resistance: Due to the decentralized nature of blockchain, identity verification records cannot be tampered with, which improves the security and credibility of the system.

[0071] The secure communication unit is used to ensure the security of the communication link during data transmission and use digital certificates to authenticate both communicating parties to prevent data from being tampered with during transmission.

[0072] In this embodiment, the secure communication unit is used to ensure the security of the communication link and prevent data from being tampered with or forged during transmission. Specific measures include: Digital certificate authentication: The system uses PKI (public key infrastructure) to authenticate the identities of both communicating parties to prevent man-in-the-middle attacks.

[0073] Secure handshake mechanism: Before establishing a communication connection, the vehicle must complete a secure handshake based on the TLS 1.3 protocol to ensure the legitimacy and security of the connection.

[0074] Data traceability: Each data interaction will generate a unique security identifier and store it on the blockchain to track the source of the data.

[0075] Among them, blockchain smart contracts include: The contract creation unit is used to generate smart contracts and sign the contract content through public key encryption technology to ensure the legality and non-tamperability of the contract; The contract execution unit is used to execute the verification logic in the smart contract and determine the legitimacy of the relay node based on the identity information of the relay node and the preset verification rules; The contract recording unit is used to record the execution results of each smart contract on the blockchain, including the verification records and verification results of the relay nodes, to ensure that all operations are traceable and cannot be tampered with.

[0076] This embodiment implements end-to-end data protection, distributed identity authentication, and tamper-proof communication through SM4 encryption + blockchain smart contracts, effectively preventing data leakage, identity forgery, and network attacks, and improving the security and credibility of the vehicle self-organizing network system.

[0077] (6) Communication control module Used to coordinate data transmission, path selection and communication protocols in vehicle self-organizing network systems based on deep reinforcement learning.

[0078] The working principle of the communication control module is as follows: Deep Reinforcement Learning Model: Input: The module obtains real-time information such as vehicle density, signal strength, network load, etc. through the signal acquisition unit, and passes this information as input to the deep reinforcement learning algorithm.

[0079] Output: The deep reinforcement learning algorithm outputs the optimal path selection and protocol switching strategy based on the current state of the network environment.

[0080] Data Transfer Coordination: The system adjusts the transmission order of data packets according to the current network load to ensure that high-priority data packets are transmitted first and avoid congestion or delays.

[0081] Path selection coordination: When dynamically selecting a transmission path, the communication control module dynamically calculates the best path based on a deep reinforcement learning algorithm, taking into account factors such as signal strength, bandwidth, and latency. The number of path hops and transmission nodes can be flexibly adjusted based on the real-time network status and the preset maximum number of hops.

[0082] Protocol switch coordination: In terms of communication protocol selection, the communication control module intelligently switches based on the current signal quality, bandwidth requirements, and network environment (such as the compatibility of V2X and Wi-Fi 6). If the signal quality is good and the bandwidth requirement is large, the Wi-Fi 6 protocol will be given priority; if low-latency communication is required and the bandwidth requirement is low, the V2X protocol will be given priority.

[0083] This embodiment realizes intelligent communication management, dynamic path selection and adaptive protocol switching through deep reinforcement learning algorithm. By coordinating data transmission, path selection and communication protocol switching, the communication control module optimizes the data transmission efficiency and network resource utilization of the Internet of Vehicles system, ensuring the efficient and stable operation of the vehicle self-organizing network in complex environments.

[0084] like Figure 2 As shown in FIG. 1 , another embodiment of the present invention is provided, which provides a vehicle self-organizing network method based on deep reinforcement learning, comprising the following steps: Step 1: Collect vehicle density, signal strength RSSI and network load data in real time; Step 2: Select relay nodes through deep reinforcement learning algorithm; Step 3: Adjust the multi-hop path according to the network load and optimize the upper limit of the number of hops of the path; Step 4: When the signal strength is lower than the preset threshold, the relay signal transmission module selects a path and builds a multi-hop relay link to borrow the network signal of other vehicles through wireless transmission; Step 5: Switch between the V2X communication protocol and the Wi-Fi 6 communication protocol according to the network environment and signal strength. The V2X communication protocol operates at a frequency band of 5.9 GHz, the Wi-Fi 6 communication protocol operates at a frequency band of 5 GHz or 6 GHz, and the frequency band interval is greater than or equal to 200 MHz. Step 6: Fusion of data from UWB technology and IMU sensors generates relative positioning information between vehicles, and performs positioning coordination based on this information; Step 7: Encrypt the data transmitted on the network, verify the identity of the relay node, and use the blockchain smart contract mechanism to ensure the validity and immutability of identity authentication; Step 8: Coordinate data transmission, path selection, and communication protocols to ensure the normal operation of the vehicle self-organizing network system based on deep reinforcement learning.

[0085] Furthermore, based on the real-time relative position data between vehicles, a weighted factor is used to calculate the priority of candidate relay nodes and adjust the path selection strategy; Based on traffic flow monitoring data, combined with the time series prediction model, the path hop allocation is optimized, and the maximum hop limit is dynamically adjusted to adapt to network load conditions with different traffic densities.

[0086] In summary, the present invention provides a vehicle self-organizing network system and method based on deep reinforcement learning, which realizes efficient, stable and secure data transmission in the Internet of Vehicles environment through key technologies such as intelligent routing optimization, relay signal transmission, multi-mode communication protocol switching, high-precision positioning collaboration and blockchain security verification. Compared with the prior art, the present invention can adaptively adjust the routing strategy, optimize the communication path in a complex dynamic environment, improve the stability of data transmission, and enhance signal coverage through an intelligent relay mechanism to avoid communication interruption. In addition, the multi-mode communication protocol switching technology is adopted to ensure that the system can adaptively adjust the communication mode in different scenarios, optimize bandwidth utilization, reduce latency, and improve data throughput. At the same time, the UWB and IMU sensor data are integrated to achieve high-precision relative positioning of vehicles and enhance the collaborative perception ability of intelligent networked vehicles. Through blockchain smart contracts and national secret SM4 encryption technology, data leakage, forgery and malicious attacks can be effectively prevented to ensure the security and reliability of Internet of Vehicles communications.

[0087] In summary, the present invention is not only suitable for intelligent transportation systems, autonomous driving fleet management and highway networking communications, but can also be widely used in emergency communications, autonomous unmanned systems and smart city infrastructure, providing strong technical support for the development of future intelligent connected vehicles, and has broad market application prospects and industrialization value.

[0088] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0089] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0090] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A vehicle self-organizing network system based on deep reinforcement learning, characterized in that: include: A dynamic routing module is used to collect vehicle density, signal strength and network load data in real time through a deep reinforcement learning algorithm, select relay nodes based on the data and adaptively adjust multi-hop paths, where the upper limit of the number of hops for path adjustment is dynamically optimized based on the network load; The relay signal transmission module is used to build a multi-hop relay link through the path selection result provided by the dynamic routing module when the signal strength is lower than the preset threshold, and borrow the network signal of other vehicles through wireless transmission; A multi-mode communication module, which is used to intelligently switch between the V2X communication protocol and the Wi-Fi 6 communication protocol according to the network environment and signal strength of the vehicle; Positioning coordination module, used to integrate positioning data between vehicles, including using UWB technology and IMU sensor data to generate high-precision relative positioning information between vehicles and perform positioning coordination; The security verification module is used to protect the security of data transmitted in the system. It uses encryption algorithms to encrypt data, verify the identity of relay nodes, and use blockchain smart contract mechanisms to ensure the validity and immutability of identity verification. A communication control module is used to coordinate data transmission, path selection and communication protocols in a vehicle ad hoc network system based on deep reinforcement learning.

2. The vehicle self-organizing network system based on deep reinforcement learning according to claim 1 is characterized in that: The dynamic routing module includes: Signal acquisition unit, used to collect vehicle density, signal strength RSSI and network load data in real time, with a sampling period of 100 milliseconds; The strategy decision unit adopts the deep Q network DQN algorithm. The input parameters include the RSSI threshold range and the bandwidth occupancy rate threshold range. The relay node selection and hop allocation strategy are output according to the input parameters. The instruction execution unit constructs a relay link according to the relay node selection and hop allocation strategy output by the policy decision unit, and adjusts the hop limit according to the network load rate. When the network load rate exceeds the preset value for three consecutive sampling periods, the hop limit is the preset value.

3. The vehicle self-organizing network system based on deep reinforcement learning according to claim 2 is characterized in that: The strategy decision unit builds an adaptive path optimization model based on the improved deep Q network DQN algorithm, optimizes channel resource allocation and transmission path, and adopts the following reward function: ； in: : Vehicle-to-vehicle channel quality indicator value based on V2X communication; : The lowest available threshold of channel quality; : Current channel load, indicating bandwidth occupancy; : The maximum available bandwidth of the channel; : Relative speed change between vehicles; : The average speed of the vehicle group; : Maximum communication distance; : Communication delay of the current link; : Number of hops of the data packet; , , , , : Optimization coefficient used to adjust the weight of each parameter; Based on the reward function, the strategy decision unit uses the reinforcement learning method to calculate the communication environment parameters in real time, dynamically adjust the channel load distribution, optimize the relay node selection and the number of hops, determine the data transmission path based on the path optimization model, and adjust the communication distance and delay control strategy according to the channel status.

4. The vehicle self-organizing network system based on deep reinforcement learning according to claim 1, characterized in that: The relay signal transmission module comprises: A signal strength detection unit is used to detect the signal strength of the current vehicle in real time. When the signal strength is lower than a preset signal strength threshold, the relay signal transmission function is triggered; The relay path selection unit selects a suitable relay node to build a relay link according to the path selection result provided by the dynamic routing module; The wireless communication unit is used to borrow the network signal of other vehicles through wireless communication to realize a multi-hop relay link, and transmit the network signal back to the current vehicle or other vehicles that need network resources through the relay link.

5. The vehicle self-organizing network system based on deep reinforcement learning according to claim 1, characterized in that: The multimode communication module comprises: A protocol switching unit, configured to select one of a V2X communication protocol and a Wi-Fi 6 communication protocol for switching according to a network environment and available signal strength, wherein the V2X communication protocol operates at a frequency band of 5.9 GHz, the Wi-Fi 6 communication protocol operates at a frequency band of 5 GHz or 6 GHz, and the frequency band interval between the two is not less than 200 MHz; A signal strength monitoring unit, used to monitor the current signal strength in real time, and when the signal strength is lower than a preset threshold, switch to other communication protocols for data transmission; The protocol adaptation strategy unit selects the communication protocol for data transmission according to the distance between the vehicle and other vehicles, bandwidth requirements and signal stability.

6. The vehicle self-organizing network system based on deep reinforcement learning according to claim 1, characterized in that: The positioning coordination module includes: A positioning information receiving unit, used to receive positioning information from other vehicles and transmit the received positioning data to the positioning processing unit; A positioning processing unit, used to fuse positioning data from UWB technology and IMU sensors to generate high-precision relative positioning information between vehicles, wherein the operating frequency band of the UWB technology is 6.5 GHz to 7.5 GHz; The collaborative positioning unit is used to coordinate positioning based on the relative positioning information between vehicles to ensure high-precision positioning results in complex environments.

7. The vehicle self-organizing network system based on deep reinforcement learning according to claim 1, characterized in that: The security verification module comprises: Data encryption unit, used to encrypt data transmitted in the network using the national secret SM4 algorithm to ensure the confidentiality and integrity of data during transmission; The identity verification unit is used to verify the identity of the relay node through the blockchain smart contract, ensure the credibility of the relay node, and prevent malicious nodes from interfering with the system; The secure communication unit is used to ensure the security of the communication link during data transmission and use digital certificates to authenticate both communicating parties to prevent data from being tampered with during transmission.

8. The vehicle self-organizing network system based on deep reinforcement learning according to claim 7, characterized in that: The blockchain smart contract includes: The contract creation unit is used to generate smart contracts and sign the contract content through public key encryption technology to ensure the legality and non-tamperability of the contract; The contract execution unit is used to execute the verification logic in the smart contract and determine the legitimacy of the relay node based on the identity information of the relay node and the preset verification rules; The contract recording unit is used to record the execution results of each smart contract on the blockchain, including the verification records and verification results of the relay nodes, to ensure that all operations are traceable and cannot be tampered with.

9. The vehicle self-organizing network method of the vehicle self-organizing network system based on deep reinforcement learning according to any one of claims 1 to 8, characterized in that: The following steps are involved: Step 1: Collect vehicle density, signal strength RSSI and network load data in real time; Step 2: Select relay nodes through deep reinforcement learning algorithm; Step 3: Adjust the multi-hop path according to the network load and optimize the upper limit of the number of hops of the path; Step 4: When the signal strength is lower than the preset threshold, the relay signal transmission module selects a path and builds a multi-hop relay link to borrow the network signal of other vehicles through wireless transmission; Step 5: Switch between the V2X communication protocol and the Wi-Fi 6 communication protocol according to the network environment and signal strength. The V2X communication protocol operates at a frequency band of 5.9 GHz, the Wi-Fi 6 communication protocol operates at a frequency band of 5 GHz or 6 GHz, and the frequency band interval is greater than or equal to 200 MHz. Step 6: Fusion of data from UWB technology and IMU sensors generates relative positioning information between vehicles, and performs positioning coordination based on this information; Step 7: Encrypt the data transmitted on the network, verify the identity of the relay node, and use the blockchain smart contract mechanism to ensure the validity and immutability of identity authentication; Step 8: Coordinate data transmission, path selection, and communication protocols to ensure the normal operation of the vehicle self-organizing network system based on deep reinforcement learning.

10. The vehicle self-organizing network method based on deep reinforcement learning according to claim 9, characterized in that: Further including: Based on the real-time relative position data between vehicles, the priority of candidate relay nodes is calculated using weighted factors, and the path selection strategy is adjusted; Based on traffic flow monitoring data, combined with the time series prediction model, the path hop allocation is optimized, and the maximum hop limit is dynamically adjusted to adapt to network load conditions with different traffic densities.

Images