Communication and road condition information perception fused driver beyond visual range perception method
By integrating wireless communication and driving environment perception technology in complex urban traffic environments, and using sensor collaborative work and semantic segmentation technology, comprehensive monitoring and information sharing of vehicle driving environment is achieved, which solves the problem that traditional driving methods are difficult to deal with emergencies and improves driving safety and perception capabilities.
Patent Information
- Application Number
- CN202411785136.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-06
- Publication Date
- 2025-05-13
AI Technical Summary
In complex urban traffic environments, traditional driving methods are difficult to effectively deal with emergencies, resulting in large blind spots for drivers and increasing the probability of traffic accidents.
By integrating wireless communication technology with driving environment perception technology, using radar, camera, Beidou and other sensors to work together, combining semantic segmentation technology and end-to-end heterogeneous networks, comprehensive and accurate monitoring and information sharing of the vehicle's driving environment can be achieved, breaking through the driver's line of sight limitations, and achieving over-visual perception.
It improves the vehicle's road conditions perception ability of complex urban environments, provides a more intelligent and safe auxiliary decision-making system, and significantly improves driving safety and road traffic efficiency.
Smart Images
Figure CN119992818A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a driver's beyond-visual-range perception method integrating communication and road condition information perception, and belongs to the technical field of wireless communication. Background Art
[0002] In the current traffic management system, drivers rely on their own sight distance, experience and judgment to make driving decisions. However, due to human physiological and cognitive limitations, especially in complex urban traffic environments, traditional driving methods are difficult to effectively respond to various emergencies. Drivers' blind spots, the impact of bad weather on visual conditions, and traffic congestion often cause drivers to have large blind spots when making judgments, increasing the probability of traffic accidents. In order to improve traffic safety issues, wireless communication and driving environment perception technologies are gradually being applied to traffic management and intelligent driving systems. Wireless communication technology, especially 5G and future 6G technology, has the characteristics of high speed, low latency and large connection, providing strong technical support for vehicle-to-vehicle and vehicle-to-road collaboration. Through vehicle networking technology, vehicles can achieve real-time communication with other vehicles, infrastructure, road sensors, etc., helping drivers obtain more extensive environmental perception information, thereby improving the accuracy and safety of driving decisions. At the same time, driving environment perception technologies, such as radar, camera, Beidou and other sensor technologies, can provide real-time and accurate vehicle surrounding environment data, make up for the blind spots of traditional visual perception, and help drivers detect potential dangers in a timely manner and respond. However, a single perception technology has limitations, especially in complex urban traffic environments, where a single sensor is easily affected by factors such as line of sight obstruction and environmental interference.
[0003] Therefore, how to effectively integrate wireless communication technology with driving environment perception technology, overcome the limitations of the driver's blind spots, and improve the comprehensive perception of urban roads has become a key technical problem in the field of intelligent transportation systems and autonomous driving. This fusion technology can not only improve the efficiency of information sharing between vehicles, but also achieve high-precision positioning of vehicles and construction of digital twin maps without prior spatial information, thereby breaking through the driver's line of sight, providing beyond-line-of-sight perception, and significantly improving driving safety and road traffic efficiency. Summary of the invention
[0004] The purpose of the present invention is to address the defects and shortcomings of the above-mentioned prior art and propose a driver's beyond-visual-range perception method that integrates communication and road condition information perception, so as to be used for beyond-visual-range perception that integrates communication and road condition information perception in complex urban environments. This method can realize real-time information sharing and efficient cross-domain transmission of environmental perception data through an end-to-end communication system, so that the vehicle can fully obtain various information of the driving environment and realize beyond-visual-range perception that integrates communication and road condition information perception. It not only improves the vehicle's road condition perception ability in complex urban environments, but also provides the driver with a more intelligent and safe auxiliary decision-making system.
[0005] The technical solution adopted by the present invention to solve the technical problem is: a driver's beyond-visual-range perception method integrating communication and road condition information perception, the method comprising the following steps:
[0006] Step 1: Divide the urban traffic environment into information zones according to transmission requirements and information importance, and perform flexible information transmission management for different types of perceived information.
[0007] Step 2: In urban traffic environments, roadside radars, cameras, BeiDou and other sensors work together to conduct comprehensive and accurate monitoring of the vehicle driving environment.
[0008] Step 3: Use semantic segmentation technology to process the road condition information data monitored by the sensor, extract the useful parts for transmission, and optimize the data transmission efficiency.
[0009] Step 4: Use an end-to-end heterogeneous network that integrates wired and wireless networks to achieve interconnection, cross-domain transmission, and efficient collaboration of processed sensor-monitored road condition information data resources.
[0010] Step 5: Analyze and study the actively captured driving environment information and the passively received base station shared road condition perception information to achieve a comprehensive understanding of the driving environment.
[0011] Step 6: Complete high-precision positioning and digital twin map construction without prior spatial information, realize the driver's beyond-visual-range perception that integrates communication and road condition information perception, and break through the driver's line of sight limitations.
[0012] Furthermore, the present invention divides the information of the urban traffic environment into zones according to the transmission requirements and the importance of information, performs flexible information transmission management for different types of perceived information, and realizes dynamic selective transmission and distribution of different information, which not only improves the data transmission efficiency, but also reduces the load on the system, while ensuring that the driver obtains the most critical and timely driving information.
[0013] Furthermore, the present invention takes into account the complex urban traffic driving environment, and through comprehensive perception of urban traffic scene data of comprehensive point cloud, street view images, and roadside intelligent sensing equipment, provides a multi-dimensional, multi-granular, multi-scale, real-time and accurate perception data source for the driver's beyond-visual-range fusion of communication and road condition information perception.
[0014] Furthermore, the semantic segmentation technology of the present invention can effectively identify important targets in the driving environment, such as pedestrians, vehicles, obstacles, traffic signs, etc., provide the vehicle with more comprehensive environmental perception information, and help the driver make more scientific and safe decisions.
[0015] Furthermore, the semantic segmentation of the present invention adopts a fully convolutional network (FCN), whose basic architecture can be divided into three parts: encoder, decoder and skip connection.
[0016] The encoder part is composed of a traditional convolutional neural network. Through multiple convolutional layers and pooling layers, the network gradually extracts low-level to high-level features of the image. Spatial downsampling, that is, the pooling layer, will gradually reduce the spatial resolution of the image and increase the receptive field, thereby not capturing more contextual information.
[0017] The decoder part uses a deconvolution layer for upsampling, gradually recovering the spatial resolution, and gradually recovering the segmentation map of the original image size from the low-resolution feature map.
[0018] Skip connections help recover image details and improve segmentation accuracy by fusing lower-level features in the encoder with higher-level features in the decoder.
[0019] In FCN, the training goal is to minimize the gap between the predicted result of each pixel and the true label. The cross entropy loss is used as the loss function to calculate the error between the predicted category and the true label of each pixel. For each pixel in the image, the network outputs the probability of each category, and the cross entropy loss function calculates the difference between the predicted category probability and the actual label.
[0020] The cross entropy loss function formula is as follows:
[0021]
[0022] Where N is the number of pixels in the image, C is the total number of categories, and 1(y i =c) is the indicator function, indicating that pixel x i Whether it belongs to category c, Is the network to pixel x i The predicted probability of class c.
[0023] By minimizing the cross entropy loss, FCN can optimize the network parameters, making the classification of each pixel more accurate and completing the semantic segmentation task better.
[0024] Furthermore, the end-to-end heterogeneous network of the present invention includes various transportation networks such as 5G, low-power wide area network, wireless local area network, time-sensitive network, etc., to ensure the transmission efficiency and reliability of data in different network environments.
[0025] Furthermore, the present invention constructs a mutually enhanced active and passive action perception model based on the actively captured driving environment information and the automatically received base station shared road condition perception information, thereby realizing real-time perception of the driving environment with integrated communication and perception.
[0026] Beneficial effects:
[0027] 1. The present invention proposes a driver's beyond-visual-range perception method that integrates communication and road condition information perception. First, the urban traffic environment is divided into information zones according to transmission requirements and information importance. Secondly, radars, cameras, Beidou and other sensors work together to achieve comprehensive and accurate monitoring of the vehicle's driving environment, and semantic segmentation is used to process the road condition information data monitored by the sensors to optimize data transmission. Then, an end-to-end heterogeneous network that integrates wired and wireless is used to achieve interconnection, cross-domain transmission and efficient collaboration of road condition information data resources monitored by sensors, providing a high-quality transmission channel for massive multi-mode data of vehicles and roads in urban traffic environments. Finally, mobile vehicles achieve high-precision positioning and digital twin map construction without prior spatial information through actively captured driving environment information and passively received base station shared road condition perception information, realizing beyond-visual-range perception that integrates communication and road condition information perception.
[0028] 2. The present invention proposes a driver's beyond-visual-range perception method that integrates communication and road condition information perception, which can be applied to complex urban traffic environments. It provides a novel and efficient solution for beyond-visual-range perception that integrates mobile vehicle communication and road condition information perception, and provides advanced technical support for high-quality communication and high-safety operation of vehicle-to-vehicle and vehicle-road collaboration in sustainable urban traffic. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 The present invention is a flow chart of a driver's beyond-visual-range perception method integrating communication and road condition information perception.
[0030] Figure 2 This is a scene diagram of a driver's beyond-visual-range perception method that integrates communication and road condition information perception according to the present invention.
[0031] Figure 3 Schematic diagram of the semantic segmentation technology of the present invention.
[0032] Figure 4 It is a schematic diagram of end-to-end heterogeneous network communication of the present invention.
[0033] Figure 5 This is a schematic diagram of collaborative positioning with the introduction of virtual nodes according to the present invention. DETAILED DESCRIPTION
[0034] The invention will be further described in detail below in conjunction with the accompanying drawings.
[0035] like Figure 1 As shown, the present invention provides a driver's beyond-visual-range perception method integrating communication and road condition information perception, the method comprising the following steps:
[0036] Step 1: Divide the urban traffic environment into information zones according to transmission requirements and information importance. For high-risk areas such as blind spots and intersections, small-scale zones are used. Information in such areas is of high urgency, such as ghost peeking, sudden obstacles, pedestrians crossing and other important safety information, which needs to be fed back to the driver immediately to ensure that the driver can respond in time. For wider areas, it mainly includes general information such as regular road conditions, traffic signs, lane changes, etc. that do not pose immediate danger. This type of information can be transmitted over a larger range with a lower transmission frequency to ensure efficient use of network resources.
[0037] Step 2: If Figure 2 As shown in the figure, in the urban traffic environment, base stations and both sides of the road are equipped with a variety of sensors, including radar, camera, Beidou and other equipment. Various sensors work together to conduct comprehensive and accurate monitoring of the vehicle driving environment. Specifically, radar is responsible for detecting and tracking dynamic information such as the distance and speed of the target object; the camera provides high-definition images for target recognition and environmental understanding; the Beidou satellite positioning system provides high-precision geographic location data. Reasonable planning of the transmission and fusion strategies of perception information such as roads, roadside, sky, vehicles, pedestrians, etc., to reduce the pressure of perception information transmission in the Internet of Vehicles and improve the efficiency of perception fusion.
[0038] Furthermore, assuming that the data fields collected by the data collection node each time are the same, assuming that it includes M fields, and the data collected from the 1st to the nth time are respectively (V1, V2, V3, V4, ..., V n ), for any interval variable of field j, its variable values are V 1j , V 2j , V 3j , ... V nj , for any field value V ij , normalized V' ij ∈[0,1].
[0039]
[0040] The variable values after normalization of the kth field are V1' k ,V2'k ,V3' k ,...,V n ' k , the comprehensive difference d(i,j) between the data collected at the i-th time and the data collected at the j-th time is calculated as follows:
[0041]
[0042] In the formula, Indicates the difference between the values of the kth field in the i-th and j-th data collection after the values of each field are normalized:
[0043]
[0044] Since the M fields may have different sensitivity to differences in the data collection of the driving system, and considering that the numerical differences of different fields have different influences on the comprehensive difference, it is necessary to set a weight for each field, which is recorded as: w1, w2, w3, ..., w M , so a weight strategy is introduced when calculating the comprehensive difference between multiple acquisitions of data:
[0045]
[0046] The weight reflects the importance of the corresponding indicator to the evaluation.
[0047] Step 3: Based on the data collection of sensor road condition information, Figure 3 As shown in the figure, semantic segmentation technology is used to process the road condition information data monitored by the sensor. Only useful target data identified after semantic segmentation is transmitted, reducing the transmission of irrelevant information. For example, the system will give priority to transmitting key perception data such as target vehicles, pedestrians, and traffic signals, ignoring background information and unimportant environmental data (such as empty roads or irrelevant objects at a distance), reducing the transmission of irrelevant information, and optimizing the data transmission rate.
[0048] Step 4: Figure 4 As shown in the figure, by precisely characterizing the statistical characteristics of delay under limited code length conditions and proactively configuring wireless transmission resources, the reliability and timeliness of transmission are guaranteed while ensuring the determinism of delay. In addition, the cross-domain interconnection of end-to-end heterogeneous networks in urban traffic scenarios such as 5G, low-power wide area networks, wireless local area networks, and time-sensitive networks is achieved through timing caching and periodic alignment strategies; through customized protection measures for different types of business flows, a scheduling relaxation strategy is designed under the condition of ensuring the transmission of security business flows, the service quality of non-security business flows is improved, and the high-quality collaborative transmission of mixed security business flows is guaranteed, and the road condition information data resources processed by semantic segmentation technology are transmitted to vehicles.
[0049] Step 5: Based on the actively captured driving environment information and the passively received base station shared road condition perception information, a mutually enhanced active and passive action perception model is constructed to achieve real-time perception of the driving environment with integrated communication and perception.
[0050] Furthermore, if the scene has K static physical nodes and one mobile physical node, Figure 5 As shown, PA is an arbitrary static physical node, let represents the position of the kth static physical node at time t, and the reflection surface side image of the static physical node PA is recorded as VA, The corresponding virtual node position is recorded as RP is an arbitrary mobile node in the two-dimensional plane, P rp is the starting position of the mobile node, P rp The corresponding side mirror image of the reflecting surface is denoted as VRP, and the position is denoted as set up is the normal of the lth reflection surface, expressed as:
[0051]
[0052] Among them, ||·|| represents the L2 norm, VRP contains the normal direction of the reflecting surface and the distance between the reflecting surface and RP, the normals of PA and VA are parallel to the normals of RP and VRP. Given VRP and PA, the position of VA can be calculated:
[0053]
[0054] Given PA and VA, the VRP position can be calculated:
[0055]
[0056] Given VRP and VA, the location of PA can be calculated:
[0057]
[0058] The introduction of VRP allows VA to use less data to characterize the radio environment. Multiple VAs on the same reflection surface share one VRP, which can realize multi-path data fusion of different PAs. By utilizing the mobile characteristics of fixed base stations on the roadside and traffic participants, and based on active and passive collaborative perception of the traffic environment, the collaborative perception of traffic participants and multiple nodes of urban traffic base stations in the urban traffic environment can be realized.
[0059] Step 6: Complete high-precision positioning and digital twin map construction without prior spatial information, realize the driver's beyond-visual-range perception that integrates communication and road condition information perception, and break through the driver's line of sight limitations.
[0060] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with the technical field within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.
Claims
1. A driver's beyond-visual-range perception method integrating communication and road condition information perception, characterized in that: The steps include: Step 1: Divide the urban traffic environment into information zones according to transmission requirements and information importance, and perform flexible information transmission management for different types of perceived information; Step 2: In urban traffic environments, roadside radars, cameras, and BeiDou sensors work together to conduct comprehensive and accurate monitoring of the vehicle driving environment; Step 3: Use semantic segmentation technology to process the road condition information data monitored by the sensor, extract the useful parts for transmission, and optimize the data transmission efficiency; Step 4: Use the end-to-end heterogeneous network that integrates wired and wireless networks to achieve interconnection, cross-domain transmission, and efficient collaboration of processed sensor-monitored road condition information data resources; Step 5: Analyze and study the actively captured driving environment information and the passively received base station shared road condition perception information to achieve a comprehensive understanding of the driving environment; Step 6: Complete high-precision positioning and digital twin map construction without prior spatial information, realize the driver's beyond-visual-range perception that integrates communication and road condition information perception, and break through the driver's line of sight limitations.
2. The driver's beyond-visual-range perception method of the communication and road condition information perception fusion according to claim 1 is characterized in that: In the step 1, the regions are divided according to the transmission requirements and the importance of information.
3. The driver's beyond-visual-range perception method of the communication and road condition information perception fusion according to claim 1 is characterized in that: In step 2, the complex urban traffic driving environment is taken into consideration, and through comprehensive perception of urban traffic scene data such as integrated point cloud, street view images, and roadside intelligent sensor equipment, a multi-dimensional, multi-granular, multi-scale, real-time and accurate perception data source is provided for the driver's beyond-visual-range perception that integrates communication and road condition information perception.
4. The driver's beyond-visual-range perception method of the communication and road condition information perception fusion according to claim 3 is characterized in that: Reasonably plan the transmission and fusion strategies of road, roadside, sky, vehicle and pedestrian perception information to reduce the pressure of vehicle network perception information transmission and improve perception fusion efficiency; Specifically, assuming that the data fields collected by the data collection node each time are the same, assuming that it includes M fields, and the data collected from the 1st to the nth time are (V1, V2, V3, V4, ..., V n ), for any interval variable of field j, its variable values are V 1j , V 2j , V 3j , ... V nj , for any field value V ij , normalized V ij ∈[0, 1]; The variable values after normalization of the kth field are V′ 1k , V′ 2k , V′ 3k , ..., V′ nk , the comprehensive difference d(i, j) between the data collected at the i-th time and the data collected at the j-th time is calculated as follows: In the formula, Indicates the difference between the values of the kth field in the i-th and j-th data collection after the values of each field are normalized: Since the M fields may have different sensitivity to differences in the data collection of the driving system, and considering that the numerical differences of different fields have different influences on the comprehensive difference, it is necessary to set a weight for each field, which is recorded as: w1, w2, w3, ..., w M , so a weight strategy is introduced when calculating the comprehensive difference between multiple acquisitions of data: The weight reflects the importance of the corresponding indicator to the evaluation.
5. The driver's beyond-visual-range perception method of the communication and road condition information perception fusion according to claim 1 is characterized in that: In step 3, the semantic segmentation technology can effectively identify important targets in the driving environment, namely pedestrians, vehicles, obstacles, and traffic signs.
6. The driver's beyond-visual-range perception method of the communication and road condition information perception fusion according to claim 5 is characterized in that: A fully convolutional network (FCN) is adopted, and its basic architecture is divided into three parts: encoder, decoder and skip connection.
7. The driver's beyond-visual-range perception method of the communication and road condition information perception fusion according to claim 1 is characterized in that: In step 4, the end-to-end heterogeneous network includes various transportation networks such as 5G, low-power wide area network, wireless local area network, time-sensitive network, etc., to ensure the transmission efficiency and reliability of data in different network environments.
8. The driver's beyond-visual-range perception method of the communication and road condition information perception fusion according to claim 1 is characterized in that: In step 5 and step 6, a mutually enhanced active and passive action perception model is constructed based on the actively captured driving environment information and the automatically received base station shared road condition perception information to achieve real-time perception of the driving environment with integrated communication and perception.