A V2X-based blind spot assisted driving method and system for intersections

Abstract

The present invention discloses a V2X-based blind spot assisted driving method and system at an intersection, which relates to the field of intelligent assisted driving technology, including collecting multimodal data through vehicle-mounted sensors and preprocessing, extracting features from the preprocessed multimodal data to form a modal feature vector; based on the multimodal feature vector, using a deep learning model for comprehensive perception to identify the position of obstacles in the blind spot of the intersection; real-time monitoring of the position changes of obstacles in the blind spot of the intersection, predicting the future movement trajectory of the obstacles; assessing the risk level of vehicles passing through the intersection based on the future movement trajectory of the obstacles; receiving real-time traffic information through V2X communication, combining the real-time traffic information with the risk level, and generating driving decisions. The present invention finally generates safer and more efficient driving decisions by combining real-time monitoring of obstacle position changes, prediction of movement trajectory, and risk assessment with V2X communication.

Description

Technical Field

[0001] The present invention relates to the field of intelligent assisted driving technology, and in particular to a V2X-based intersection blind spot assisted driving method and system. Background Art

[0002] With the rapid development of intelligent transportation systems, assisted driving technology based on vehicle-to-everything (V2X) communication has gradually become a research hotspot. V2X technology enables information sharing and collaborative decision-making between vehicles through communication between vehicles (V2V), vehicles and infrastructure (V2I), vehicles and pedestrians (V2P), and vehicles and networks (V2N). This technology has shown great potential in improving road safety, reducing traffic accidents, and improving traffic efficiency. At present, V2X technology has been widely used in many scenarios such as traffic signal optimization, emergency vehicle priority, and road condition warning.

[0003] However, existing V2X-based assisted driving systems still face challenges when dealing with complex intersections, especially those with blind spots. First, traditional systems mainly rely on a single sensor (such as a camera or radar) for environmental perception, which can easily lead to inaccurate perception or omission of key information in complex and changing intersection environments. For example, the performance of cameras degrades at night or in bad weather conditions, and radars have difficulty distinguishing between stationary and slow-moving obstacles. Second, existing V2X systems have deficiencies in data fusion and real-time communication, resulting in delayed and incomplete information transmission, affecting the prediction accuracy and real-time performance of obstacle motion trajectories. These problems not only affect the reliability and safety of the system, but also limit its widespread application in actual traffic scenarios. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a V2X-based intersection blind spot assisted driving method and system to solve the problems of inaccurate single sensor perception and insufficient V2X real-time communication in the prior art.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] On the first aspect, an embodiment of the present invention provides a V2X-based intersection blind spot assisted driving method, which includes: through the combination of multimodal data fusion and deep learning model, the present invention realizes more comprehensive and accurate environmental perception, overcomes the limitations of a single sensor, improves the accuracy and reliability of environmental perception, and can work stably under various weather and lighting conditions, reducing the probability of missed detection and false detection; at the same time, through real-time monitoring of obstacle position changes, prediction of motion trajectory and risk assessment combined with V2X communication, ultimately generating safer and more efficient driving decisions.

[0008] As a preferred solution of the V2X-based blind spot assisted driving method at intersections of the present invention, wherein: the multimodal data is collected and preprocessed by the vehicle-mounted sensor, and the specific steps are as follows:

[0009] Collect image data, radar data, and lidar point cloud data through on-board sensors;

[0010] Gaussian filter is used to denoise the image data;

[0011] Coordinate transformation and Kalman filtering are used to correct radar data;

[0012] The RANSAC algorithm is used to filter the lidar point cloud data.

[0013] As a preferred solution of the V2X-based blind spot assisted driving method at intersections of the present invention, wherein: the feature extraction of the pre-processed multimodal data to form a modal feature vector is performed, and the specific steps are as follows:

[0014] Use CNN to extract features from image data;

[0015] Use RNN to extract time series features of radar data and lidar point cloud data;

[0016] Through the feature splicing method, the feature vectors of different sensors are spliced ​​to form a multimodal feature vector F.

[0017] As a preferred solution of the V2X-based blind spot assisted driving method for intersections of the present invention, wherein: based on the multimodal feature vector, a deep learning model is used for comprehensive perception to identify the position of obstacles in the blind spot of the field of vision, and the specific steps are as follows:

[0018] Use historical data to train the deep learning model, perform comprehensive perception through convolutional layers and pooling layers, generate high-level feature representations, and concatenate feature vectors of different modalities to form multimodal feature vectors. Input the multimodal feature vectors into the first fully connected layer to generate intermediate feature representations, perform nonlinear transformations through the ReLU activation function, and input the feature representations after nonlinear transformation into the second fully connected layer to generate the coordinates of the obstacle positions in the blind spot.

[0019] Update deep learning model parameters through back-propagation algorithm and iteratively optimize deep learning model;

[0020] The multimodal feature vector Input the trained deep learning model for comprehensive perception and identify the location coordinates of obstacles in the blind spot of the intersection. The expression is:

[0021]

[0022] in, represents the position coordinates of the ith obstacle in the blind spot of the intersection, is the multimodal feature vector, represents the first fully connected layer of the deep learning model, Represents the second fully connected layer of the deep learning model.

[0023] As a preferred solution of the V2X-based blind spot assisted driving method at an intersection of the present invention, the specific steps of real-time monitoring of the position change of obstacles in the blind spot of the intersection and predicting the future movement trajectory of the obstacles are as follows:

[0024] The vehicle-mounted sensors monitor the position changes of obstacles in the blind spots of intersections in real time. The Kalman filter is used to predict the future movement trajectory of the obstacle at the current moment based on the obstacle state at the previous moment. Then, the position and speed of the obstacle are updated based on the new detection data of the vehicle-mounted sensors to predict the future movement trajectory of the obstacle. The expression is:

[0025]

[0026] in, It is The obstacle is in The movement trajectory of time, is the i-th obstacle at the The position and speed at the moment, Indicates The obstacle is in The moment The particle positions and velocities, is the weight of the jth particle, M is the number of particles, and k represents the time step;

[0027] The future motion trajectory of the obstacle is formed by the combination of the moving speed and position coordinates of the obstacle at the future moment.

[0028] As a preferred solution of the V2X-based blind spot assisted driving method at an intersection of the present invention, wherein: the risk level of a vehicle passing through an intersection is evaluated according to the future motion trajectory of an obstacle, and the specific steps are as follows:

[0029] According to the future motion trajectory of the obstacle, combined with the minimum distance between the vehicle and each obstacle and the speed difference between the vehicle and each obstacle, the collision risk score between each obstacle and the vehicle is calculated. The expression is:

[0030]

[0031] in, represents the position of the vehicle at time t, represents the position coordinates of the ith obstacle at time t, Indicates the time range during which there is a risk of collision between the vehicle and the obstacle. represents the vehicle speed, represents the speed of the ith obstacle, A weighting factor representing the speed difference, represents the collision risk score between the ith obstacle and the vehicle, and t represents the time interval The specific time point in

[0032] Based on historical data, a low risk threshold R is defined. l and high risk threshold R h , based on the collision risk score , classify obstacles into different risk levels;

[0033] when <R l When , it is considered that the current obstacle poses less threat to the vehicle and is at a lower risk;

[0034] When R h ≤ <R l When , it is considered that the current obstacle poses a medium risk to the vehicle;

[0035] when ≥R h When , it is considered that the current obstacle poses a greater threat to the vehicle and is at a higher risk level.

[0036] As a preferred solution of the V2X-based blind spot assisted driving method at intersections of the present invention, wherein: the real-time traffic information is received through V2X communication, and the driving decision is generated by combining the real-time traffic information with the risk level. The specific steps are as follows:

[0037] Receive real-time traffic information from other vehicles, roadside units and traffic management centers through V2X communication, combine real-time traffic information with risk assessment results, and generate driving decisions;

[0038] In low-risk situations, if the traffic light ahead is about to turn red, the vehicle should slow down and prepare to stop. If the light is green, the vehicle should continue to drive as planned. If the road is slippery, the vehicle should reduce speed appropriately. If pedestrians are approaching, the vehicle should slow down and maintain a safe distance.

[0039] In medium-risk situations, if the traffic light ahead is about to turn red, the vehicle should slow down and prepare to stop; if the light is green, the vehicle should slow down and maintain a safe distance; if the road is slippery, the vehicle should reduce speed appropriately and maintain a safe distance; if pedestrians are approaching, the vehicle should slow down and maintain a safe distance and be prepared to avoid them at any time;

[0040] In high-risk situations, regardless of the current status of the traffic light, the vehicle should slow down immediately and prepare to stop. If the road is slippery, the vehicle should slow down immediately and maintain a safe distance while looking for the best avoidance path and timing. If pedestrians approach, the vehicle should take avoidance measures immediately.

[0041] In the second aspect, the present invention provides a V2X-based intersection blind spot assisted driving system, including a data acquisition and preprocessing module, an obstacle position recognition module, an obstacle trajectory prediction module, a risk assessment module and a decision generation module; the data acquisition and preprocessing module is used to collect and preprocess multimodal data through vehicle-mounted sensors, and extract features from the preprocessed multimodal data to form a modal feature vector; the obstacle position recognition module is used to perform comprehensive perception based on the multimodal feature vector and use a deep learning model to identify the position of obstacles in the blind spot of the intersection; the obstacle trajectory prediction module is used to monitor the changes in the position of obstacles in the blind spot of the intersection in real time and predict the future movement trajectory of the obstacle; the risk assessment module is used to evaluate the risk level of the vehicle passing through the intersection according to the future movement trajectory of the obstacle; the decision generation module is used to receive real-time traffic information through V2X communication, and generate driving decisions based on the real-time traffic information and the risk level.

[0042] In a third aspect, an embodiment of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the V2X-based intersection blind spot assisted driving method as described in the first aspect of the present invention is implemented.

[0043] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the V2X-based intersection blind spot assisted driving method as described in the first aspect of the present invention is implemented.

[0044] The beneficial effects of the present invention are as follows: by combining multimodal data fusion with a deep learning model, the present invention achieves more comprehensive and accurate environmental perception, overcomes the limitations of a single sensor, improves the accuracy and reliability of environmental perception, and can work stably under various weather and lighting conditions, reducing the probability of missed detection and false detection; at the same time, by combining real-time monitoring of obstacle position changes, prediction of motion trajectories, and assessment of risks with V2X communication, ultimately safer and more efficient driving decisions are generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0046] Figure 1 This is a flow chart of the V2X-based blind spot intersection assisted driving method in Example 1.

[0047] Figure 2 This is a flow chart of the V2X-based intersection blind spot assisted driving system in Example 1. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0051] Example 1, reference Figure 1 and Figure 2, which is the first embodiment of the present invention, and provides a V2X-based intersection blind spot assisted driving method, comprising the following steps:

[0052] S1. Collect multimodal data through vehicle-mounted sensors and perform preprocessing, extract features from the preprocessed multimodal data, and form a modal feature vector.

[0053] Going a step further, image data, radar data, and lidar point cloud data are collected through on-board sensors (such as cameras, radars, and lidars);

[0054] Gaussian filter is used to denoise the image data;

[0055] Coordinate transformation and Kalman filtering are used to correct radar data. The specific process is as follows: the obstacle position detected by the radar is converted from the radar coordinate system to the vehicle coordinate system through coordinate transformation, and then the Kalman filter is used to combine historical data and new detection data to accurately estimate the obstacle's position and speed, thereby improving the accuracy and reliability of radar data.

[0056] The RANSAC algorithm is used to filter the lidar point cloud data; the specific process is: the RANSAC algorithm is used to randomly select point cloud samples to fit the model, remove outliers, and retain inliers, thereby effectively removing noise from the lidar point cloud data and improving the cleanliness and reliability of the data.

[0057] Use CNN to extract image data features; the specific process is: process the image data collected by the camera through the convolutional neural network (CNN), extract local features and high-level features in the image layer by layer, and finally generate a feature vector of the image data. For example, the convolution layer and pooling layer of CNN can capture information such as edges, textures, and shapes in the image. After multiple layers of processing, a feature vector that can represent the image content is generated.

[0058] Use RNN to extract the time series features of radar data and lidar point cloud data; the specific process is: use recurrent neural network (RNN) to process the time series data collected by radar and lidar, capture the time dependency and dynamic changes in the data, and generate a time series feature vector. For example, RNN can remember the radar and lidar point cloud data of the previous moment, combine the data of the current moment, extract the target motion features in the continuous time, and generate a feature vector containing time information.

[0059] Through the feature splicing method, the feature vectors of different sensors are spliced ​​to form a multimodal feature vector F, which is expressed as:

[0060]

[0061] in, is the multimodal feature vector, is the multimodal feature vector, is the radar feature vector, is the laser radar feature vector.

[0062] It should also be noted that multimodal data is collected and preprocessed through on-board sensors, and Gaussian filters, coordinate transformations, Kalman filters, RANSAC algorithms and other methods are used to improve data accuracy and reliability. CNN and RNN are then used to extract image data and time series features of radar and lidar, respectively. Finally, a multimodal feature vector is formed through feature splicing, which achieves comprehensiveness of data and richness of features, and significantly improves the perception accuracy and robustness of the system.

[0063] S2. Based on multimodal feature vectors, a deep learning model is used for comprehensive perception to identify the location of obstacles in the blind spots of intersections.

[0064] Furthermore, the deep learning model is trained using historical data, and comprehensive perception is performed through convolutional layers and pooling layers to generate high-level feature representations. Feature vectors of different modalities are concatenated to form multimodal feature vectors, which are input into the first fully connected layer to generate intermediate feature representations. Nonlinear transformation is performed through the ReLU activation function, and the feature representation after nonlinear transformation is input into the second fully connected layer to generate the coordinates of the obstacle position in the blind spot.

[0065] The deep learning model parameters are updated through the back-propagation algorithm, and the deep learning model is iteratively optimized until the expected goal of the deep learning model is reached; the specific process is as follows: First, the historical data is forward-propagated to the deep learning model to calculate the loss value between the model's predicted output and the true label. Then, the loss value is back-propagated from the output layer to the input layer through the back-propagation algorithm, and the gradient of each layer is calculated. According to the calculated gradient, an optimization algorithm (such as gradient descent) is used to update the model's parameters and gradually reduce the loss value. This process is iterated repeatedly until the model's predicted output reaches the expected goal, that is, the loss value converges to a lower level, and the performance of the deep learning model reaches the optimal level.

[0066] The multimodal feature vector Input the trained deep learning model for comprehensive perception and identify the location coordinates of obstacles in the blind spot of the intersection. The expression is:

[0067]

[0068] in, represents the position coordinates of the ith obstacle in the blind spot of the intersection, is the multimodal feature vector, represents the first fully connected layer of the deep learning model, Represents the second fully connected layer of the deep learning model;

[0069] It should be noted that the calculation process of this expression is: First, the multimodal feature vector F is input into the first fully connected layer , generate intermediate feature representations; then, perform nonlinear transformation on the intermediate feature representations through the ReLU activation function to increase the expressive power of the model; finally, input the feature representations after nonlinear transformation into the second fully connected layer , generate the obstacle position coordinates in the blind spot ; This process improves the accuracy and robustness of obstacle location recognition through multi-level feature extraction and nonlinear transformation.

[0070] S3. Monitor the position changes of obstacles in the blind spots of intersections in real time and predict the future movement trajectory of obstacles.

[0071] Furthermore, the vehicle-mounted sensors monitor the position changes of obstacles in the blind spots of the intersection in real time, and use the Kalman filter to predict the future movement trajectory of the obstacle at the current moment based on the obstacle state at the previous moment. Then, based on the new detection data of the vehicle-mounted sensors (video data such as radar data and lidar point cloud data), the position and speed of the obstacle are updated to predict the future movement trajectory of the obstacle. The expression is:

[0072]

[0073] in, It is The obstacle is in The movement trajectory of time, is the i-th obstacle at the The position and speed at the moment, Indicates The obstacle is in The moment The particle positions and velocities, is the weight of the jth particle, M is the number of particles, and k represents the time step;

[0074] The future motion trajectory of the obstacle is formed by the combination of the moving speed and position coordinates of the obstacle at the future moment.

[0075] It should be noted that the specific process of calculating the future motion trajectory of the obstacle is: according to the obstacle state at the previous moment, combined with the positions of multiple particles at the current moment and their corresponding weights, calculate the weighted average new position; then, add the position at the previous moment and the weighted average new position to obtain the position of the ith obstacle at the ith moment. The process of estimating the motion trajectory of the obstacle at each moment effectively estimates and updates the motion trajectory of the obstacle through the particle filtering algorithm, which improves the accuracy and real-time performance of the prediction.

[0076] It should also be noted that in the particle filter algorithm, each particle j represents a possible obstacle state. By maintaining and updating these particles, the future trajectory of the obstacle can be estimated more accurately. For example, the first particle may indicate that the obstacle is 10 meters ahead and traveling at a speed of 20 km / h, while the second particle may indicate that the obstacle is 12 meters ahead and traveling at a speed of 18 km / h.

[0077] S4. Evaluate the risk level of vehicles passing through the intersection based on the future movement trajectory of the obstacle.

[0078] Furthermore, according to the future movement trajectory of the obstacle, combined with the minimum distance between the vehicle and each obstacle and the speed difference between the vehicle and each obstacle, the collision risk score between each obstacle and the vehicle is calculated, and the expression is:

[0079]

[0080] in, represents the position of the vehicle at time t, represents the position coordinates of the ith obstacle at time t, Indicates the time range in which there is a risk of collision between the vehicle and the obstacle in the future. represents the vehicle speed, represents the speed of the ith obstacle, A weighting factor representing the speed difference, represents the collision risk score between the ith obstacle and the vehicle, and t represents the time interval The specific time point in

[0081] It should be noted that the calculation process of the obstacle-vehicle collision risk score is as follows: first, the minimum distance between the vehicle and the obstacle within the time interval is calculated and its reciprocal is taken to reflect the characteristic that the smaller the distance, the higher the risk; then, the difference between the vehicle speed and the obstacle speed is calculated and multiplied by the weight factor to reflect the impact of the speed difference on the risk; finally, these two parts are added together to obtain the obstacle-vehicle collision risk score. This process comprehensively evaluates the risk level of vehicles passing through intersections by comprehensively considering the distance and speed differences.

[0082] Based on historical data, a low risk threshold R is defined. l and high risk threshold R h , based on the collision risk score , classify obstacles into different risk levels;

[0083] when <R l When , it is considered that the current obstacle poses less threat to the vehicle and is at a lower risk;

[0084] When R h ≤ <R l When , it is considered that the current obstacle poses a medium risk to the vehicle;

[0085] when ≥R h When , it is considered that the current obstacle poses a greater threat to the vehicle and is at a higher risk level.

[0086] It should also be noted that by calculating the collision risk score between obstacles and vehicles, the minimum distance and speed difference between vehicles and obstacles can be comprehensively considered to comprehensively assess the risk level of vehicles passing through intersections. This process not only improves the accuracy and real-time performance of risk assessment, but also divides obstacles into different risk levels by defining low-risk thresholds and high-risk thresholds, providing a reliable basis for generating safer and more efficient driving decisions.

[0087] S5. Receive real-time traffic information through V2X communication, combine the real-time traffic information with the risk level, and generate driving decisions.

[0088] Furthermore, V2X communication is used to receive real-time traffic information from other vehicles, roadside units and traffic management centers, and real-time traffic information is combined with risk assessment results to generate driving decisions;

[0089] In low-risk situations, if the traffic light ahead is about to turn red, the vehicle should slow down and prepare to stop. If the light is green, the vehicle should continue to drive as planned. If the road is slippery, the vehicle should reduce speed appropriately. If pedestrians are approaching, the vehicle should slow down and maintain a safe distance.

[0090] In medium-risk situations, if the traffic light ahead is about to turn red, the vehicle should slow down and prepare to stop; if the light is green, the vehicle should slow down and maintain a safe distance; if the road is slippery, the vehicle should reduce speed appropriately and maintain a safe distance; if pedestrians are approaching, the vehicle should slow down and maintain a safe distance and be prepared to avoid them at any time;

[0091] In high-risk situations, regardless of the current status of the traffic light, the vehicle should slow down immediately and prepare to stop. If the road is slippery, the vehicle should slow down immediately and maintain a safe distance while looking for the best avoidance path and timing. If pedestrians approach, the vehicle should take avoidance measures immediately.

[0092] Real-time traffic information includes the status of traffic lights ahead, slippery road conditions, and pedestrian dynamics;

[0093] It should also be noted that by receiving real-time traffic information from other vehicles, roadside units and traffic management centers through V2X communication and combining this information with risk assessment results, the present invention can more accurately judge the risk level of the current driving environment, thereby generating safer and more efficient driving decisions, effectively reducing the probability of traffic accidents and improving the safety of the overall traffic system.

[0094] The present embodiment also provides a blind spot assisted driving system for intersections based on V2X, including: a data acquisition and preprocessing module, an obstacle position recognition module, an obstacle trajectory prediction module, a risk assessment module and a decision generation module; the data acquisition and preprocessing module is used to collect and preprocess multimodal data through vehicle-mounted sensors, and extract features from the preprocessed multimodal data to form a modal feature vector; the obstacle position recognition module is used to perform comprehensive perception based on the multimodal feature vector and adopt a deep learning model to identify the position of obstacles in the blind spot of the intersection; the obstacle trajectory prediction module is used to monitor the position changes of obstacles in the blind spot of the intersection in real time and predict the future movement trajectory of the obstacle; the risk assessment module is used to evaluate the risk level of the vehicle passing through the intersection according to the future movement trajectory of the obstacle; the decision generation module is used to receive real-time traffic information through V2X communication, and generate driving decisions based on the real-time traffic information and the risk level.

[0095] This embodiment also provides a computer device, which is suitable for the case of a V2X-based intersection blind spot assisted driving method, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to implement the V2X-based intersection blind spot assisted driving method proposed in the above embodiment.

[0096] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0097] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for implementing blind spot assisted driving at intersections based on V2X as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0098] In summary, the present invention achieves more comprehensive and accurate environmental perception through the combination of multimodal data fusion and deep learning model, overcomes the limitations of a single sensor, improves the accuracy and reliability of environmental perception, can work stably under various weather and lighting conditions, and reduces the probability of missed detection and false detection; at the same time, through real-time monitoring of obstacle position changes, prediction of motion trajectory and risk assessment combined with V2X communication, ultimately safer and more efficient driving decisions are generated.

[0099] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A V2X-based blind spot assisted driving method at an intersection, characterized in that: include, Collect multimodal data through vehicle-mounted sensors and perform preprocessing, extract features from the preprocessed multimodal data to form a modal feature vector; Based on multimodal feature vectors, a deep learning model is used for comprehensive perception to identify the location of obstacles in the blind spots of intersections; Monitor the position changes of obstacles in the blind spots of intersections in real time and predict the future movement trajectory of obstacles; Assess the risk level of vehicles passing through the intersection based on the future movement trajectory of obstacles; Receive real-time traffic information through V2X communication, combine it with risk level, and generate driving decisions; The method uses a deep learning model based on multimodal feature vectors for comprehensive perception and identifies the location of obstacles in the blind spot. The specific steps are as follows: Use historical data to train the deep learning model, perform comprehensive perception through convolutional layers and pooling layers, generate high-level feature representations, and concatenate feature vectors of different modalities to form multimodal feature vectors. Input the multimodal feature vectors into the first fully connected layer to generate intermediate feature representations, perform nonlinear transformations through the ReLU activation function, and input the feature representations after nonlinear transformation into the second fully connected layer to generate the coordinates of the obstacle positions in the blind spot. Update deep learning model parameters through back-propagation algorithm and iteratively optimize deep learning model; The multimodal feature vector Input the trained deep learning model for comprehensive perception and identify the location coordinates of obstacles in the blind spot of the intersection. The expression is: in, represents the position coordinates of the ith obstacle in the blind spot of the intersection, is the multimodal feature vector, represents the first fully connected layer of the deep learning model, Represents the second fully connected layer of the deep learning model; The specific steps of real-time monitoring of the position change of obstacles in the blind spot of the intersection and predicting the future movement trajectory of the obstacles are as follows: The vehicle-mounted sensors monitor the position changes of obstacles in the blind spots of intersections in real time. The Kalman filter is used to predict the future movement trajectory of the obstacle at the current moment based on the obstacle state at the previous moment. Then, the position and speed of the obstacle are updated based on the new detection data of the vehicle-mounted sensors to predict the future movement trajectory of the obstacle. The expression is: in, It is The obstacle is in The movement trajectory of time, is the i-th obstacle at the The position and speed at the moment, Indicates The obstacle is in The moment The particle positions and velocities, is the weight of the jth particle, M is the number of particles, and k represents the time step; The future motion trajectory of the obstacle is formed by the combination of the moving speed and position coordinates of the obstacle at the future moment.

2. The V2X-based blind spot intersection assisted driving method according to claim 1, characterized in that: The multimodal data is collected and preprocessed by the vehicle-mounted sensor. The specific steps are as follows: Collect image data, radar data, and lidar point cloud data through on-board sensors; Gaussian filter is used to denoise the image data; Coordinate transformation and Kalman filtering are used to correct radar data; The RANSAC algorithm is used to filter the lidar point cloud data.

3. The V2X-based blind spot intersection assisted driving method as claimed in claim 2, characterized in that: The feature extraction of the preprocessed multimodal data to form a modal feature vector is performed in the following specific steps: Use CNN to extract features from image data; Use RNN to extract time series features of radar data and lidar point cloud data; Through the feature splicing method, the feature vectors of different sensors are spliced ​​to form a multimodal feature vector.

4. The V2X-based blind spot intersection assisted driving method as claimed in claim 3, characterized in that: The specific steps of evaluating the risk level of a vehicle passing through an intersection according to the future movement trajectory of an obstacle are as follows: According to the future motion trajectory of the obstacle, combined with the minimum distance between the vehicle and each obstacle and the speed difference between the vehicle and each obstacle, the collision risk score between each obstacle and the vehicle is calculated. The expression is: in, represents the position of the vehicle at time t, represents the position coordinates of the ith obstacle at time t, Indicates the time range during which there is a risk of collision between the vehicle and the obstacle. represents the vehicle speed, represents the speed of the ith obstacle, A weighting factor representing the speed difference, represents the collision risk score between the ith obstacle and the vehicle, and t represents the time interval The specific time point in Based on historical data, a low risk threshold R is defined. l and high risk threshold R h , based on the collision risk score , classify obstacles into different risk levels; when <R l When , it is considered that the current obstacle poses less threat to the vehicle and is at a lower risk; When R h ≤ <R l When , it is considered that the current obstacle poses a medium risk to the vehicle; when ≥R h When , it is considered that the current obstacle poses a greater threat to the vehicle and is at a higher risk level.

5. The V2X-based blind spot intersection assisted driving method as claimed in claim 4, characterized in that: The specific steps of receiving real-time traffic information through V2X communication and combining the real-time traffic information with the risk level to generate driving decisions are as follows: Receive real-time traffic information from other vehicles, roadside units and traffic management centers through V2X communication, combine real-time traffic information with risk assessment results, and generate driving decisions; In low-risk situations, if the traffic light ahead is about to turn red, the vehicle should slow down and prepare to stop. If the light is green, the vehicle should continue to drive as planned. If the road is slippery, the vehicle should reduce speed appropriately. If pedestrians are approaching, the vehicle should slow down and maintain a safe distance. In medium-risk situations, if the traffic light ahead is about to turn red, the vehicle should slow down and prepare to stop; if the light is green, the vehicle should slow down and maintain a safe distance; if the road is slippery, the vehicle should reduce speed appropriately and maintain a safe distance; if pedestrians are approaching, the vehicle should slow down and maintain a safe distance and be prepared to avoid them at any time; In high-risk situations, regardless of the current status of the traffic light, the vehicle should slow down immediately and prepare to stop. If the road is slippery, the vehicle should slow down immediately and maintain a safe distance while looking for the best avoidance path and timing. If pedestrians approach, the vehicle should take avoidance measures immediately.

6. A V2X-based intersection blind spot assisted driving system, based on the V2X-based intersection blind spot assisted driving method according to any one of claims 1 to 5, characterized in that: Including data acquisition and preprocessing module, obstacle location recognition module, obstacle trajectory prediction module, risk assessment module and decision generation module; The data acquisition and preprocessing module is used to collect multimodal data through vehicle-mounted sensors and perform preprocessing, and extract features from the preprocessed multimodal data to form a modal feature vector; The obstacle location recognition module is used to identify the location of obstacles in the blind spot of the intersection based on multi-modal feature vectors and a deep learning model for comprehensive perception; The obstacle trajectory prediction module is used to monitor the position changes of obstacles in the blind spot of the intersection in real time and predict the future movement trajectory of the obstacles; The risk assessment module is used to assess the risk level of vehicles passing through the intersection based on the future movement trajectory of obstacles; The decision generation module is used to receive real-time traffic information through V2X communication, combine the real-time traffic information with the risk level, and generate driving decisions.

Images