Vehicle lane changing prompting method and device, electronic equipment and storage medium

Abstract

The invention discloses a vehicle lane changing prompting method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting the current state information, the current surrounding environment data and the current position information of a vehicle, and carrying out the lane changing prediction of the vehicle through a prediction model based on the current state information, the current surrounding environment data and the current position information, obtaining a current collision probability and a lane changing distance percentage of the vehicle, determining a lane changing reminding distance of the current driving scene according to the lane changing distance percentage, and determining target lane changing reminding information according to the current collision probability, and controlling the augmented reality head-up display of the vehicle to carry out lane change prompting at the lane change prompting distance according to the target lane change prompting information. According to the method, the prediction model is utilized to perform real-time reasoning on the multi-modal data acquired by the vehicle in real time, and the lane changing prompt is adaptive to the driving state of the user, so that the intelligence and individuation of the lane changing prompt of the vehicle are realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of intelligent driving, and particularly relates to a vehicle lane changing prompting method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the rapid development of vehicle intelligence, the advanced driving assistance system has become a core means for improving road traffic safety and efficiency. As an important innovation of human-computer interaction, the augmented reality head-up display (AR-HUD) technology accurately fuses virtual navigation instructions, safety warnings and other information with real road scenes, so that drivers can obtain key information without looking down.

[0003] At present, when a vehicle reaches a certain preset distance before a lane changing intersection set by a driver through GPS positioning and a navigation path of the vehicle, the AR-HUD triggers and displays a lane changing prompt. This fixed distance triggering lane changing prompt mechanism lacks comprehensive analysis of real-time driving conditions, and is difficult to adapt to dynamically changing driving scenes and lane changing conditions, which may easily lead to the timing of the lane changing prompt being too early or too late, affecting user experience and driving safety. SUMMARY

[0004] The purpose of the embodiments of the application is to provide a vehicle lane changing prompting method, device, electronic device and storage medium, which can solve the problem that the current fixed distance triggering lane changing prompt mechanism is difficult to adapt to dynamically changing driving scenes and lane changing conditions, which may easily lead to the timing of the lane changing prompt being too early or too late, affecting user experience and driving safety.

[0005] In order to solve the above technical problems, the application is implemented as follows: In a first aspect, the embodiments of the application provide a vehicle lane changing prompting method, which comprises: collecting current state information, current surrounding environment data and current position information of a vehicle; using a prediction model to perform lane changing prediction on the vehicle based on the current state information, the current surrounding environment data and the current position information, to obtain a current collision probability and a lane changing distance percentage of the vehicle; wherein the prediction model is obtained by iterative training according to historical state information, historical surrounding environment data and historical position information; determining a lane changing prompting distance of a current driving scene according to the lane changing distance percentage, and determining target lane changing prompt information according to the current collision probability; controlling the augmented reality head-up display of the vehicle to perform lane changing prompting at the lane changing prompting distance according to the target lane changing prompt information.

[0006] Optionally, the lane change prediction on the vehicle based on the current state information, the current surrounding environment data and the current position information by using the prediction model comprises: feature extraction is performed on the current state information, the current surrounding environment data and the current position information respectively, and a current state feature vector, a current visual feature vector and a current environment feature vector are obtained correspondingly; the current state feature vector, the current visual feature vector and the current environment feature vector are uniformly represented as a shared feature vector; the shared feature vector is input into a lightweight decision head of the prediction model, and the first activation function corresponding to the collision probability and the second activation function corresponding to the lane change distance in the lightweight decision head are used for lane change prediction, so as to obtain the current collision probability and the lane change distance percentage of the vehicle.

[0007] Optionally, the lane change reminder distance of the current driving scene is determined according to the lane change distance percentage, and the target lane change prompt information is determined according to the current collision probability, comprising: the lane change reminder distance of the current driving scene is obtained by calculating the lane change distance percentage and a preset basic distance; the current collision probability is compared with a preset probability threshold to determine a collision risk level; the target lane change prompt information corresponding to the collision risk level is determined according to a preset corresponding relationship between the collision risk level and the lane change prompt information.

[0008] Optionally, after the augmented reality head-up display of the vehicle is controlled to perform lane change prompting according to the target lane change prompt information at the lane change reminder distance, the method further comprises: the current state information, the current surrounding environment data and the current position information collected are stored into a preset database as historical state information, historical surrounding environment data and historical position information; when the data storage amount of the preset database exceeds a preset threshold or according to a preset period, the historical state information, the historical surrounding environment data and the historical position information are used to perform an iterative training on the prediction model.

[0009] Optionally, before the lane change prediction on the vehicle based on the current state information, the current surrounding environment data and the current position information by using the prediction model, the method further comprises: obtaining historical state information, historical surrounding environment data and historical position information; feature extraction is performed on the historical state information, the historical surrounding environment data and the historical position information respectively, and a state feature vector, a visual feature vector and an environment feature vector are obtained correspondingly; fuse the state feature vector, the visual feature vector and the environment feature vector to obtain a fusion feature vector; adopting a vehicle lane line space-time graph constructed in advance to predict trajectory information of the vehicle within a preset time length; inputting the fusion feature vector and the trajectory information into a lightweight decision head for training to obtain a prediction model for outputting a current collision probability and a lane changing distance percentage of the vehicle.

[0010] Optionally, the step of fusing the state feature vector, the visual feature vector and the environment feature vector to obtain a fusion feature vector comprises: mapping the state feature vector, the visual feature vector and the environment feature vector to a unified semantic space, taking the visual feature vector as a query vector and taking the state feature vector and the environment feature vector as a pair of key vector and value vector; determining a dot product similarity of the query vector and the key vector to obtain an attention weight matrix; adopting the attention weight matrix to perform weighted summation on the value vector to obtain a fusion feature vector.

[0011] Optionally, the step of adopting a vehicle lane line space-time graph constructed in advance to predict trajectory information of the vehicle within a preset time length comprises: previously determining a vehicle node and a lane line node according to the historical state information, the historical surrounding environment data and the historical position information; constructing a vehicle lane line space-time graph based on the vehicle node, the lane line node and a pre-determined edge weight; performing convolution processing on the vehicle lane line space-time graph to predict trajectory information of the vehicle within a preset time length.

[0012] In a second aspect, an embodiment of the present application provides a vehicle lane changing prompting device, which comprises: an information acquisition module configured to acquire current state information, current surrounding environment data and current position information of a vehicle; a lane changing prediction module configured to perform lane changing prediction on the vehicle based on the current state information, the current surrounding environment data and the current position information of the vehicle by using a prediction model to obtain a current collision probability and a lane changing distance percentage of the vehicle, wherein the prediction model is obtained by iterative training according to historical state information, historical surrounding environment data and historical position information; a parameter determination module configured to determine a lane changing reminding distance of a current driving scene according to the lane changing distance percentage and determine target lane changing prompting information according to the current collision probability; a control module configured to control an augmented reality head-up display of the vehicle to perform lane changing prompting according to the target lane changing prompting information at the lane changing reminding distance.

[0013] According to another aspect of the present application, an electronic device is also provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the vehicle lane change prompting method as described above.

[0014] According to another aspect of the present application, a readable storage medium is also provided, the readable storage medium storing a computer program, the computer program being executed by a processor to implement the vehicle lane change prompting method as described above.

[0015] The vehicle lane change prompting method provided by the embodiments of the present application collects the current state information, the current surrounding environment data and the current position information of the vehicle, uses a prediction model to predict the lane change of the vehicle based on the current state information, the current surrounding environment data and the current position information, to obtain the current collision probability and the lane change distance percentage of the vehicle; wherein the prediction model is obtained by iterative training according to historical state information, historical surrounding environment data and historical position information; according to the lane change distance percentage, the lane change reminding distance of the current driving scene is determined, and according to the current collision probability, the target lane change prompting information is determined, and the augmented reality head-up display of the vehicle is controlled to perform lane change prompting according to the target lane change prompting information at the lane change reminding distance. The embodiments of the present application collect and fuse the multi-modal data in the driving process of the vehicle, use the prediction model to infer and predict the real-time collected multi-modal data, accurately obtain the collision probability when the vehicle performs lane change and the lane change distance percentage for prompting lane change, determine the effective lane change prompting distance and prompting information according to the collision probability and the lane change distance percentage, realize the self-adaptation of the vehicle lane change prompting and the user driving state, timely and effectively perform lane change prompting, meet the personalized intelligent driving prompting needs of the user in different driving states, improve the intelligentization and personalization of the vehicle lane change prompting, and further significantly improve the driving safety and user experience.

[0016] The above description is only a summary of the technical solutions of the present application. In order to enable the technical means of the present application to be more clearly understood, and to be implemented according to the content of the description, and in order to enable the above and other purposes, characteristics and advantages of the present application to be more apparent and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0017] Various other advantages and benefits will become apparent to those of ordinary skill in the art, upon reading the following detailed description of the preferred embodiments. The accompanying drawings are intended to further aid the full and complete understanding of the preferred embodiments. Throughout the drawings, the same reference symbols are used for the same components. In the drawings: Figure 1 This is a flowchart illustrating the steps of a vehicle lane change notification method provided in an embodiment of this application; Figure 2 This is a flowchart of another vehicle lane change notification method provided in an embodiment of this application; Figure 3 This is a flowchart illustrating the prediction model in a vehicle lane change warning method provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a vehicle lane change warning device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0020] The vehicle lane change prompting method, device, electronic device, and storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0021] Existing AR-HUD lane change warning mechanisms in vehicles suffer from the following problems and shortcomings: Based on a preset fixed distance (e.g., 50 meters), the timing of lane change warnings may be too early or too late. Fixed-distance triggering of lane change warnings cannot adapt to individual needs, may conflict with user intuition or experience, and easily lead to driver neglect or misunderstanding of the warnings, increasing cognitive load and reducing user trust and willingness to use the AR-HUD system. Furthermore, they typically only provide simple collision warnings during lane change execution, lacking prediction and assessment of potential risks before the lane change, and cannot effectively avoid collision risks in complex scenarios. Therefore, to address the problems of existing fixed-distance triggering lane change warning mechanisms failing to adapt to dynamically changing driving scenarios and lane change conditions, and easily leading to lane change warnings being too early or too late, affecting user experience and driving safety, this application achieves adaptive lane change warnings through multimodal data fusion, lane change condition inference, and safety assessment.

[0022] Reference Figure 1 This document illustrates a flowchart of the vehicle lane change warning method provided in an embodiment of this application. The method is applied to a vehicle and is connected to a cloud. The method may include: Step 101: Collect the vehicle's current status information, current surrounding environment data, and current location information.

[0023] This application proposes an adaptive lane change warning method based on multimodal data fusion. By collecting multi-source data, performing predictive model training and real-time lane change inference, and combining driving state and driving environment, the method dynamically adjusts the lane change warning distance and predicts the lane change collision probability. This achieves adaptive lane change warnings to the user's driving state, enabling timely and effective lane change warnings, thus improving driving safety and user experience. Specifically, the vehicle-side deploys a trained and updated predictive model. This model is iteratively trained based on pre-collected historical state information, historical surrounding environment data, and historical location information. Details of the predictive model training can be found in another embodiment of this application and will not be elaborated here.

[0024] It should be noted that the execution subject of this application embodiment can be other data processing systems or platforms such as vehicle terminal, cloud, etc. This embodiment takes the vehicle terminal as an example for explanation. When the vehicle terminal starts the in-vehicle navigation and the augmented reality head-up display function is turned on, the current status information of the vehicle, the current surrounding environment data, and the current location information are collected.

[0025] The vehicle's head-up display (HUD) function aims to allow drivers to obtain key information without looking down at the instrument panel, thereby improving driving safety and experience. In this embodiment, the HUD function adopts AR-HUD, which accurately integrates virtual navigation instructions, safety warnings, and other information with real road scenes. For example, by setting a function switch in the vehicle's infotainment system, drivers can choose whether to enable the AR-HUD adaptive lane change prompt function. When the function switch is on, the vehicle collects the vehicle's current status information, current surrounding environment data, and current location information in real time.

[0026] It should be noted that, in this embodiment, the current state information includes the vehicle's current speed, acceleration, yaw rate, steering wheel angle, etc., which are used to capture the driver's acceleration, deceleration, steering and other driving operations. The current surrounding environment data includes video data collected by the vehicle's front, rear, left rear, and right rearview cameras, specifically including data such as the road conditions ahead, vehicles approaching from behind, and vehicles in the blind spots on both sides, which are used to capture the temporal dynamic information within each field of view. The current location information includes map data and the vehicle's current position, the current distance between the vehicle and the target intersection, which are used to capture environmental context information such as the complex structure of the intersection, lane connection relationships, and the vehicle's precise position on the map.

[0027] Step 102: Using a prediction model based on the current state information, current surrounding environment data, and current location information, the vehicle's lane change prediction is performed to obtain the vehicle's current collision probability and lane change distance percentage. The prediction model is obtained through iterative training based on the vehicle's historical state information, historical surrounding environment data, and historical location information.

[0028] In this embodiment, the real-time collected vehicle current state information, current surrounding environment data, and current location information are transmitted to the prediction model. The prediction model can extract features from the current state information, current surrounding environment data, and current location information, and integrate them into a set of multi-dimensional feature vectors. Through the lightweight decision head of the prediction model, based on the multi-modal feature vector fused from the current state feature vector, current visual feature vector, and current environment feature vector, lane change prediction is performed on the vehicle to obtain the vehicle's current collision probability and lane change distance percentage. The current collision probability represents the likelihood of a collision when the vehicle changes lanes in the current driving scenario, with a value ranging from 0 to 1. The higher the collision probability, the greater the risk of lane change. The lane change distance percentage represents the percentage of the suggested lane change distance relative to the base distance, with a value ranging from 20% to 180%. The higher the lane change distance percentage, the farther the suggested lane change distance. The base distance is a benchmark value pre-calibrated by the manufacturer, usually set according to factors such as vehicle type, road conditions, and driving habits. For example, the base distance can be calibrated to 50 meters. By using a predictive model to infer from real-time collected and processed multimodal data, lane change prediction can be completed in a short time, obtaining the current collision probability and lane change distance percentage, providing a decision-making basis for subsequent lane change prompts.

[0029] It should be noted that in this embodiment, the prediction model is iteratively trained based on the vehicle's historical state information, historical surrounding environment data, and historical location information. The prediction model includes a modal feature extraction layer and a lightweight decision head. The modal feature extraction layer includes three parallel branches: a vehicle dynamic parameter branch, a visual feature branch, and an environmental context branch. The vehicle dynamic parameter branch is used to process the state information and output a state feature vector, which may include network architectures such as LSTM (Long Short-Term Memory) network and joint attention module. LSTM is used to capture the temporal dependencies of vehicle speed, steering angle, yaw rate, etc., and the joint attention module is used to dynamically adjust the weights of different input features in different driving scenarios. For example, rapid acceleration with a small steering angle may indicate an aggressive lane change, while slow deceleration with a large steering angle may indicate cornering preparation. The visual feature branch processes surrounding environmental data and outputs visual feature vectors. This can include network architectures such as the lightweight Vision Transformer (ViT-Tiny), 3D convolutional kernels, and a spatial attention mechanism / Transformer encoder. The lightweight ViT-Tiny segment video frames and encodes them into sequence features, possessing global modeling capabilities and low computational cost, making it suitable for in-vehicle deployment. The 3D convolutional kernels (3×3×3) capture temporal dynamic information of the video frame sequence (e.g., a 100ms frame sampling interval). The spatial attention mechanism / Transformer encoder fuses visual features from multiple cameras (front-view, rear-view, side-view, etc.) to form a unified panoramic scene representation. The environmental context branch includes a graph attention network (GAT), which fuses map road network topology information with GPS positioning to capture intersection structures, lane connections, and vehicle positional relationships within the graph structure.

[0030] Step 103: Determine the lane change warning distance for the current driving scenario based on the lane change distance percentage, and determine the target lane change warning information based on the current collision probability.

[0031] In this embodiment, the lane change distance percentage and a pre-set base distance are calculated to obtain the lane change warning distance for the current driving scenario. The base distance is a pre-set reference value for the lane change warning distance, and the lane change warning distance is obtained by adjusting the base distance using the lane change distance percentage. This distance is used to trigger a lane change warning at the lane change warning distance. Considering lane change safety, a target lane change warning message is determined based on the current collision probability. The target lane change warning message includes the content of the warning message and the presentation format of the warning message content.

[0032] For example, after determining the lane change warning distance, the target lane change warning message is determined based on the current collision probability. For instance, if the current collision probability is higher than a preset probability threshold, the target lane change warning message is displayed in red: "High risk of collision, please proceed with caution!", strongly alerting the driver that the current lane change carries a high risk. If the current collision probability is lower than or equal to the preset probability threshold, the target lane change warning message is displayed in blue: "Lane change collision risk is low, you may consider changing lanes!", indicating that the current lane change risk is low or that it is advisable to change lanes. Providing different colored lane change warning messages based on the current collision probability helps users promptly identify and respond to potential lane change risks.

[0033] Step 104: Control the vehicle's augmented reality head-up display to provide lane change prompts according to the target lane change prompt information at the lane change reminder distance.

[0034] In this embodiment, after determining the lane change warning distance and the target lane change warning information corresponding to the current collision probability in the current driving scenario, the vehicle controls the augmented reality head-up display to provide lane change warnings at the lane change warning distance according to the target lane change warning information. The AR-HUD can provide warnings through text, icons, or dynamic arrows to ensure that the user can clearly receive the lane change warning information. The vehicle adapts to the driving state through the AR-HUD and provides lane change warnings to the user in a timely and effective manner, thereby improving driving safety and comfort.

[0035] The vehicle lane change warning method provided in this application collects the vehicle's current state information, current surrounding environment data, and current location information. A prediction model is then used to predict the vehicle's lane change based on these information, obtaining the vehicle's current collision probability and lane change distance percentage. The prediction model is iteratively trained using historical state information, historical surrounding environment data, and historical location information. Based on the lane change distance percentage, the lane change warning distance for the current driving scenario is determined. Furthermore, based on the current collision probability, a target lane change warning message is determined, and the vehicle's augmented reality head-up display is controlled to provide lane change warnings at the target lane change warning distance. This application embodiment collects and fuses multimodal data during vehicle driving, and uses a predictive model to infer and predict the real-time collected multimodal data. This accurately obtains the collision probability when the vehicle performs a lane change and the percentage of lane change distance used to prompt the lane change. Based on the collision probability and the percentage of lane change distance, the effective lane change prompt distance and prompt information are determined. This enables the vehicle lane change prompt to adapt to the user's driving state, and executes the lane change prompt in a timely and effective manner. It meets the personalized intelligent driving prompt needs of users under different driving states, improves the intelligence and personalization of vehicle lane change prompts, and further significantly improves driving safety and user experience.

[0036] In some embodiments of this application, step 102 uses a prediction model to predict the vehicle's lane change based on the current state information, current surrounding environment data, and current location information, to obtain the vehicle's current collision probability and lane change distance percentage. Specifically, this may include the following steps: Sub-step 1021 involves extracting features from the current state information, the current surrounding environment data, and the current location information, respectively, to obtain the current state feature vector, the current visual feature vector, and the current environment feature vector. Sub-step 1022: Unify the current state feature vector, current visual feature vector, and current environment feature vector into a unified feature representation to obtain a shared feature vector; Sub-step 1022: Input the shared feature vector into the lightweight decision head of the prediction model, and use the first activation function corresponding to the collision probability and the second activation function corresponding to the lane change distance in the lightweight decision head to predict the lane change and obtain the current collision probability and lane change distance percentage of the vehicle.

[0037] In this embodiment, the vehicle transmits the collected current state information, current surrounding environment data, and current location information to the prediction model. The prediction model then uses these data to predict lane changes, obtaining the vehicle's current collision probability and lane change distance percentage. Specifically, the collected multimodal data is first processed, and features are extracted from the current state information, current surrounding environment data, and current location information to obtain current state feature vectors, current visual feature vectors, and current environment feature vectors. These vectors correspond to the current state information, the current surrounding environment data, and the current location information.

[0038] In this embodiment, the current state information includes the vehicle's current speed, acceleration, yaw rate, steering wheel angle, etc. There is a strong temporal dependency between the current state information. Therefore, a Long Short-Term Memory (LSTM) network can be used to extract temporal features from the current state information to capture the continuity and trend of vehicle motion. Alternatively, a joint attention module can be used to focus on the correlation between, for example, vehicle speed and steering angle, to capture the driver's acceleration, deceleration, steering, and other driving operations, thereby obtaining the current state feature vector. In this embodiment, the current state feature vector can be a 128-dimensional dynamic feature vector with an update frequency of 10Hz to ensure real-time perception of the vehicle's state.

[0039] In this embodiment, the current surrounding environment data includes video data collected by the vehicle's front, rear, left rear, and right rear-view cameras. Specifically, this includes data such as road conditions ahead, vehicles approaching from behind, and vehicles in the blind spots on both sides. The video data from each path can be extracted independently or in parallel using a lightweight Vision Transformer (ViT-Tiny) architecture. To reduce computational load and adapt to vehicle deployment, the input resolution of each video data path is uniformly adjusted to 224x224 pixels. To capture temporal dynamic information within each field of view, a 3D convolutional kernel is introduced to process the video frame sequence of the video data. The frame sampling interval is set to 100 milliseconds to achieve a balance between timeliness and computational load. After processing, each video data path generates its own visual feature vector. The visual feature vectors from different fields of view are fused using a spatial attention mechanism or a Transformer encoder to achieve multi-view visual feature fusion. This maps the visual feature vectors, which include local information from different perspectives, onto a unified global driving scene representation, effectively aggregating the surrounding environment data captured by different cameras to obtain the current visual feature vector corresponding to the current surrounding environment data.

[0040] In this embodiment, the current location information includes map data, the vehicle's current location, and the current distance between the vehicle and the target intersection. Specifically, the vehicle map data is associated with high-precision, real-time GPS positioning. The vehicle map data includes intersection topology, lane information, speed limit information, etc. The vehicle map data and GPS positioning can be fused using a graph attention network (GAT). GAT can effectively process graph structure data, capture the complex structure of intersections, lane connections, and the precise location of the vehicle on the map, and encode them into a fixed-dimensional environmental feature vector to obtain the vehicle's current environmental feature vector.

[0041] In this embodiment, the current state feature vector, current visual feature vector, and current environment feature vector are unified into a shared feature vector. This shared feature vector is then input into the lightweight decision head of the prediction model. Lane change prediction is performed using the first activation function corresponding to the collision probability and the second activation function corresponding to the lane change distance in the lightweight decision head, yielding the vehicle's current collision probability and lane change distance percentage. The lightweight decision head, a part of the prediction model, is used for rapid inference and decision-making, supporting efficient operation on the vehicle. It consists of a shared feature extraction layer and two parallel task branches. The shared feature extraction layer fuses multimodal features, integrating and normalizing the state feature vector, visual feature vector, and environment feature vector, and mapping them to an intermediate dimension, such as 256, to form a unified feature representation. The parallel task branches include a collision probability branch and a lane change distance branch. The collision probability branch predicts the collision probability based on the shared feature vector, while the lane change distance branch predicts the lane change distance percentage based on the shared feature vector.

[0042] In its implementation, the lightweight decision head processes the shared feature vector after multimodal fusion, calculating it through a collision probability branch and a lane change distance branch. In the collision probability branch, the first activation function of the fully connected layer is Sigmoid, outputting a collision probability value between 0 and 1 based on the shared feature vector. In the lane change distance percentage branch, the second activation function of the fully connected layer is ReLU or linear activation, scaling the shared feature vector to a percentage between 20% and 180% to obtain the vehicle's current collision probability and lane change distance percentage. The current collision probability represents the likelihood of a collision when the vehicle changes lanes in the current driving scenario, ranging from 0 to 1; a higher collision probability indicates a greater risk of lane change. The lane change distance percentage represents the percentage of the suggested lane change distance relative to the base distance, ranging from 20% to 180%; a higher percentage indicates a longer suggested lane change distance.

[0043] This application embodiment uses a prediction model to perform inference prediction on real-time, multimodal data, and completes lane change prediction in a short time to obtain the current collision probability and lane change distance percentage, providing a decision basis for subsequent lane change prompts.

[0044] In some embodiments of this application, step 103 determines the lane change warning distance for the current driving scenario based on the lane change distance percentage, and determines the target lane change warning information based on the current collision probability, which may specifically include: Sub-step 1031 calculates the lane change distance percentage and the preset base distance to obtain the lane change reminder distance for the current driving scenario; Sub-step 1032: Compare the current collision probability with the preset probability threshold to determine the collision risk level; Sub-step 1033: Determine the target lane change prompt information corresponding to the collision risk level based on the pre-set correspondence between collision risk level and lane change prompt information.

[0045] In this embodiment, the lane change distance percentage and a pre-set base distance are used to calculate the lane change warning distance for the current driving scenario. The base distance is a pre-set benchmark value for the lane change warning distance, which is usually set based on factors such as vehicle type, road conditions, and driving habits. The base distance can be set to 50 meters, and no specific limitation is made here. The lane change distance percentage is the percentage of the suggested lane change distance output by the prediction model relative to the base distance. Specifically, in this embodiment, the lane change warning distance for the current driving scenario is calculated based on the lane change distance percentage and the base distance. It can dynamically adjust the lane change warning distance based on the lane change distance percentage adapted to the current driving state and road environment, providing effective lane change prompts. The calculation formula for the lane change warning distance is as follows: Lane change warning distance = Base distance + (Base distance * Lane change distance percentage - Base distance) * Max(Distance / mileage threshold for collected data, 1) It should be noted that the maximum value between the ratio of the distance to the mileage threshold of the collected data and 1 is used as the weighting coefficient. The distance of the collected data mainly refers to the user's actual driving data, such as the distance the vehicle traveled (km) when the data was collected. Theoretically, the more data collected from the user, the closer it is to the user's usage habits. The mileage threshold can be 10,000 km, which is adjusted according to the actual driving situation. Based on the calculation of the lane change reminder distance based on the base distance and the percentage of lane change distance, a weighting coefficient reflecting the user's driving habits is introduced for fine-tuning to ensure that the final lane change reminder distance is adaptive to the user's driving state and meets the user's personalized intelligent driving prompt needs under different driving states.

[0046] In this embodiment, to ensure that the lane change prompt information output by the final augmented reality head-up display accurately matches the actual situation, the target lane change prompt information is determined based on the current collision probability. First, the current collision probability is compared with a preset probability threshold to determine the collision risk level. Then, based on the pre-set correspondence between collision risk levels and lane change prompt information, the target lane change prompt information corresponding to the collision risk level is determined. The preset probability threshold can be a pre-defined probability threshold used to classify different collision risk levels. The target lane change prompt information includes the content of the prompt information and the presentation format of the content. Taking a preset probability threshold of 0.4 as an example, if the current collision probability is greater than 0.4, it is determined to be high risk; if the current collision probability is less than or equal to 0.4, it is determined to be low risk. If it is high risk, the target lane change prompt information is displayed in red as "Lane change carries a high collision risk, please change lanes with caution!", strongly warning the driver that the current lane change carries a high risk. If it is low risk, the target lane change prompt information is displayed in blue as "Lane change collision risk is low, you can change lanes when the opportunity arises!", indicating that the current lane change risk is low or acceptable. Based on the current collision probability, different colored lane change prompts are provided to help users identify and respond to potential lane change risks in a timely manner. This embodiment does not limit the specific content and form of the target lane change prompts, and can be adjusted and set according to actual needs and vehicle models.

[0047] This application's embodiments dynamically adjust the lane change warning distance based on the lane change distance percentage, accurately assess the collision risk of lane change behavior based on the current collision probability, help drivers identify potential dangers in a timely manner, realize the intelligence and personalization of vehicle lane change prompts, and significantly improve driving safety and user experience.

[0048] In some embodiments of this application, after step 104 controls the augmented reality head-up display of the vehicle to provide a lane change prompt at the lane change reminder distance according to the target lane change prompt information, it may further include: The collected current status information, current surrounding environment data, and current location information are stored in a preset database as historical status information, historical surrounding environment data, and historical location information. If the data storage volume of the preset database exceeds the preset threshold, or if the historical status information, historical surrounding environment data, and historical location information are reported to the cloud according to the preset period, the cloud will respond to the data reported by the vehicle and perform an iterative training of the prediction model.

[0049] In this embodiment, after a lane change prompt is completed, the collected current status information, current surrounding environment data, and current location information are stored in a preset database as historical status information, historical surrounding environment data, and historical location information, so as to continuously accumulate rich driving data and provide a data foundation for the continuous optimization of the prediction model for subsequent model iteration training and optimization. The preset database can be a local database on the vehicle or a cloud database, and this embodiment does not make specific limitations.

[0050] In this embodiment, if the data storage volume of the preset database exceeds a preset threshold, or if it follows a preset cycle, iterative training of the prediction model is performed using historical state information, historical surrounding environment data, and historical location information. The data storage volume in the preset database is monitored in real time. When the data storage volume exceeds the preset threshold, or if it follows a preset cycle, such as every 7 days, iterative training of the prediction model is triggered. Specifically, this may include: cleaning, normalizing, and extracting features from historical state information, historical surrounding environment data, and historical location information to generate a training set; using the training set to train the prediction model; using the distance from the intersection when the vehicle changes lanes and the collision result as a validation set; and using the difference between the validation set and the result output using the training set as iterative parameters in backpropagation to update the prediction model weights, optimize the prediction model parameters, improve the accuracy of the prediction model output results, and update the latest trained prediction model to the vehicle, replacing the old model and achieving real-time model updates.

[0051] This application embodiment achieves continuous optimization of the prediction model and real-time updates of the lane change prompt function through iterative training of the prediction model, ensuring that the model can continuously learn and adapt to new driving states and environmental changes, thereby improving the accuracy and reliability of the prediction model's decision-making.

[0052] Reference Figure 2 In some embodiments of this application, before step 102 uses a prediction model to predict lane changes for the vehicle based on the current state information, current surrounding environment data, and current location information, it may further include: Step 201: Obtain historical status information, historical surrounding environment data, and historical location information; In this embodiment of the application, the state information, surrounding environment data and location information collected during vehicle driving are cached in a preset database as the data basis for training the prediction model. During model training, historical driving data can be obtained from the preset database. The prediction model is obtained by iterative training based on the historical state information, historical surrounding environment data and historical location information of the vehicle. The historical driving data includes historical state information, historical surrounding environment data and historical location information, providing data support for the training and optimization of the model.

[0053] Step 202: Extract features from historical state information, historical surrounding environment data, and historical location information to obtain state feature vector, visual feature vector, and environmental feature vector respectively. Step 203: Fuse the state feature vector, visual feature vector, and environmental feature vector to obtain a fused feature vector; In this embodiment, feature extraction and feature fusion are performed on the acquired training data to obtain a fused feature vector. Feature extraction is performed on historical state information, historical surrounding environment data, and historical location information to obtain corresponding state feature vectors, visual feature vectors, and environmental feature vectors. Historical state information includes vehicle speed, acceleration, yaw rate, steering wheel angle, etc. Historical surrounding environment data includes video data collected by the vehicle's front, rear, left rear, and right rearview cameras, specifically including data on road conditions ahead, vehicles approaching from behind, and vehicles in blind spots on both sides. Historical location information includes map data, the vehicle's current position, and the current distance between the vehicle and the target intersection. In this embodiment, the process of extracting features from historical state information, historical surrounding environment data, and historical location information to obtain corresponding state feature vectors, visual feature vectors, and environmental feature vectors is similar to the aforementioned process of extracting features from current state information, current surrounding environment data, and current location information to obtain corresponding current state feature vectors, current visual feature vectors, and current environmental feature vectors. Referring to the above embodiments, further details are omitted here.

[0054] In this embodiment, the state feature vector, visual feature vector, and environmental feature vector are mapped to a unified semantic space. The visual feature vector is used as the query vector, and the state feature vector and environmental feature vector are used as a pair of key vectors and value vectors. The dot product similarity between the query vector and the key vector is determined to obtain the attention weight matrix. The attention weight matrix is ​​used to perform a weighted summation on the value vector to obtain the fused feature vector.

[0055] Step 204: The trajectory information of the vehicle within a preset time period is predicted using a pre-constructed vehicle lane line spatiotemporal map.

[0056] In this embodiment, a pre-constructed vehicle lane line spatiotemporal graph is used to predict the trajectory information of vehicles within a preset time period. The construction of the vehicle lane line spatiotemporal graph includes node definition and edge weight definition. Node definition refers to the nodes in the constructed graph, including vehicles (own vehicle and neighboring vehicles) and key lane line points (such as intersection convergence points and lane line endpoints). Edge weight definition includes relative speed, relative distance, and semantic relationships between nodes. Graph convolution operations can be used to capture the complex interactions between traffic participants, primarily the own vehicle and neighboring vehicles, and combined with temporal information, predict the trajectory information of the own vehicle and neighboring vehicles within a preset time period. The preset time period can be 5 seconds, but is not specifically limited here. The predicted trajectory information of vehicles within the preset time period provides important reference for lane change prompts.

[0057] Step 205: Input the fused feature vector and trajectory information into the lightweight decision head for training to obtain a prediction model for outputting the vehicle's current collision probability and lane change distance percentage.

[0058] In this embodiment, the fused feature vector and trajectory information are input into the lightweight decision head for training to obtain a prediction model. Specifically, the fused feature vector and trajectory information are input into the lightweight decision head, and through multiple rounds of iterative training, the model parameters are optimized to obtain a prediction model that outputs the vehicle's current collision probability and lane change distance percentage. In this embodiment, the lightweight decision head includes a collision probability branch and a lane change distance branch. The system or experts label an ideal lane change distance percentage based on collision results from real-world or simulated driving scenarios, driver preferences (such as aggressive or conservative driving styles), and real-time road conditions (such as congestion or rain) as a validation set. The collision probability branch can be trained using the Huber loss function, which is insensitive to outliers and provides more robust collision probability predictions. The loss weight of this branch can be set to 0.3. Since collision probability prediction is inherently an uncertain task, and the Huber loss is more robust, assigning it a weight of 0.3 is already a strong constraint, thus prioritizing driving safety and ensuring that safety is the primary consideration under any circumstances. The lane change distance branch can use the standard mean squared error loss. The loss weight of this branch can be set to 0.7. The output results of the two branches are compared with the corresponding validation sets to calculate the loss. The losses of the two branches are weighted and summed to obtain the total loss. The gradient of the total loss with respect to all parameters of the model (including the two branches) is calculated. The model parameters are updated according to the gradient. The gradient update objective is to minimize the total loss. The above training process is repeated until the model performance converges, and the trained prediction model is obtained.

[0059] This application embodiment utilizes multi-dimensional historical data for model training to obtain a predictive model for outputting the vehicle's current collision probability and lane change distance percentage. This enables lane change prompts to adapt to the user's driving state, providing personalized lane change prompts for different driving states. This enhances the intelligence and personalization of vehicle lane change prompts, significantly improving driving safety and user experience.

[0060] Reference Figure 3 This diagram illustrates the flowchart of the prediction model in the vehicle lane change warning method provided in this application embodiment. The acquired training data is subjected to feature extraction and fusion to generate a fused feature vector. Specifically, based on historical state information, historical surrounding environment data, and historical location information, state feature vectors, visual feature vectors, and environmental feature vectors are obtained. These vectors are then fused to obtain the fused feature vector. The fused feature vector and trajectory information are input into a lightweight decision head for training, resulting in a prediction model that outputs the vehicle's current collision probability and lane change distance percentage. The lightweight decision head includes a collision distance branch and a lane change distance branch. The collision distance branch predicts the collision probability that the current lane change operation may cause, while the lane change distance branch predicts a dynamically adjusted lane change distance percentage, ranging from 20% to 180%. This percentage is used to adjust the lane change warning distance of the AR-HUD to adapt to the driver's personalized preferences and real-time road conditions. For example, a lane change distance percentage of 120% means a warning is given at a location farther than the base distance, while a lane change distance percentage of 20% means a warning is given very close to the intersection.

[0061] In some embodiments of this application, step 203 fuses the state feature vector, visual feature vector, and environmental feature vector to obtain a fused feature vector, which may specifically include the following steps: Sub-step 2031 involves extracting features from historical state information, historical surrounding environment data, and historical location information to obtain state feature vector, visual feature vector, and environmental feature vector, respectively. Sub-step 2032 maps the state feature vector, visual feature vector, and environmental feature vector to a unified semantic space, and uses the visual feature vector as the query vector, and the state feature vector and environmental feature vector as a pair of key vector and value vector; Sub-step 2033: Determine the dot product similarity between the query vector and the key vector to obtain the attention weight matrix. Use the attention weight matrix to perform a weighted summation on the value vectors to obtain the fused feature vector.

[0062] In this embodiment, historical state information, historical surrounding environment data, and historical location information are used as training data. Feature extraction is performed on the historical state information, historical surrounding environment data, and historical location information respectively to obtain state feature vector, visual feature vector, and environmental feature vector, so as to fuse the feature vectors of multimodal data to obtain a fused feature vector.

[0063] As a specific implementation of this application embodiment, historical state information includes vehicle speed, acceleration, yaw rate, steering wheel angle, etc. Key features such as vehicle speed are extracted from historical dynamic parameters to form a state feature vector; historical surrounding environment data includes video data collected by the vehicle's front, rear, left rear, and right rearview cameras, specifically including data such as road conditions ahead, vehicles approaching from behind, and vehicles in blind spots on both sides, etc. Key features such as lane lines, vehicles, and pedestrians are extracted from the video data to generate a visual feature vector; historical location information includes map data and the vehicle's current position and the current distance between the vehicle and the target intersection, key features are extracted to generate an environmental feature vector containing lane line information, the distance between the vehicle and the intersection, etc.

[0064] Specifically, feature vectors from different modalities are mapped to a unified semantic space. Visual feature vectors are used as query vectors, and state and environment feature vectors are used as a key and value vector pair. A learnable projection matrix can be used to map these vectors to a unified semantic space. This mapping eliminates heterogeneity between different modalities, allowing features from each modality to interact within the same space. The unified semantic space is a high-dimensional space, where each dimension represents a semantic feature. The mapping aligns and fuses features from different modalities within this space. Because visual feature vectors contain the most intuitive and richest real-time environmental information, they are used as query vectors. These query vectors are used to actively "query" and "focus" on relevant information from other modalities. State and environment feature vectors are used as key and value vectors, respectively. The key vector represents the feature index, and the value vector represents the specific content of the feature.

[0065] Specifically, the fusion of multimodal features involves calculating the dot product similarity between the query vector and the key vector to obtain an attention weight matrix. The dot product similarity represents the correlation between the query vector and the key vector; a larger value indicates a higher correlation. The dot product similarity matrix, composed of the dot product similarities of the query vector and the key vector, is then normalized to obtain the attention weight matrix. The attention weight matrix is ​​then used to perform a weighted summation of the value vectors to obtain the fused feature vector. The fused feature vector is a weighted combination of the features of each modality in a unified semantic space, which can intelligently weigh the importance of each modality of information, especially in complex or ambiguous scenarios.

[0066] This application embodiment extracts features from historical state information, historical surrounding environment data, and historical location information respectively, and performs multimodal feature fusion to eliminate heterogeneity between different modalities, generating high-quality fused feature vectors to provide input for subsequent model training and significantly improve the prediction accuracy of the model.

[0067] In some embodiments of this application, step 204 uses a pre-constructed vehicle lane line spatiotemporal map to predict the vehicle's trajectory information within a preset time period, which may specifically include the following steps: Sub-step 2041: Determine vehicle nodes and lane line nodes in advance based on historical status information, historical surrounding environment data, and historical location information; Sub-step 2042: Construct a vehicle lane line spatiotemporal graph based on vehicle nodes, lane line nodes, and pre-determined edge weights; Sub-step 2043 involves convolution processing of the vehicle lane line spatiotemporal map to predict the vehicle's trajectory information within a preset time period.

[0068] In this embodiment, vehicle nodes and lane line nodes are pre-determined based on historical state information, historical surrounding environment data, and historical location information. Vehicle nodes contain attribute information about the vehicle and surrounding vehicles, such as position, speed, and acceleration. Lane line nodes contain attribute information about key lane line points, such as intersection convergence points and the location of lane line endpoints. Based on the vehicle nodes, lane line nodes, and pre-determined edge weights, a vehicle lane line spatiotemporal graph is constructed. Edge weights are determined by the relative speed, relative distance, and semantic relationships between nodes. For example, the greater the relative speed between the vehicle and its neighboring vehicle, the greater the edge weight. Edge weights can also be determined by the relative distance between nodes; for example, the smaller the relative distance between the vehicle and its neighboring vehicle, the greater the edge weight. Alternatively, edge weights can be determined by the semantic relationships between nodes, such as whether the vehicle and its neighboring vehicle are traveling in the same direction or are in adjacent lanes. No specific limitations are imposed here.

[0069] It should be noted that the vehicle lane line spatiotemporal graph is a dynamic graph, containing vehicle nodes, lane line nodes, and edges connecting the nodes. The edges in the vehicle lane line spatiotemporal graph represent the interaction relationships between nodes, and the edge weights are dynamically determined by factors such as relative speed, relative distance, or semantic relationships. The vehicle lane line spatiotemporal graph not only includes a spatial dimension—the positions of vehicle and lane line nodes—but also a temporal dimension—the dynamic changes of the nodes over time. The node attributes in the graph are updated over time, and the edge weights are also dynamically adjusted over time.

[0070] In this embodiment, convolution processing is performed on the spatiotemporal graph of vehicle lane lines to predict the trajectory information of vehicles within a preset time period. Specifically, a three-layer graph convolution operation can be used to capture the complex interactions between vehicle nodes, i.e., the self-vehicle and its neighboring vehicles. Each layer of convolution updates the attribute information of the nodes and adjusts the edge weights. Based on graph convolution, temporal convolution is performed by combining temporal information to capture the dynamic changes of nodes over time. Through multi-layer graph convolution and temporal convolution, the trajectory information of the self-vehicle and its neighboring vehicles within the future preset time period is predicted. The trajectory information includes dynamic parameters such as vehicle position, speed, and acceleration, providing important reference for lane change prompts.

[0071] This application embodiment constructs a spatiotemporal graph convolutional network to capture the complex interactions and temporal dynamics between vehicles and neighboring vehicles, thereby achieving accurate prediction of the future trajectory of vehicles. This provides important reference for lane change prompts and enables timely identification and response to potential lane change risks.

[0072] Reference Figure 4 The diagram shows a structural schematic of a vehicle lane change warning device according to an embodiment of this application. The device includes: The information acquisition module 301 is used to collect the vehicle's current status information, current surrounding environment data, and current location information; The lane change prediction module 302 is used to use a prediction model to predict the lane change of the vehicle based on the current state information, current surrounding environment data and current location information, and to obtain the current collision probability and lane change distance percentage of the vehicle; wherein, the prediction model is obtained by iterative training based on the historical state information, historical surrounding environment data and historical location information of the vehicle. The parameter determination module 303 is used to determine the lane change warning distance for the current driving scenario based on the lane change distance percentage, and to determine the target lane change warning information based on the current collision probability; The control module 304 is used to control the vehicle's augmented reality head-up display to provide lane change prompts at the lane change reminder distance according to the target lane change prompt information.

[0073] Optionally, the lane change prediction module 302 includes: The extraction submodule is used to extract features from the current state information, the current surrounding environment data and the current location information respectively, and obtain the current state feature vector, the current visual feature vector and the current environment feature vector respectively. The first processing submodule is used to unify the current state feature vector, the current visual feature vector, and the current environment feature vector into a shared feature vector; The prediction submodule is used to input the shared feature vector into the lightweight decision head of the prediction model, and to perform lane change prediction using the first activation function corresponding to the collision probability and the second activation function corresponding to the lane change distance in the lightweight decision head, so as to obtain the current collision probability and lane change distance percentage of the vehicle.

[0074] Optionally, the parameter determination module 303 includes: The calculation submodule is used to calculate the lane change distance percentage and the preset base distance to obtain the lane change reminder distance for the current driving scenario; The comparison submodule is used to compare the current collision probability with a preset probability threshold to determine the collision risk level; The first determining submodule is used to determine the target lane change prompt information corresponding to the collision risk level based on the pre-set correspondence between the collision risk level and the lane change prompt information.

[0075] Optionally, the device further includes: The data storage module is used to store the collected current status information, current surrounding environment data and current location information into a preset database as historical status information, historical surrounding environment data and historical location information; The model update module is used to detect when the data storage volume of the preset database exceeds a preset threshold, or to perform an iterative training of the prediction model using the historical state information, historical surrounding environment data, and historical location information according to a preset period.

[0076] Optionally, the device further includes: The data acquisition module is used to acquire historical status information, historical surrounding environment data, and historical location information; The feature extraction module is used to extract features from the historical state information, historical surrounding environment data and historical location information respectively, and obtain state feature vector, visual feature vector and environmental feature vector respectively. The feature processing module is used to fuse the state feature vector, visual feature vector and environmental feature vector to obtain a fused feature vector; The trajectory prediction module is used to predict the trajectory information of a vehicle within a preset time period using a pre-built spatiotemporal map of vehicle lane lines. The model training module is used to input the fused feature vector and the trajectory information into the lightweight decision head for training, so as to obtain a prediction model for outputting the vehicle's current collision probability and lane change distance percentage.

[0077] Optionally, the feature processing module includes: The second processing submodule is used to map the state feature vector, visual feature vector and environmental feature vector to a unified semantic space, and use the visual feature vector as a query vector, and the state feature vector and the environmental feature vector as a pair of key vector and value vector; The second determining submodule is used to determine the dot product similarity between the query vector and the key vector to obtain the attention weight matrix; The third processing submodule is used to perform a weighted summation of the value vector using the attention weight matrix to obtain a fused feature vector.

[0078] Optionally, the trajectory prediction module includes: The third determination submodule is used to determine vehicle nodes and lane line nodes in advance based on the historical state information, historical surrounding environment data and historical location information. The graph construction submodule is used to construct a vehicle lane line spatiotemporal graph based on the vehicle nodes, the lane line nodes, and the pre-determined edge weights. The fourth processing submodule is used to perform convolution processing on the vehicle lane line spatiotemporal map to predict the vehicle's trajectory information within a preset time period.

[0079] The vehicle lane change warning device provided in this application embodiment can realize all the processes implemented by the vehicle lane change warning method in the above embodiments of this application. To avoid repetition, it will not be described again here.

[0080] This application embodiment collects and fuses multimodal data during vehicle driving, and uses a predictive model to infer and predict the real-time collected multimodal data. This accurately obtains the collision probability when the vehicle performs a lane change and the percentage of lane change distance used to prompt the lane change. Based on the collision probability and the percentage of lane change distance, the effective lane change prompt distance and prompt information are determined. This enables the vehicle lane change prompt to adapt to the user's driving state, and executes the lane change prompt in a timely and effective manner. It meets the personalized intelligent driving prompt needs of users under different driving states, improves the intelligence and personalization of vehicle lane change prompts, and further significantly improves driving safety and user experience.

[0081] Reference Figure 5 This application also provides an electronic device, such as... Figure 5 As shown, it includes a processor 401, a communication interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404. Processor 401, memory 403 for storing processor-executable instructions; The processor 401 is configured to execute the instructions to implement the vehicle lane change notification method as described above.

[0082] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0083] The communication interface is used for communication between the aforementioned terminal and other devices.

[0084] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0085] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0086] In another embodiment provided in this application, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the vehicle lane change prompting method described in any of the above embodiments.

[0087] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0088] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0090] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for indicating lane changes for vehicles, characterized in that, The method includes: Collect the vehicle's current status information, current surrounding environment data, and current location information; A prediction model is used to predict lane changes for the vehicle based on the current state information, current surrounding environment data, and current location information, to obtain the vehicle's current collision probability and lane change distance percentage; wherein, the prediction model is obtained by iterative training based on historical state information, historical surrounding environment data, and historical location information; Based on the lane change distance percentage, determine the lane change warning distance for the current driving scenario, and based on the current collision probability, determine the target lane change warning information; The augmented reality head-up display controlling the vehicle provides lane change prompts at the specified lane change warning distance, based on the target lane change prompt information.

2. The method according to claim 1, characterized in that, The prediction model uses the current state information, current surrounding environment data, and current location information to predict lane changes for the vehicle, obtaining the vehicle's current collision probability and lane change distance percentage, including: Feature extraction is performed on the current state information, the current surrounding environment data, and the current location information respectively to obtain the current state feature vector, the current visual feature vector, and the current environment feature vector. The current state feature vector, the current visual feature vector, and the current environment feature vector are represented by a unified feature vector to obtain a shared feature vector. The shared feature vector is input into the lightweight decision head of the prediction model. The collision probability is used as the first activation function and the lane change distance is used as the second activation function in the lightweight decision head to predict the lane change and obtain the vehicle's current collision probability and lane change distance percentage.

3. The method according to claim 1, characterized in that, The process of determining the lane change warning distance for the current driving scenario based on the lane change distance percentage, and determining the target lane change warning information based on the current collision probability, includes: The lane change distance percentage and the preset base distance are used to calculate the lane change warning distance for the current driving scenario; The current collision probability is compared with a preset probability threshold to determine the collision risk level; Based on the pre-defined correspondence between collision risk levels and lane change warning information, the target lane change warning information corresponding to the collision risk level is determined.

4. The method according to claim 1, characterized in that, After the augmented reality head-up display controlling the vehicle provides a lane change prompt at the lane change warning distance according to the target lane change prompt information, it also includes: The collected current status information, current surrounding environment data, and current location information are stored in a preset database as historical status information, historical surrounding environment data, and historical location information. If the data storage volume of the preset database exceeds the preset threshold, or if the historical state information, historical surrounding environment data, and historical location information are used to perform an iterative training of the prediction model according to the preset period, the prediction model will be trained once.

5. The method according to claim 1 or 4, characterized in that, Before using the prediction model to predict vehicle lane changes based on the current state information, current surrounding environment data, and current location information, the method further includes: Acquire historical status information, historical surrounding environment data, and historical location information; Feature extraction is performed on the historical state information, historical surrounding environment data, and historical location information respectively to obtain state feature vector, visual feature vector, and environmental feature vector. The state feature vector, visual feature vector, and environmental feature vector are fused to obtain a fused feature vector. The trajectory information of a vehicle within a preset time period is predicted using a pre-constructed spatiotemporal map of vehicle lane lines. The fused feature vector and the trajectory information are input into the lightweight decision head for training to obtain a prediction model for outputting the vehicle's current collision probability and lane change distance percentage.

6. The method according to claim 5, characterized in that, The process of fusing the state feature vector, visual feature vector, and environmental feature vector to obtain a fused feature vector includes: The state feature vector, visual feature vector, and environmental feature vector are mapped to a unified semantic space, and the visual feature vector is used as a query vector, while the state feature vector and the environmental feature vector are used as a pair of key vector and value vector. Determine the dot product similarity between the query vector and the key vector to obtain the attention weight matrix; The value vector is weighted and summed using the attention weight matrix to obtain the fused feature vector.

7. The method according to claim 5, characterized in that, The method of predicting vehicle trajectory information within a preset time period using a pre-constructed vehicle lane line spatiotemporal map includes: Vehicle nodes and lane line nodes are determined in advance based on the historical state information, historical surrounding environment data, and historical location information; Based on the vehicle nodes, the lane line nodes, and the pre-determined edge weights, a vehicle lane line spatiotemporal graph is constructed. The vehicle lane line spatiotemporal map is convolved to predict the vehicle's trajectory information within a preset time period.

8. A vehicle lane change warning device, characterized in that, The device includes: The information acquisition module is used to collect the vehicle's current status information, current surrounding environment data, and current location information; The lane change prediction module is used to predict lane changes of the vehicle based on the current state information, current surrounding environment data, and current location information using a prediction model, and to obtain the vehicle's current collision probability and lane change distance percentage; wherein, the prediction model is obtained by iterative training based on historical state information, historical surrounding environment data, and historical location information. The parameter determination module is used to determine the lane change warning distance for the current driving scenario based on the lane change distance percentage, and to determine the target lane change warning information based on the current collision probability; The control module is used to control the vehicle's augmented reality head-up display to provide lane change prompts at the lane change reminder distance according to the target lane change prompt information.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the instructions to implement the vehicle lane change notification method as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that, A computer program is stored on the readable storage medium, which, when executed by a processor, implements the vehicle lane change notification method as described in any one of claims 1 to 7.

Images