A driving assistance method, device, vehicle, and storage medium

Abstract

This application discloses a driving assistance method, device, vehicle, and storage medium, relating to the field of vehicle control technology. The method includes: acquiring a video stream obtained by a camera capturing the vehicle's surrounding environment and acquiring the vehicle's driving status information at the current moment; inputting each frame of the video stream into a pre-trained target detection model to obtain target information in each frame, and determining dynamic and static targets based on the target information; determining the static environment information of each frame based on the category and position of the static targets in each frame, and determining the motion trajectory of each dynamic target within a historical time period based on the position of the dynamic targets in each frame; inputting the motion trajectory of each dynamic target within the historical time period, the static environment information of each frame, and the driving status information into a pre-trained driving decision prediction model to obtain the driving decision result of the vehicle at the next moment, thus predicting the driving decision result at the next moment.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a driving assistance method, device, vehicle, and storage medium. Background Technology

[0002] With the increasing popularity of private cars, more and more novice drivers are starting to drive independently on the road. Existing driver assistance technologies use onboard radar or cameras to monitor vehicles or obstacles ahead. When a potential collision hazard is detected, an early warning is issued to remind the driver to take timely braking or evasive action.

[0003] However, the above solutions cannot be fully applied to complex and ever-changing real-world road conditions, nor can they accurately predict the correct driving decisions for the next moment. As a result, novice drivers lacking driving experience are unable to take effective measures when encountering emergencies, which increases the risk of traffic accidents and threatens personal safety. Summary of the Invention

[0004] This application provides a driving assistance method, device, vehicle, and storage medium that can predict the driving decision result at the next moment, thereby realizing driving assistance functions and solving the problem in the prior art that it is impossible to accurately predict the correct driving decision at the next moment.

[0005] In a first aspect, embodiments of this application provide a driving assistance method applied to a vehicle, the vehicle being equipped with a camera, the method comprising:

[0006] Acquire video streams from the camera capturing the vehicle's surrounding environment and obtain the vehicle's current driving status information;

[0007] Each frame of the video stream is input into a pre-trained target detection model to obtain target information in each frame. Based on the target information, dynamic and static targets are identified. The target information includes category and location.

[0008] The static environment information of each frame is determined based on the category and location of static targets in each frame, and the motion trajectory of each dynamic target in the historical time period is determined based on the location of dynamic targets in each frame.

[0009] The motion trajectory of each dynamic target within a historical time period, the static environmental information of each frame, and the driving state information are input into a pre-trained driving decision prediction model to obtain the driving decision result of the vehicle at the next moment, so as to assist the driver in driving the vehicle.

[0010] Secondly, embodiments of this application provide a driving assistance device applied to a vehicle, wherein a camera is installed on the vehicle, and the device includes:

[0011] The acquisition module is used to acquire video streams obtained by the camera from the vehicle's surrounding environment and to acquire the vehicle's driving status information at the current moment.

[0012] The target detection module is used to input each frame of the video stream into the pre-trained target detection model to obtain the target information in each frame and to determine the dynamic and static targets based on the target information, which includes the category and location.

[0013] The determination module is used to determine the static environment information of each frame image based on the category and position of static targets in each frame image, and to determine the motion trajectory of each dynamic target in the historical time period based on the position of dynamic targets in each frame image.

[0014] The decision prediction module is used to input the motion trajectory of each dynamic target in the historical time period, the static environmental information of each frame image and the driving state information into the pre-trained driving decision prediction model to obtain the driving decision result of the vehicle in the next moment, so as to assist the driver in driving the vehicle.

[0015] Thirdly, embodiments of this application provide a vehicle, the vehicle comprising:

[0016] At least one processor; and a memory communicatively connected to the at least one processor;

[0017] The memory stores a computer program that can be executed by at least one processor, which enables the at least one processor to perform the driving assistance method of any embodiment of this application.

[0018] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a driving assistance method as described in any embodiment of this application.

[0019] In this embodiment, a video stream of the vehicle's surrounding environment captured by a camera can be acquired, along with the vehicle's driving status information at the current moment. Each frame of the video stream is input into a pre-trained target detection model to obtain target information in each frame. Based on this target information, dynamic and static targets are identified, including their category and location. The static environment information of each frame is determined based on the category and location of the static targets, and the motion trajectory of each dynamic target within a historical time period is determined based on the location of the dynamic targets in each frame. The motion trajectory of each dynamic target within a historical time period, the static environment information of each frame, and the driving status information are input into a pre-trained driving decision prediction model to obtain the driving decision result for the vehicle at the next moment, thereby assisting the driver in driving the vehicle. In the above technical solution, a pre-trained target detection model can accurately detect target information in each frame of the image, thereby improving the accuracy of detecting surrounding environment information. Then, a pre-trained driving decision prediction model can accurately capture driving decision patterns in different driving scenarios, thus making it applicable to complex and ever-changing real road conditions. Furthermore, by comprehensively considering multiple factors such as the motion trajectory of dynamic targets, static environmental information, and driving status information, the driving decision prediction model can accurately predict the driving decision result of the vehicle at the next moment, solving the problem of existing technologies being unable to accurately predict the correct driving decision at the next moment. This realizes driving assistance functions, thereby reducing the risk of traffic accidents, protecting the personal safety of vehicle occupants and other road users, and improving the user experience. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic flowchart of a driving assistance method provided in an embodiment of this application;

[0022] Figure 2 This is another schematic flowchart of the driving assistance method provided in the embodiments of this application;

[0023] Figure 3 This is a schematic diagram of a driving assistance device provided in an embodiment of this application;

[0024] Figure 4 This is a structural schematic diagram of a vehicle provided in an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0026] It should be noted that the terms "first," "second," "target," and "original," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein. Furthermore, the terms "comprising," "having," and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] Figure 1 This is a flowchart illustrating a driving assistance method provided in an embodiment of this application. This embodiment can be applied to scenarios where it is necessary to predict the vehicle's driving decisions in the future to assist the driver. The driving assistance method provided in this embodiment can be executed by a driving assistance device provided in this application, which can be implemented through software and / or hardware. In a specific embodiment, the driving assistance device can be integrated into a vehicle, which may include a vehicle controller and has a camera installed on it. The executing entity for this method can be the vehicle controller, see [link to relevant documentation]. Figure 1 The driving assistance method in this embodiment includes, but is not limited to, the following steps:

[0028] S110. Obtain the video stream obtained by the camera capturing the surrounding environment of the vehicle, and obtain the vehicle's driving status information at the current moment.

[0029] The video stream is a dynamic data sequence composed of continuous image frames, used to characterize the vehicle's surrounding environment within a historical time period, such as the real-time images of the road in front of the vehicle, the sides of the vehicle, and the rear of the vehicle; the historical time period is a time interval before the current moment, and the length of the historical time period can be determined according to the actual situation, such as a few seconds or minutes in the past.

[0030] Driving status information refers to various parameters of a vehicle during its operation, reflecting its motion state. For example, driving status information may include speed, acceleration, steering angle, and gear position.

[0031] Specifically, when it is necessary to predict the driving decisions of a vehicle in the future, video streams obtained by the vehicle's cameras during historical time periods can be acquired. For example, during the driving process, the vehicle's cameras can capture the vehicle's surrounding environment in real time and send the captured images to the vehicle controller in the form of video streams. At this time, the vehicle controller can capture the video streams during historical time periods.

[0032] Then, the vehicle's driving status information at the current moment can be obtained. For example, the vehicle's speed at the current moment can be measured by a speed sensor installed near the transmission output shaft or on the wheels; the vehicle's acceleration at the current moment can be measured by an inertial measurement unit; or the vehicle's steering angle at the current moment can be measured by a steering angle sensor installed on the steering column or steering gear.

[0033] S120. Input each frame of the video stream into the pre-trained target detection model to obtain the target information in each frame, and determine the dynamic and static targets based on the target information.

[0034] The object detection model is a pre-trained neural network model used to detect various target objects in the input image, such as vehicles, pedestrians, traffic signs, and trees, and to detect the target information corresponding to each target. The object detection model is pre-trained on a large amount of labeled image data, allowing it to learn the feature patterns of different targets, such as the shape, color, and texture of vehicles, enabling it to accurately detect targets in new images. For example, the object detection model can be an SSD (Single Shot MultiBoxDetector) model.

[0035] The target information is the output of the target detection model after detecting each frame of the image. The target information can include the target's category and location. The category is the type of the target, such as vehicles, pedestrians, and traffic signs. The location is the bounding box coordinates of the target, which can accurately locate the target's position in the image. The bounding box coordinates can be pixel coordinates relative to the top left corner of the image. In this case, the bounding box coordinates can include the top left corner coordinates, the bottom left corner coordinates, the top right corner coordinates, the bottom right corner coordinates, and the center coordinates of the bounding box.

[0036] Dynamic targets are objects in the vehicle's surrounding environment whose position changes rapidly over time; for example, dynamic targets may include other vehicles in motion, pedestrians walking, and moving animals. Static targets are objects in the vehicle's surrounding environment whose position is relatively fixed; for example, static targets may include roadside buildings, traffic signs, lane markings, pedestrian crossings, streetlights, and trees.

[0037] Specifically, after obtaining the video stream and driving status information, the video stream can be decomposed according to the frame rate to obtain each frame image. Then, the format of each frame image is adjusted, such as scaling and color space conversion, so that the format of each frame image meets the input requirements of the object detection model.

[0038] Then, each frame of image can be input into the pre-trained target detection model in chronological order. The target detection model then performs target detection on each frame of image to obtain the target information of each target in each frame. That is, for the current frame of image, the current frame can be input into the target detection model, and the target detection model can perform feature extraction, target localization and classification on the current frame to determine the category and bounding box coordinates of each target in the current frame, thereby obtaining the target information corresponding to each target in the current frame.

[0039] Subsequently, dynamic and static targets can be identified based on target information. For example, dynamic and static targets can be determined based on the target's position in each frame of the image. Specifically, based on the target's position in each frame, it can be determined whether the target's position has changed within a historical time period. If the target's position change within a historical time period exceeds a preset distance threshold, it indicates that the target has changed its position within that time period, and thus the target can be identified as a dynamic target. If the target's position change within a historical time period does not exceed the preset distance threshold, it indicates that the target has not changed its position within that time period, and thus the target can be identified as a static target. Here, the position change is the sum of the target's position changes between adjacent frames within a historical time period; the preset distance threshold is a pre-set distance value used to determine whether the target has changed its position within a historical time period. This can filter out target position changes caused by detection errors or other minor interference factors, thereby more accurately distinguishing between dynamic and static targets. By reasonably setting the preset distance threshold, the accuracy of judging the target's dynamic or static attributes can be improved, thus providing a more reliable basis for subsequent driving decisions.

[0040] S130. Determine the static environment information of each frame image based on the category and location of the static target in each frame image, and determine the motion trajectory of each dynamic target in the historical time period based on the location of the dynamic target in each frame image.

[0041] Static environment information, constructed based on the category and location of static targets in each image frame, describes the relatively stable environment around the vehicle, providing background information for subsequent predictive driving decisions. For example, static environment information may include the location of lane boundaries and the location and content of traffic signs.

[0042] A motion trajectory is the path through which the position of a dynamic target changes over a historical period of time. It reflects information such as the direction and speed of the dynamic target's movement and is of great significance for predicting the future behavior of the dynamic target and determining driving decisions (such as whether to avoid it or maintain a safe distance). The motion trajectory can include the position, speed and acceleration of the dynamic target at each trajectory point.

[0043] Specifically, after identifying static and dynamic targets, any frame can be selected as the current frame. Static targets are then selected from the targets included in the current frame. Machine learning methods are then used to determine the static environment information of the current frame based on the category and location of the static targets, thus depicting the relatively stable environment around the vehicle. For example, if the static target is a traffic sign, optical character recognition (OCR) technology can be used to identify the specific content within the bounding box of the static target, such as speed limits and directional indicators. The bounding box is the area formed by the coordinates of its top-left, bottom-left, top-right, and bottom-right corners. If the static target is a lane line, edge detection algorithms can be used to detect the position of the lane line within the bounding box of the static target, thus detecting the lane boundary. Afterward, other frames can be selected as the current frame, and the above process can be repeated to obtain the static environment information for each frame.

[0044] Then, the motion trajectory of each dynamic target within a historical time period can be determined based on the position of the dynamic target in each frame of the image. For example, any dynamic target can be selected as the current dynamic target, and the position of the current dynamic target in each frame of the image can be obtained from the target information in each frame of the image. Then, these position points are connected sequentially in chronological order to obtain the motion trajectory of the current dynamic target within a historical time period. The position of each trajectory point is the position of the current dynamic target in the corresponding frame of the image. The time interval between adjacent frames is determined according to the frame rate of the video stream. Then, the velocity corresponding to each trajectory point is determined based on the position change between adjacent trajectory points and the time interval using the central difference method. The acceleration corresponding to each trajectory point is determined based on the velocity change between trajectory points and the time interval using the second-order central difference method. This gives the motion trajectory of the current dynamic target within a historical time period. After that, other dynamic targets are selected as the current dynamic targets, and the above process is repeated to obtain the motion trajectory of each dynamic target within a historical time period.

[0045] For example, the calculation process for the velocity of the third trajectory point is as follows: Calculate the position change between the second and third trajectory points (denoted as the first displacement), and calculate the position change between the third and fourth trajectory points (denoted as the second displacement). Then, calculate the sum of the first and second displacements, and calculate the ratio of this sum to twice the time interval to obtain the velocity of the third trajectory point. The calculation process for the acceleration of the third trajectory point is as follows: Calculate the difference between the velocity of the fourth trajectory point and the velocity of the second trajectory point, and calculate the ratio of this difference to twice the time interval to obtain the acceleration of the third trajectory point.

[0046] S140. Input the motion trajectory of each dynamic target in the historical time period, the static environment information of each frame image and the driving state information into the pre-trained driving decision prediction model to obtain the driving decision result of the vehicle at the next moment, so as to assist the driver in driving the vehicle.

[0047] Among them, the driving decision prediction model is a pre-trained neural network model used to predict the driving decision results of the vehicle in the future. By learning a large amount of driving scenario data, the driving decision prediction model can capture the relationship between the vehicle's driving status information, the motion trajectory of dynamic targets and static environment information and the driving decision results, thereby learning the driving decision patterns under different driving scenarios.

[0048] The driving decision result is the output of the driving decision prediction model, used to characterize the driving action the vehicle should take in the next moment. It can assist the driver in driving the vehicle and provide timely and reasonable driving suggestions. For example, the driving decision result may include speed control instructions (such as accelerating, decelerating, or maintaining the current speed), steering operation instructions (such as turning left or right at a certain angle, or maintaining straight driving), and following distance adjustment instructions (such as moving closer to or further away from the target vehicle in front).

[0049] Specifically, after obtaining the static environment information of each frame and the motion trajectory of each dynamic target within a historical time period, the motion trajectory of each dynamic target within a historical time period, the static environment information of each frame and the driving state information can be input into a pre-trained driving decision prediction model. At this time, the driving decision prediction model can extract features from the static environment information of each frame to obtain the feature map corresponding to each frame. Then, based on the feature map of each frame, the motion trajectory of each dynamic target within a historical time period, and the driving state information, the driving decision result of the vehicle at the next moment can be predicted. Specifically, the driving decision prediction model can predict the surrounding environment information at the next moment, such as the motion trend of the dynamic target at the next moment, based on the feature map corresponding to each frame and the motion trajectory of each dynamic target within a historical time period. It can also use the learned driving decision patterns under different driving scenarios to process the driving state information and the surrounding environment information at the next moment to predict the driving decision result of the vehicle at the next moment.

[0050] Optionally, after determining the driving decision, the result can be displayed, for example, on the vehicle's central control screen or announced via voice, to assist the driver, especially novice drivers. For example, when changing lanes, the system can remind novice drivers to check their rearview mirrors and turn on their turn signals in advance; when driving on narrow roads, the system can remind novice drivers to be aware of their distance from roadside obstacles; and when encountering a roundabout, the system can explain in detail the rules and precautions for entering and exiting the roundabout.

[0051] The technical solution of this application embodiment can accurately detect target information in each frame of image through a pre-trained target detection model, thereby improving the detection accuracy of surrounding environment information. Then, through a pre-trained driving decision prediction model, it can accurately capture driving decision patterns in different driving scenarios, thus being fully applicable to complex and ever-changing real road conditions. Furthermore, by comprehensively considering multiple factors such as the motion trajectory of dynamic targets, static environmental information, and driving status information, the driving decision prediction model can accurately predict the driving decision result of the vehicle at the next moment, solving the problem of the inability to accurately predict the correct driving decision at the next moment in the prior art, realizing driving assistance functions, thereby reducing the risk of traffic accidents, protecting the personal safety of vehicle occupants and other road users, and improving the user experience.

[0052] The following further describes a driving assistance method provided by an embodiment of this application. Figure 2 This is another schematic flowchart of the driving assistance method provided in this application. The embodiments of this application are optimizations based on the above embodiments. See also... Figure 2 The method in this embodiment includes, but is not limited to, the following steps:

[0053] S201. Obtain the video stream obtained by the camera capturing the surrounding environment of the vehicle, and obtain the vehicle's driving status information at the current moment.

[0054] S202. Input each frame of the video stream into the pre-trained target detection model to obtain the target information in each frame, and determine the dynamic and static targets based on the target information.

[0055] Specifically, after obtaining the target information, dynamic and static targets can be determined based on this information. That is, if a category falls within a preset static category table, the target corresponding to that category is identified as a static target; if a category falls within a preset dynamic category table, the target corresponding to that category is identified as a dynamic target. The preset static category table is a pre-set list used to store categories identified as static targets in a driving scenario; the preset dynamic category table is a pre-set list used to store categories identified as dynamic targets in a driving scenario. The targets corresponding to these categories typically change position within the scene. By using the preset static and dynamic category tables, targets can be quickly classified without tracking them to determine if their position has changed. This reduces implementation complexity, improves classification efficiency, and reduces misclassifications caused by position detection errors or transient visual interference, thereby improving classification accuracy and providing an accurate data foundation for subsequent understanding of the driving scenario.

[0056] Optionally, the training process of the object detection model is as follows: acquire multiple sample images and label the category and location of each object in each sample image to obtain the object information label corresponding to each sample image; input the sample images into the initial object detection model, use the object information label corresponding to the sample images to guide the training output of the initial object detection model, train the initial object detection model, and obtain the object detection model.

[0057] The target information label includes category and location; the initial target detection model is an untrained model framework used to learn various features of different targets in order to achieve the target detection function.

[0058] Specifically, a large amount of image data including various road traffic conditions and traffic signs can be collected to obtain multiple sample images. These sample images can cover different weather conditions (such as sunny, rainy, and cloudy days), different time scenarios (such as day and night), and different road types (such as urban roads, highways, and rural roads). Next, the multiple sample images are labeled, indicating the category and location of each target in each sample image, obtaining the target information label corresponding to each sample image. Then, the sample images are input into the initial target detection model to obtain the training output, and the loss value between the training output and the target information label corresponding to the sample image is calculated. Then, the parameters of the initial target detection model are updated using the backpropagation algorithm to minimize the loss value, obtaining the optimal initial target detection model, i.e., the target detection model. Training with a large number of labeled sample images allows the target detection model to learn various features of different targets and be exposed to different driving scenarios, thereby improving the detection accuracy of the target detection model.

[0059] S203. Determine the static environment information of each frame of image based on the category and location of static targets in each frame.

[0060] Specifically, S204 or S206 can then be executed.

[0061] S204. Label each frame image based on the position of the dynamic target in each frame image to obtain the target frame image sequence.

[0062] The target frame image sequence is a series of frame images obtained by annotating each frame image based on the position of the dynamic target in each frame image; in the target frame image sequence, the position of the dynamic target in the frame image is clearly marked in each frame image.

[0063] Specifically, after obtaining the static environment information of each frame image, annotation tools can be used to annotate the position of the dynamic target in each frame image based on the position of the dynamic target in that frame image, thus obtaining a series of image frames. For example, using bounding boxes, the dynamic target is annotated in the frame image based on the bounding box coordinates of the frame image. Then, these frame images are arranged in chronological order to form a target frame image sequence.

[0064] S205. Input the target frame image sequence into the pre-trained trajectory detection model to obtain the motion trajectory of each dynamic target within the historical time period.

[0065] The trajectory detection model is a pre-trained neural network model used to extract the motion trajectory of a target from a series of image sequences with target location annotations. Especially when the vehicle is moving fast or the target is partially occluded, the trajectory detection model can infer the position and state of the target by analyzing the information of previous frame images, reducing the occurrence of tracking loss. For example, the trajectory detection model is a three-dimensional convolutional neural network (3D-CNN).

[0066] Specifically, after obtaining the target frame image sequence, a pre-trained trajectory detection model can be acquired and input into it. This model then uses multiple convolutional, pooling, and fully connected layers to extract spatiotemporal features from the target frame image sequence. Based on these extracted features, it captures the position and state of each dynamic target in each frame, continuously tracking each target. Then, in chronological order, the position and state of each target in each frame are used to form its corresponding motion trajectory, thus obtaining the motion trajectory of each target over a historical time period. Afterward, step S208 can be executed.

[0067] S206. Determine the initial position of each dynamic target based on the position of the dynamic target in each frame of the image.

[0068] The initial position is the location of the dynamic target in the corresponding frame image when it first appears in the video stream within the historical time period, and it serves as the starting point for subsequent tracking of the dynamic target's motion trajectory.

[0069] Specifically, after obtaining the static environment information of each frame, the position of the dynamic target in the corresponding frame when it first appears in the video stream can be determined as the initial position of the dynamic target. For example, since the historical time period is a relatively short time range, the frame image when the dynamic target first appears in the video stream can be considered as the first frame image, and the position of the dynamic target in the first frame image can be determined as the initial position of the dynamic target. However, it is not excluded that the frame image when the dynamic target first appears in the video stream is another frame image. In this case, the frame image when the dynamic target first appears in the video stream can be determined, and the position of the dynamic target in that frame image can be determined as the initial position of the dynamic target.

[0070] S207. Based on the initial position of each dynamic target, continuously track the dynamic targets to obtain the motion trajectory of each dynamic target within a historical time period.

[0071] Specifically, after obtaining the initial position, target tracking technology can be used to continuously track the dynamic targets based on their initial positions, capturing the position and state of each dynamic target in each frame of the image. This yields the motion trajectory of each dynamic target over a historical time period, improving the efficiency of target tracking. For example, the target tracking technology can be a Kalman filter or an optical flow algorithm. The specific tracking process can refer to the standard tracking process of the Kalman filter or the standard tracking process of the optical flow algorithm, which will not be elaborated here. Afterwards, step S208 can be executed.

[0072] It should be noted that S204 and S205, along with S206 and S207, are parallel schemes, and any scheme can be selected for execution according to the actual situation. This embodiment does not make specific limitations on this.

[0073] S208. Input the motion trajectory of each dynamic target in the historical time period, the static environment information of each frame image and the driving state information into the pre-trained driving decision prediction model to obtain the driving decision result of the vehicle at the next moment, so as to assist the driver in driving the vehicle.

[0074] Optionally, the driving decision prediction model may include a feature extraction module and a driving decision module; the feature extraction module is used for feature extraction, and for example, the feature extraction module can be R-CNN (Region-based Convolutional Neural Networks); the driving decision module is used to learn the dependencies and change patterns of the input driving environment over time, predict the surrounding environment information at the next moment, and combine it with the vehicle's driving state information to predict the driving decision result at the next moment, and for example, the driving decision module can be a Long Short-Term Memory (LSTM) network; the training process of the driving decision prediction model is as follows, including Sa1-Sa4:

[0075] Sa1, Obtain the sample dataset.

[0076] The sample dataset includes sample driving state information, sample static environment information, sample dynamic target trajectory, and driving decision result label. The sample static environment information is a sequence, including sample static environment information corresponding to multiple frames of images. The driving decision result label is the correct driving decision result corresponding to the sample driving state information, sample static environment information, and sample dynamic target trajectory, that is, the correct driving decision of the vehicle at the next moment corresponding to the sample driving state information.

[0077] Sa2. Input the static environment information of the sample into the feature extraction module to obtain the target feature map sequence.

[0078] The target feature map sequence is the output result obtained after inputting the static environmental information of the sample into the feature extraction module.

[0079] Specifically, after obtaining the sample dataset, the sample static environment information can be input into the feature extraction module. At this time, the feature extraction module performs feature extraction on the sample static environment information of each frame of the image in the sample static environment information to obtain the target feature map corresponding to each frame of the image. Then, the target feature maps are arranged in chronological order to form a target feature map sequence.

[0080] Sa3. Input the sample driving status information, sample dynamic target motion trajectory and target feature map sequence into the driving decision module to obtain the driving decision prediction result.

[0081] Among them, the driving decision prediction result is the prediction result of the driving decision module.

[0082] Specifically, after the target feature map sequence, the sample driving state information, the sample dynamic target motion trajectory, and the target feature map sequence can be input into the driving decision module. At this time, the driving decision module can extract the dependence and change pattern of the sample dynamic target motion trajectory and the target feature map sequence in the time series, predict the surrounding environment information of the next moment corresponding to the sample driving state information, and then predict the driving decision of the vehicle at the next moment corresponding to the sample driving state information based on the surrounding environment information and the sample driving state information, that is, the driving decision prediction result.

[0083] Sa4. Calculate the loss value between the driving decision prediction result and the driving decision result label, and use the backpropagation algorithm to optimize the feature extraction module and the driving decision module to minimize the loss value and obtain the driving decision prediction model.

[0084] Specifically, after obtaining the driving decision prediction result, a supervised learning approach can be used to compare the driving decision prediction result with the driving decision result label, and calculate the loss value corresponding to the loss function. At this time, the loss function can comprehensively consider multiple aspects of error, such as driving behavior classification errors (e.g., incorrect judgment of whether braking is needed) and speed control errors (e.g., deviation between actual speed and ideal speed). Then, the backpropagation algorithm is used to update and adjust the parameters of the feature extraction module and the driving decision module according to the gradient information of the loss function, continuously optimizing the weights and biases of the feature extraction module and the driving decision module to minimize the loss value. This allows the feature extraction module and the driving decision module to gradually learn the mapping relationship from the input data to the driving decision result label, such as the vehicle's speed adjustment mode and steering timing under different traffic environments and target motion conditions, thereby obtaining the driving decision prediction model.

[0085] In this embodiment, by integrating sample driving state information, sample static environment information, and sample dynamic target motion trajectory, the driving decision prediction model can comprehensively consider various factors in the driving scenario. Furthermore, the sample dataset contains various driving scenario information, covering multiple road conditions, weather conditions, and traffic conditions, enabling the feature extraction module to effectively extract key features from complex static environment information and allowing the driving decision module to better learn driving decision patterns under different driving scenarios, thereby improving the prediction accuracy of the driving decision prediction model.

[0086] S209. Obtain the real-time driving status information of the vehicle after the current moment.

[0087] Among them, real-time driving status information is data about the vehicle's driving status collected in real time by various on-board sensors (such as speed sensors, acceleration sensors, gyroscopes, and on-board positioning systems) after the current moment, which can accurately reflect the vehicle's driving status.

[0088] Specifically, after obtaining the driving decision result, the vehicle's driving status information can be obtained in real time starting from the current moment, and the real-time driving status information of the vehicle after the current moment and before the next moment can be obtained. At this time, there are multiple real-time driving status information.

[0089] S210. Determine the target state of the vehicle at the next moment based on real-time driving status information.

[0090] The target state is the state that the vehicle will reach in the next moment, obtained through certain prediction or calculation methods based on the vehicle's real-time driving status information.

[0091] Specifically, after obtaining real-time driving status information, key parameters such as the vehicle's speed, position, and steering angle at the next moment can be predicted based on the real-time driving status information, thereby obtaining the vehicle's target state at the next moment. For example, based on the speed and acceleration in the real-time driving status information, the speed that the vehicle can reach at the next moment can be predicted, and the position that the vehicle can reach at the next moment can be predicted, thus obtaining the predicted position; based on the steering angle and steering speed in the real-time driving status information, the steering angle that the vehicle can reach at the next moment can be predicted, thus obtaining the predicted steering angle.

[0092] S211. Determine whether the target state and the expected state corresponding to the driving decision result are the same.

[0093] The desired state is the ideal state that the vehicle should reach in the next moment, determined based on the driving decision. For example, when the driving decision is to turn right at the intersection ahead, the desired state could be that the vehicle appropriately slows down as it approaches the intersection, gradually adjusts the steering angle to a suitable right turn angle, and accurately enters the right turn lane.

[0094] Specifically, the target state can be compared with the desired state. The comparison can include key parameters such as speed, position, and steering angle. For example, it can be compared whether the predicted speed in the target state matches the speed range required by the desired state, whether the predicted position in the target state matches the lane and location the vehicle should be in in the desired state, and whether the predicted steering angle in the target state meets the turning operation requirements in the desired state. This determines whether the target state is the same as the desired state corresponding to the driving decision. When the target state is the same as the desired state corresponding to the driving decision, it indicates that the driver is driving the vehicle according to the driving decision. At this time, S201 can be executed to continue to determine the driving decision results at other future moments. When the target state is different from the desired state corresponding to the driving decision, it indicates that the driver is not driving the vehicle according to the driving decision. At this time, S212 can be executed.

[0095] S212. When the target state is different from the expected state corresponding to the driving decision result, display the driving decision result to remind the driver to drive the vehicle according to the driving decision result.

[0096] Specifically, when the target state and the expected state corresponding to the driving decision result are different, the driving decision result can be displayed on the vehicle's central control screen or announced by voice to remind the driver to drive the vehicle according to the driving decision result, especially for novice drivers.

[0097] For example, if there is a pedestrian crossing ahead and the driving decision is to slow down and stop before the pedestrian crossing, while the target state determined based on real-time driving status information is to accelerate, the driving decision result can be played out via voice to remind the driver to slow down and stop before the pedestrian crossing.

[0098] Optionally, after obtaining the driving decision result, the actual driving status information of the vehicle at each moment in the historical time period can be obtained; based on the actual driving status information at each moment and the expected state corresponding to the driving decision result at each moment, the driving behavior score corresponding to each moment can be determined; based on the driving behavior score corresponding to each moment, the driver's total driving behavior score in the historical time period can be determined.

[0099] Among them, the actual driving status information is the real driving status data of the vehicle within a historical time period.

[0100] Driving behavior score is a quantitative evaluation of how well a driver's driving behavior at a specific moment matches the expected state. Overall driving behavior score is an overall evaluation of a driver's driving behavior over the entire historical period, obtained by comprehensively considering the driving behavior scores at each moment within that period.

[0101] Specifically, the actual state of the vehicle at each moment can be determined based on the actual driving status information at each moment. Then, it is determined whether the actual state at each moment is the same as the expected state corresponding to the driving decision at each moment. If the actual state is the same as the expected state corresponding to the driving decision, the driving behavior score corresponding to that moment is determined as the first preset score; if the actual state is different from the expected state corresponding to the driving decision, the driving behavior score corresponding to that moment is determined as the second preset score, and the first preset score is greater than the second preset score. Then, the sum of the driving behavior scores corresponding to each moment within the historical time period is determined as the total driving behavior score, and this total driving behavior score is displayed to inform the driver of their driving performance within the historical time period. The first preset score and the second preset score are preset values ​​that can be adjusted according to actual conditions; for example, the first preset score is 1 and the second preset score is 0.

[0102] In this embodiment, an objective and accurate score can be given based on the degree to which the driver's operation complies with traffic rules, so that the driver can clearly quantify his driving performance, especially novice drivers, thereby promoting the continuous improvement of their driving skills.

[0103] Optionally, to help novice drivers learn and improve their driving skills more effectively, after a period of driving, the driver's driving behavior during this period can be compared with the corresponding driving decision results. Discrepancies between the driving behavior and the corresponding driving decision results can be recorded, and an analysis report can be generated based on the records. This report can accurately point out traffic conditions that the driver needs to pay special attention to and the correct operating methods to take, thereby helping the driver learn valuable experience from actual driving. Simultaneously, targeted suggestions can be generated based on the analysis report to help the driver clarify the direction for improvement and continuously enhance their driving skills.

[0104] The technical solution of this application embodiment, after determining the dynamic and static targets based on the target information, can annotate each frame of the image based on the position of the dynamic target in each frame to obtain a target frame image sequence, providing accurate position information input for the subsequent trajectory detection model. The target frame image sequence is then input into a pre-trained trajectory detection model to obtain the motion trajectory of each dynamic target within a historical time period. Through the pre-trained trajectory detection model, the positional changes of the dynamic target between different frames can be accurately tracked, especially when the vehicle is moving rapidly or the target is partially occluded, thereby improving the accuracy and efficiency of trajectory determination and providing an accurate data foundation for the subsequent driving decision prediction model. Alternatively, the initial position of each dynamic target can be determined based on the position of the dynamic target in each frame of the image, providing an accurate starting reference point for target tracking. Based on the initial position of each dynamic target, continuous tracking is performed to obtain the motion trajectory of each dynamic target within a historical time period. This can completely record the motion path of the dynamic target within a historical time period and improve the efficiency of target tracking, thereby improving the accuracy and efficiency of trajectory determination and providing an accurate data foundation for the subsequent driving decision prediction model.

[0105] After obtaining the vehicle's driving decision result for the next moment, the system can acquire the vehicle's real-time driving status information after the current moment and determine the vehicle's target state for the next moment based on the real-time driving status information. Then, when the target state is different from the expected state corresponding to the driving decision result, the system displays the driving decision result to remind the driver to drive the vehicle according to the driving decision result. By comparing the target state with the expected state, the system can promptly detect the difference between the vehicle's actual driving trend and the driving decision result, and promptly remind the driver to drive according to the driving decision result. This can help the driver better cope with complex traffic scenarios, thereby reducing the risk of traffic accidents and protecting the personal safety of the people in the vehicle and other traffic participants on the road.

[0106] Figure 3 This is a structural schematic diagram of a driving assistance device provided in an embodiment of this application, referring to... Figure 3 The driving assistance device may include:

[0107] The acquisition module 310 is used to acquire the video stream obtained by the camera capturing the surrounding environment of the vehicle, and to acquire the vehicle's driving status information at the current moment.

[0108] The target detection module 320 is used to input each frame of the video stream into the pre-trained target detection model to obtain the target information in each frame of the image, and to determine the dynamic and static targets based on the target information. The target information includes the category and location.

[0109] The determination module 330 is used to determine the static environment information of each frame image based on the category and position of the static target in each frame image, and to determine the motion trajectory of each dynamic target in the historical time period based on the position of the dynamic target in each frame image.

[0110] The decision prediction module 340 is used to input the motion trajectory of each dynamic target in the historical time period, the static environment information of each frame image and the driving state information into the pre-trained driving decision prediction model to obtain the driving decision result of the vehicle in the next moment, so as to assist the driver in driving the vehicle.

[0111] In one embodiment, the determining module 330 determines the motion trajectory of each dynamic target within a historical time period based on the position of the dynamic target in each frame image, including: labeling each frame image based on the position of the dynamic target in each frame image to obtain a target frame image sequence; and inputting the target frame image sequence into a pre-trained trajectory detection model to obtain the motion trajectory of each dynamic target within a historical time period.

[0112] In one embodiment, the determining module 330 determines the motion trajectory of each dynamic target within a historical time period based on the position of the dynamic target in each frame image, including: determining the initial position of each dynamic target based on the position of the dynamic target in each frame image; and continuously tracking the dynamic target based on the initial position of each dynamic target to obtain the motion trajectory of each dynamic target within a historical time period.

[0113] In one embodiment, the driving decision prediction model includes a feature extraction module and a driving decision module. The training process of the driving decision prediction model in the decision prediction module 340 is as follows: A sample dataset is acquired, including sample driving state information, sample static environment information, sample dynamic target trajectory, and driving decision result labels; the sample static environment information is input into the feature extraction module to obtain a target feature map sequence; the sample driving state information, sample dynamic target trajectory, and target feature map sequence are input into the driving decision module to obtain the driving decision prediction result; the loss value between the driving decision prediction result and the driving decision result label is calculated, and the backpropagation algorithm is used to optimize the feature extraction module and the driving decision module to minimize the loss value, thus obtaining the driving decision prediction model.

[0114] In one embodiment, the target detection module 320 determines dynamic targets and static targets based on target information, including: when the category is in a preset static category table, determining the target corresponding to the category as a static target; when the category is in a preset dynamic category table, determining the target corresponding to the category as a dynamic target.

[0115] In one embodiment, the driving assistance device further includes a display module, which is specifically used to: acquire real-time driving status information of the vehicle after the current moment; determine the target state of the vehicle at the next moment based on the real-time driving status information; and display the driving decision result when the target state is different from the expected state corresponding to the driving decision result, so as to remind the driver to drive the vehicle according to the driving decision result.

[0116] In one embodiment, the driving assistance device further includes a driving score module, which is specifically used to: acquire the actual driving status information of the vehicle at each moment in a historical time period; determine the driving behavior score corresponding to each moment based on the actual driving status information at each moment and the expected state corresponding to the driving decision result at each moment; and determine the driver's total driving behavior score in the historical time period based on the driving behavior score corresponding to each moment.

[0117] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is merely an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0118] The driving assistance device provided in this embodiment can be applied to the driving assistance method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0119] Figure 4 This is a structural schematic diagram of a vehicle provided in an embodiment of this application. Figure 4 A block diagram of an exemplary vehicle 11 suitable for implementing embodiments of this application is shown. Figure 4 The vehicle 11 shown is merely an example and should not impose any limitations on the functionality and scope of use of this embodiment.

[0120] like Figure 4 As shown, vehicle 11 is represented in the form of a general-purpose computing electronic device. Components of vehicle 11 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0121] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0122] Vehicle 11 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by vehicle 11, including volatile and non-volatile media, removable and non-removable media.

[0123] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Vehicle 11 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 As not shown, disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disc drives for reading and writing to removable non-volatile optical discs (e.g., CD-ROMs, DVD-ROMs, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0124] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this application.

[0125] Vehicle 11 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with vehicle 11, and / or with any device that enables vehicle 11 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, vehicle 11 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20.

[0126] like Figure 4 As shown, network adapter 20 communicates with other modules of vehicle 11 via bus 18. It should be understood that, although... Figure 4 As not shown, other hardware and / or software modules may be used in conjunction with vehicle 11, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0127] The processing unit 16 executes various functional applications and page displays by running programs stored in the system memory 28, such as implementing a driving assistance method provided in any embodiment of this application.

[0128] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a driving assistance method, such as that provided in any embodiment of this application.

[0129] The computer storage medium of this embodiment can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0130] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0131] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0132] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0133] Those skilled in the art will understand that the modules or steps described above in this application can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0134] Furthermore, the acquisition, storage, use, and processing of data in this application's technical solution all comply with relevant national laws and regulations.

[0135] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the inventive concept of this application, and the scope of this application is determined by the scope of the appended claims.

Claims

1. A driving assistance method, characterized in that, Applied to a vehicle equipped with a camera, the method includes: The system acquires video streams of the vehicle's surrounding environment captured by the camera and obtains the vehicle's driving status information at the current moment. Each frame of the video stream is input into a pre-trained target detection model to obtain target information in each frame. Dynamic and static targets are then determined based on the target information, which includes category and location. The static environment information of each frame of the image is determined based on the category and position of the static target in each frame of the image, and the motion trajectory of each dynamic target in the historical time period is determined based on the position of the dynamic target in each frame of the image. The motion trajectory of each dynamic target within a historical time period, the static environment information of each frame of image, and the driving state information are input into a pre-trained driving decision prediction model to obtain the driving decision result of the vehicle at the next moment, so as to assist the driver in driving the vehicle. The driving decision prediction model includes a feature extraction module and a driving decision module. The training process of the driving decision prediction model is as follows: A sample dataset is acquired, which includes sample driving state information, sample static environment information, sample dynamic target trajectory, and driving decision result labels. The sample static environment information is input into the feature extraction module to obtain a target feature map sequence. The sample driving state information, sample dynamic target trajectory, and target feature map sequence are input into the driving decision module to obtain the driving decision prediction result. The loss value between the driving decision prediction result and the driving decision result label is calculated, and the backpropagation algorithm is used to optimize the feature extraction module and the driving decision module to minimize the loss value, thus obtaining the driving decision prediction model.

2. The driving assistance method according to claim 1, characterized in that, Determining the motion trajectory of each dynamic target within a historical time period based on the position of the dynamic target in each frame image includes: Each frame of the image is labeled based on the position of the dynamic target in each frame of the image to obtain a sequence of target frame images; The target frame image sequence is input into a pre-trained trajectory detection model to obtain the motion trajectory of each dynamic target within a historical time period.

3. The driving assistance method according to claim 1, characterized in that, Determining the motion trajectory of each dynamic target within a historical time period based on the position of the dynamic target in each frame image includes: The initial position of each dynamic target is determined based on the position of the dynamic target in each frame of the image; Based on the initial position of each dynamic target, the dynamic targets are continuously tracked to obtain the motion trajectory of each dynamic target within a historical time period.

4. The driving assistance method according to claim 1, characterized in that, The step of determining dynamic and static targets based on the target information includes: When a category is within a preset static category table, the target corresponding to that category is determined to be a static target; When a category is within a preset dynamic category table, the target corresponding to that category is determined to be a dynamic target.

5. The driving assistance method according to claim 1, characterized in that, The method further includes: Obtain the real-time driving status information of the vehicle after the current moment; The target state of the vehicle at the next moment is determined based on the real-time driving status information; When the target state is different from the expected state corresponding to the driving decision result, the driving decision result is displayed to remind the driver to drive the vehicle according to the driving decision result.

6. The driving assistance method according to claim 1, characterized in that, The method further includes: Obtain the actual driving status information of the vehicle at each moment within the historical time period; Based on the actual driving status information at each moment and the expected state corresponding to the driving decision result at each moment, the driving behavior score corresponding to each moment is determined; The driver's total driving behavior score for the historical time period is determined based on the driving behavior score corresponding to each moment.

7. A driving assistance device, characterized in that, Applied to a vehicle, wherein the vehicle is equipped with a camera, the device includes: The acquisition module is used to acquire the video stream obtained by the camera capturing the surrounding environment of the vehicle, and to acquire the driving status information of the vehicle at the current moment. The target detection module is used to input each frame of the video stream into a pre-trained target detection model to obtain target information in each frame of the video stream, and to determine dynamic targets and static targets based on the target information. The target information includes category and location. The determination module is used to determine the static environment information of each frame image based on the category and position of the static target in each frame image, and to determine the motion trajectory of each dynamic target in the historical time period based on the position of the dynamic target in each frame image. The decision prediction module is used to input the motion trajectory of each dynamic target in the historical time period, the static environment information of each frame image and the driving state information into the pre-trained driving decision prediction model to obtain the driving decision result of the vehicle at the next moment, so as to assist the driver in driving the vehicle. The driving decision prediction model includes a feature extraction module and a driving decision module. The training process of the driving decision prediction model is as follows: A sample dataset is acquired, which includes sample driving state information, sample static environment information, sample dynamic target trajectory, and driving decision result labels. The sample static environment information is input into the feature extraction module to obtain a target feature map sequence. The sample driving state information, sample dynamic target trajectory, and target feature map sequence are input into the driving decision module to obtain the driving decision prediction result. The loss value between the driving decision prediction result and the driving decision result label is calculated, and the backpropagation algorithm is used to optimize the feature extraction module and the driving decision module to minimize the loss value, thus obtaining the driving decision prediction model.

8. A vehicle, characterized in that, The vehicles include: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the driving assistance method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the driving assistance method as described in any one of claims 1 to 6.

Images