A driver lane-changing intention recognition method, device, equipment and medium

Abstract

The present application relates to a kind of driver's lane change intention identification method, device, equipment and medium, disclose the driving image set of driver in current time period is obtained, wherein the driving image set includes multiple frames of continuous driving image;Detect the first driving behavior feature corresponding to driver in each frame driving image, and based on the first driving behavior feature, the first lane change intention corresponding to driving image is identified;Analysis driver's first lane change intention in each frame driving image, determine the second lane change intention of driver in current time period.This application is by identifying the lane change intention of continuous multiple frames of driving image, and the lane change intention of multiple frames of driving image is comprehensively analyzed, obtains the final lane change intention, to this by using continuous multiple frames of driving image for analysis, it can avoid the contingency of using single frame image identification intention, realizes the accurate identification of lane change intention.Can be accurately identified in the case where steering lamp is not opened driver lane change intention.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, specifically to a method, device, equipment, and medium for recognizing a driver's lane change intention. Background Technology

[0002] Currently, in Advanced Driving Assistance Systems (ADAS), lane assist and side assist functions often issue unnecessary warnings and "take over the steering wheel" from the driver. The root cause is that the current system cannot accurately identify the driver's lane change intention when the turn signal is not activated. This means that the system cannot truly trigger the function only when the vehicle deviates unintentionally in a dangerous situation, and it cannot correctly issue warnings in dangerous situations where the turn signal is not activated. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a method, apparatus, device and medium for recognizing a driver's lane change intention, in order to solve the problem that driving assistance systems cannot accurately recognize a driver's lane change intention when the turn signal is not activated.

[0004] In a first aspect, embodiments of the present invention provide a method for recognizing a driver's lane-changing intention, the method comprising:

[0005] Acquire a set of driving images of the driver within the current time period, wherein the set of driving images includes multiple consecutive frames of driving images, and the driving images are images of the driver performing driving operations inside the target vehicle;

[0006] Detect the first driving behavior feature corresponding to the driver in each frame of driving image, and based on the first driving behavior feature, identify the first lane change intention corresponding to the driving image;

[0007] Analyze the driver's first lane change intention in each frame of the driving image to determine the driver's second lane change intention in the current time period.

[0008] The method provided in this application identifies lane-changing intentions from multiple consecutive frames of driving images and performs comprehensive analysis to obtain the final lane-changing intention. By using multiple consecutive frames of driving images for analysis, the method avoids the randomness of intention recognition using a single frame image, achieving accurate recognition of lane-changing intentions. It can accurately identify the driver's lane-changing intention even without the turn signal being activated, and can only trigger the corresponding function in dangerous situations where the vehicle unintentionally deviates from its lane. It can also issue an alarm even without the turn signal being activated.

[0009] Furthermore, the step of detecting the first driving behavior feature corresponding to the driver in each frame of the driving image, and identifying the first lane change intention corresponding to the driving image based on the first driving behavior feature, includes:

[0010] The driving image is input into a pre-trained recognition model, wherein the recognition model includes a first processing network and a first feature extraction network;

[0011] The driving image is divided into multiple grids by the first processing network, a first grid is determined from the multiple grids, and the first grid is transmitted to the feature extraction network, wherein the first grid is used to predict the first position of the driver in the driving image;

[0012] The first feature extraction network predicts the driver's first position in the driving image based on the first grid, extracts the first driving behavior feature based on the first position, and determines the probability value of the first driving behavior feature belonging to different lane change categories.

[0013] The first lane change intention corresponding to the driving image is generated based on the lane change category and the probability value corresponding to the lane change category.

[0014] The method provided in the embodiments of this application

[0015] Furthermore, the step of dividing the driving image into multiple grids through the first processing network, determining a first grid from the multiple grids, and transmitting the first grid to the feature extraction network includes:

[0016] The driving image is divided into multiple grids using the first processing network;

[0017] Detect the target grid that matches the center of the driver from multiple grids;

[0018] The target grid is determined as a first grid for predicting the driver's first position in the driving image, and the first grid is transmitted to the feature extraction network.

[0019] The method provided in this application divides a driving image into multiple grids and uses the driver's center point to determine the first grid for predicting the driver's first position in the driving image. This facilitates the rapid prediction of the driver's position using the grid, which is beneficial for extracting the driver's driving behavior features based on that position. Furthermore, by restricting the task of predicting the driver's position to this grid, the model can more effectively learn the driver's center position, improving the accuracy of localization.

[0020] Furthermore, the step of predicting the driver's first position in the driving image based on the first grid using the first feature extraction network, extracting a first driving behavior feature based on the first position, and determining the probability value of the first driving behavior feature belonging to different lane change categories includes:

[0021] The first feature extraction network uses the first grid to predict multiple bounding boxes corresponding to the driver and determines the first conditional probability corresponding to the bounding boxes, wherein the bounding boxes are used to characterize the driver's first position in the driving image;

[0022] Extract the first driving behavior feature within the bounding box, and predict the second conditional probability that the first driving behavior feature belongs to different lane change categories;

[0023] Based on the joint calculation of the first conditional probability and the second conditional probability, the probability values ​​of the first driving behavior feature belonging to different lane change categories are obtained.

[0024] The method provided in this application, by jointly calculating the first conditional probability of the bounding box and the second case probability of the first driving behavior, can accurately determine the probability value of the lane change category corresponding to the driving behavior features in each frame of driving image. This provides a reliable basis for subsequent analysis of the driver's lane change intention by combining multiple images.

[0025] Furthermore, the training method for the recognition model includes:

[0026] Acquire sample images, wherein the sample images include second driving behavior features triggered by the driver performing a lane change operation;

[0027] Obtain the label information corresponding to the sample image, wherein the label information is used to mark the actual position of the driver in the sample image and the actual lane change category corresponding to the second driving behavior feature;

[0028] The recognition model is obtained by training a preset neural network model using the sample images and the label information.

[0029] Furthermore, the step of training a preset neural network model using the sample images and the label information to obtain the recognition model includes:

[0030] The sample image and the label information are input into the preset neural network model, wherein the preset neural network model includes a second processing network and a second feature extraction network;

[0031] The sample image is divided into multiple grids by the second processing network, a second grid is determined from the multiple grids, and the second grid is transmitted to the second feature extraction network, wherein the second grid is used to predict the second position of the driver in the sample image;

[0032] The second feature extraction network predicts the driver's second position in the driving image based on the second grid, extracts second driving behavior features based on the second position, and determines the initial probability value of the second driving behavior features belonging to different lane change categories.

[0033] The classification error is calculated based on the initial probability value of the lane change category and the actual lane change category;

[0034] The prediction error and cross-union ratio error are calculated based on the second position and the actual position.

[0035] The target loss is calculated based on the classification error, the prediction error, and the intersection-union ratio error.

[0036] The preset neural network model is optimized using the target loss, and the optimized neural network model is used as the recognition model.

[0037] The method provided in this application uses the driver's historical driving images for training, which can accurately capture the driver's driving behavior characteristics during lane changing operations, such as specific patterns and range of motion during lane changing. This helps the model to learn the driver's driving habits in depth, thereby accurately identifying the driver's lane changing intention.

[0038] Furthermore, the analysis of the driver's first lane-changing intention in each frame of the driving image to determine the driver's second lane-changing intention in the current time period includes:

[0039] The lane change category with the highest probability value in the first lane change intention is selected as the candidate lane change category, and the number of the candidate lane change categories is determined.

[0040] The candidate lane change category with the largest number is determined as the second lane change intention.

[0041] Secondly, embodiments of the present invention provide a driver's lane change intention recognition device, the device comprising:

[0042] The acquisition module is used to acquire a set of driving images of the driver within the current time period, wherein the set of driving images includes multiple consecutive frames of driving images, and the driving images are images of the driver performing driving operations inside the target vehicle;

[0043] The detection module is used to detect the first driving behavior feature corresponding to the driver in each frame of driving image, and based on the first driving behavior feature, identify the first lane change intention corresponding to the driving image;

[0044] The analysis module is used to analyze the driver's first lane change intention in each frame of driving image and determine the driver's second lane change intention in the current time period.

[0045] Thirdly, embodiments of the present invention provide a computer device, including: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0046] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that cause a computer to perform the method described in the first aspect or any of its corresponding embodiments.

[0047] The embodiments of this application have the following beneficial effects:

[0048] (1) The method provided in this application identifies the lane-changing intention of multiple consecutive frames of driving images and performs comprehensive analysis on the lane-changing intention of multiple frames of driving images to obtain the final lane-changing intention. By using multiple consecutive frames of driving images for analysis, the randomness of identifying intentions using single-frame images can be avoided, and accurate identification of lane-changing intentions can be achieved. It can accurately identify the driver's lane-changing intention even when the turn signal is not turned on, and can trigger the corresponding function only when the vehicle deviates unintentionally in a dangerous situation. At the same time, it can also issue an alarm even when the turn signal is not turned on.

[0049] (2) The method provided in this application divides the driving image into multiple grids and uses the driver's center point to determine the first grid for predicting the driver's first position in the driving image. This facilitates the rapid prediction of the driver's position through the grid, which is beneficial for extracting the driver's driving behavior features based on that position. At the same time, by restricting the task of predicting the driver's position to this grid, the model can learn the driver's center position more focusedly, improving the accuracy of localization.

[0050] (3) The method provided in this application, by jointly calculating the first conditional probability of the bounding box and the second case probability of the first driving behavior, can accurately determine the probability value of the lane change category corresponding to the driving behavior features in each frame of driving image. This provides a reliable basis for subsequent analysis of the driver's lane change intention by combining multiple images.

[0051] (4) The method provided in this application uses the driver’s historical driving images for training, which can accurately capture the driver’s driving behavior characteristics when changing lanes, such as the specific patterns and range of motion when changing lanes, so that the model can learn the driver’s driving habits in depth and thus accurately identify the driver’s lane changing intention. Attached Figure Description

[0052] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating a method for a driver's lane-changing intention according to some embodiments of the present invention;

[0054] Figure 2 This is a schematic diagram of an identification model according to some embodiments of the present invention;

[0055] Figure 3 This is a schematic diagram of a driving image segmented according to some embodiments of the present invention;

[0056] Figure 4 This is a schematic diagram of the processing procedure for driving images according to some embodiments of the present invention;

[0057] Figure 5 This is a flowchart illustrating a training method for a recognition model according to some embodiments of the present invention;

[0058] Figure 6 This is a structural block diagram of a driver's lane-changing intention device according to an embodiment of the present invention;

[0059] Figure 7 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] According to an embodiment of the present invention, a method for recognizing a driver's lane change intention is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0062] This embodiment provides a method, apparatus, device, and medium for recognizing a driver's lane change intention. Figure 1 This is a flowchart of a driver's lane change intention recognition method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0063] Step S11: Obtain a set of driving images of the driver within the current time period. The set of driving images includes multiple consecutive frames of driving images, which are images of the driver performing driving operations inside the target vehicle.

[0064] The method provided in this application is applied to the control terminal of a target vehicle. The control terminal can send acquisition commands to the camera device deployed in the target vehicle in real time, or it can send acquisition commands even when the target vehicle's turn signals are not activated. This facilitates the accurate identification of the driver's lane-changing intention by the acquired driving images when the target vehicle's turn signals are not activated. The camera device acquires images of the driver in the target vehicle within the current time period according to the acquisition command, obtaining a driving image set, which includes multiple consecutive frames of driving images. The camera device transmits the acquired driving image set to the control terminal, facilitating the control terminal's subsequent identification of the driver's lane-changing intention.

[0065] Step S12: Detect the first driving behavior feature corresponding to the driver in each frame of driving image, and identify the first lane change intention corresponding to the driving image based on the first driving behavior feature.

[0066] In this embodiment of the application, the first driving behavior feature corresponding to the driver in each frame of driving image is detected, and the first lane change intention corresponding to the driving image is identified based on the first driving behavior feature, including the following steps A1-A4:

[0067] Step A1: Input the driving image into a pre-trained recognition model, wherein the recognition model includes a first processing network and a first feature extraction network.

[0068] In this embodiment, a driving image is input to a pre-trained recognition model, which includes a first processing network and a first feature extraction network. The recognition model in this application can be a YOLO neural network model, with the following structure: Figure 2As shown in the diagram. The first feature extraction network uses MobileNet, which accelerates image detection while increasing network complexity through a more efficient convolutional computation method. A 3x3 convolutional kernel is used, with zero-padding width and stride both set to 1, ensuring that the output feature map size of a single convolutional layer is the same as the input image. A 2x2 max-pooling layer is used for downsampling, progressively aggregating lower-level image information to higher-level structural information. A leaky ReLU function is used as the activation function between each layer, which, compared to the ReLU function, increases the smoothness of the activation function and prevents gradient vanishing for negative outputs.

[0069] Step A2: The driving image is divided into multiple grids by the first processing network, the first grid is determined from the multiple grids, and the first grid is transmitted to the feature extraction network, wherein the first grid is used to predict the first position of the driver in the driving image.

[0070] In this embodiment of the application, the driving image is divided into multiple grids by a first processing network, a first grid is determined from the multiple grids, and the first grid is transmitted to the feature extraction network, including the following steps A201-A203:

[0071] Step A201: The driving image is divided into multiple grids using the first processing network.

[0072] In this embodiment of the application, the first processing network divides the driving image into 7×7 grids, as shown in the following figure. Figure 3 As shown.

[0073] Step A202: Detect the target grid that matches the driver's center from multiple grids.

[0074] In this embodiment of the application, the center point coordinates of the driver in the driving image are first detected, and the center point coordinates are matched with the coordinate range corresponding to each grid. The grid that is successfully matched is determined as the target grid.

[0075] Step A203: The target grid is determined as the first grid for predicting the driver's first position in the driving image, and the first grid is transmitted to the feature extraction network.

[0076] In this embodiment, after obtaining the target mesh, the target mesh is determined as the mesh used to predict the driver's first position in the driving image. The first position can be understood as the position of the bounding box corresponding to the driver. Then, the first mesh is transmitted to the feature extraction network.

[0077] It should be noted that since the driver's center position is often the most important and prominent part, by limiting the task of predicting the driver's position to this grid, the model can learn the driver's center position more focusedly, thus improving the accuracy of positioning.

[0078] Step A3: The first feature extraction network predicts the driver's first position in the driving image based on the first grid, extracts the first driving behavior feature based on the first position, and determines the probability value of the first driving behavior feature belonging to different lane change categories.

[0079] The method provided in this application divides a driving image into multiple grids and uses the driver's center point to determine the first grid for predicting the driver's first position in the driving image. This facilitates the rapid prediction of the driver's position through the grids and is beneficial for extracting the driver's driving behavior features based on that position.

[0080] In this embodiment of the application, a first feature extraction network predicts the driver's first position in the driving image based on a first grid, extracts a first driving behavior feature based on the first position, and determines the probability value of the first driving behavior feature belonging to different lane change categories, including the following steps A301-A303:

[0081] Step A301: The first feature extraction network uses the first grid to predict multiple bounding boxes corresponding to the driver and determines the first conditional probability corresponding to the bounding boxes, wherein the bounding boxes are used to represent the first position of the driver in the driving image.

[0082] In this embodiment, the first feature extraction network predicts the bounding box corresponding to the driver based on the first grid. Each bounding box contains 5 elements (x, y, w, h, confidence), where x and y represent the coordinates of the bounding box, w and h represent the width and height of the bounding box, respectively, and confidence represents the confidence level. In this embodiment, the confidence level is used as the first conditional probability of the bounding box. The first conditional probability is used to reflect whether there is a driver in the bounding box and the accuracy of the driver's position.

[0083] Step A302: Extract the first driving behavior feature within the bounding box and predict the second conditional probability that the first driving behavior feature belongs to different lane change categories.

[0084] In this embodiment of the application, determining the first conditional probability corresponding to the bounding box can be as follows: the first feature extraction network extracts the first driving behavior features from the bounding box, classifies the first driving behavior features, obtains the category scores of each lane change category corresponding to the first driving behavior, and finally converts the category scores into a second conditional probability through the softmax function.

[0085] It should be noted that driving behavior characteristics include facial features, head features, and hand features, etc.

[0086] Lane change categories include: no lane change, left lane change, and right lane change. Different combinations of features correspond to different lane change categories.

[0087] Step A303: Based on the first conditional probability and the second conditional probability, a joint calculation is performed to obtain the probability value of the first driving behavior feature belonging to different lane change categories.

[0088] The method provided in this application, by jointly calculating the first conditional probability of the bounding box and the second case probability of the first driving behavior, can accurately determine the probability value of the lane change category corresponding to the driving behavior features in each frame of driving image. This provides a reliable basis for subsequent analysis of the driver's lane change intention by combining multiple images.

[0089] Step A4: Generate the first lane change intention corresponding to the driving image based on the lane change category and the probability value corresponding to the lane change category.

[0090] In this embodiment of the application, each frame of the driving image in the driving image set is input into the recognition model to obtain the lane change category and the probability value corresponding to the lane change category for each frame of the driving image. Finally, the lane change category and the probability value corresponding to the lane change are used as the first lane change intention of the driving image.

[0091] The method provided in this application utilizes a recognition model to determine a first grid from multiple grids to predict the driver's first position in a driving image. This facilitates subsequent prediction of the driver's position using the first grid, allowing the model to better learn visual features and contextual information. This enhances the model's recognition ability, more accurately distinguishes target features from the background, and thus improves detection accuracy.

[0092] Step S13: Analyze the driver's first lane change intention in each frame of driving image to determine the driver's second lane change intention in the current time period.

[0093] In this embodiment of the application, analyzing the driver's first lane change intention in each frame of driving image and determining the driver's second lane change intention in the current time period includes: taking the lane change category with the highest probability value in the first lane change intention as the candidate lane change category and determining the number of candidate lane change categories; and determining the candidate lane change category with the largest number as the second lane change intention.

[0094] In the embodiments of this application, such as Figure 4As shown, the driving image set includes multiple driving images, namely driving image p1, driving image p2, driving image p3, and driving image p4. These driving images are sequentially input into the recognition model, which then sequentially outputs the lane change intention q1 corresponding to driving image p1, the lane change intention q2 corresponding to driving image p2, the lane change intention q3 corresponding to driving image p3, and the lane change intention q4 corresponding to driving image p4. For example, the lane change category with the highest probability value in lane change intention q1 is no lane change; the lane change category with the highest probability value in lane change intention q2 is left lane change; the lane change category with the highest probability value in lane change intention q3 is left lane change; and the lane change category with the highest probability value in lane change intention q4 is left lane change. Since the number of lane change categories corresponding to left lane change is 3, left lane change is determined as the second lane change intention.

[0095] The method provided in this application identifies lane-changing intentions from multiple consecutive frames of driving images and performs comprehensive analysis to obtain the final lane-changing intention. By using multiple consecutive frames of driving images for analysis, the method avoids the randomness of intention recognition using a single frame image, achieving accurate recognition of lane-changing intentions. It can accurately identify the driver's lane-changing intention even without the turn signal being activated, and can only trigger the corresponding function in dangerous situations where the vehicle unintentionally deviates from its lane. It can also issue an alarm even without the turn signal being activated.

[0096] In the embodiments of this application, such as Figure 5 As shown, the training method for the recognition model includes the following steps:

[0097] Step S21: Obtain a sample image, wherein the sample image includes second driving behavior features triggered by the driver performing a lane change operation.

[0098] In this embodiment, the driver's historical driving record can be obtained, and historical driving images can be extracted from the historical driving record. The historical driving images are then filtered to select target driving images related to lane changing operations. First, the target driving images are scaled to 224x224 pixels. In addition, considering the diversity of human actions and expressions, in order to make the training data more reasonable, this embodiment improves the network's generalization ability by using a flip matrix to flip the images horizontally and vertically, perform affine transformations at different scales, and apply Gaussian blur to the dataset images to obtain sample images.

[0099] Step S22: Obtain the label information corresponding to the sample image, wherein the label information is used to mark the actual position of the driver in the sample image and the actual lane change category corresponding to the second driving behavior feature.

[0100] In this embodiment, the label information can be pre-annotated using image annotation software. The label information is used to annotate the driver's actual position in the sample image, and also to annotate the actual lane change category corresponding to the second driving behavior feature in the sample image. The label information is further used to annotate the second driving behavior feature in the sample image and the actual lane change category corresponding to the second driving behavior feature. For example, the second driving behavior feature can be annotated using anchor boxes, and the actual lane change category corresponding to the anchor box can be added. Actual lane change categories include: no lane change, left lane change, and right lane change.

[0101] Step S23: Train a preset neural network model using sample images and label information to obtain a recognition model.

[0102] In this embodiment of the application, a preset neural network model is trained using sample images and label information to obtain a recognition model, including the following process:

[0103] (1) Input the sample image and label information into the preset neural network model, wherein the preset neural network model includes a second processing network and a second feature extraction network.

[0104] (2) The sample image is divided into multiple grids by the second processing network, the second grid is determined from the multiple grids, and the second grid is transmitted to the second feature extraction network, wherein the second grid is used to predict the second position of the driver in the sample image.

[0105] (3) The second feature extraction network predicts the driver’s second position in the driving image based on the second grid, extracts the second driving behavior features based on the second position, and determines the initial probability value of the second driving behavior features belonging to different lane change categories.

[0106] (4) Calculate the classification error based on the initial probability value of the lane change category and the actual lane change category.

[0107] In this embodiment of the application, the calculation process for classification error is as follows:

[0108]

[0109] Wherein, classError classification error, λ noobj The weights of the classification loss function are s, the standard deviation is B, the number of bounding boxes is p, the second conditional probability is C, the confidence level is i.e. the first conditional probability, and i and j represent the horizontal and vertical indices of the grid.

[0110] (5) Calculate the prediction error and crossover ratio error based on the second position and the actual position.

[0111] In this embodiment of the application, the calculation process for the prediction error is as follows:

[0112]

[0113] Where coordError is the prediction error, λ coord denoted by s, where s is the weight of the loss function, s is the standard deviation, B is the number of bounding boxes, p represents the second conditional probability, C is the confidence level (i.e., the first conditional probability), and x and y are the coordinates of the bounding boxes.

[0114] In this embodiment of the application, the calculation process for the crossover-union ratio error is as follows:

[0115]

[0116] Where iouError is the cross-union ratio error, λ coord s is the weight of the loss function, s is the standard deviation, B is the number of bounding boxes, p represents the second conditional probability, C is the confidence level, i.e. the first conditional probability, w is the width of the bounding box, and h is the height of the bounding box.

[0117] (6) Calculate the target loss based on the classification error, prediction error and crossover ratio error.

[0118] In the embodiments of this application,

[0119] (7) Optimize the preset neural network model using the target loss, and use the optimized neural network model as the recognition model.

[0120] In this embodiment, the network parameters of the processing network and feature processing network in the preset neural network model are optimized and adjusted using the target loss until the target loss of the adjusted preset neural network model is less than a preset threshold. Then, the training is determined to be complete, and the trained preset neural network model is used as the recognition model.

[0121] The method provided in this application uses the driver's historical driving images for training, which can accurately capture the driver's driving behavior characteristics during lane changing operations, such as specific patterns and range of motion during lane changing. This helps the model to learn the driver's driving habits in depth, thereby accurately identifying the driver's lane changing intention.

[0122] This embodiment also provides a driver lane change intention recognition device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0123] This embodiment provides a driver's lane change intention recognition device, such as... Figure 6 As shown, it includes:

[0124] The acquisition module 61 is used to acquire a set of driving images of the driver in the current time period, wherein the set of driving images includes multiple consecutive driving images, and the driving images are images of the driver performing driving operations in the target vehicle;

[0125] The detection module 62 is used to detect the first driving behavior feature corresponding to the driver in each frame of driving image, and based on the first driving behavior feature, identify the first lane change intention corresponding to the driving image.

[0126] Analysis module 63 is used to analyze the driver's first lane change intention in each frame of driving image and determine the driver's second lane change intention in the current time period.

[0127] In this embodiment of the application, the detection module 62 includes:

[0128] An input unit is used to input a driving image into a pre-trained recognition model, wherein the recognition model includes a first processing network and a first feature extraction network;

[0129] The first processing unit is configured to divide the driving image into multiple grids through a first processing network, determine the first grid from the multiple grids, and transmit the first grid to the feature extraction network, wherein the first grid is used to predict the first position of the driver in the driving image;

[0130] The second processing unit is used to predict the driver's first position in the driving image based on the first grid through the first feature extraction network, extract the first driving behavior feature based on the first position, and determine the probability value of the first driving behavior feature belonging to different lane change categories.

[0131] The generation unit is used to generate the first lane change intention corresponding to the driving image based on the lane change category and the probability value corresponding to the lane change category.

[0132] In this embodiment of the application, the first processing unit is configured to divide a driving image into multiple grids through a first processing network; detect a target grid that matches the center of the driver from the multiple grids; determine the target grid as a first grid for predicting the first position of the driver in the driving image; and transmit the first grid to a feature extraction network.

[0133] In this embodiment of the application, the second processing unit is configured to predict multiple bounding boxes corresponding to the driver using a first feature extraction network and a first grid, and determine a first conditional probability corresponding to the bounding box, wherein the bounding box is used to represent the first position of the driver in the driving image; extract a first driving behavior feature within the bounding box, and predict a second conditional probability that the first driving behavior feature belongs to different lane change categories; and perform joint calculation based on the first conditional probability and the second conditional probability to obtain the probability value that the first driving behavior feature belongs to different lane change categories.

[0134] In this embodiment of the application, the device further includes: a training module, used to acquire sample images, wherein the sample images include second driving behavior features triggered by the driver performing a lane change operation; acquire label information corresponding to the sample images, wherein the label information is used to mark the actual position of the driver in the sample images and the actual lane change category corresponding to the second driving behavior features; and train a preset neural network model using the sample images and label information to obtain a recognition model.

[0135] In this embodiment, the training module is used to input sample images and label information into a preset neural network model, wherein the preset neural network model includes a second processing network and a second feature extraction network; the second processing network divides the sample image into multiple grids, determines a second grid from the multiple grids, and transmits the second grid to the second feature extraction network, wherein the second grid is used to predict the second position of the driver in the sample image; the second feature extraction network predicts the second position of the driver in the driving image based on the second grid, extracts second driving behavior features based on the second position, and determines the initial probability values ​​of the second driving behavior features belonging to different lane change categories; the classification error is calculated based on the initial probability values ​​of the lane change categories and the actual lane change categories; the prediction error and cross-union ratio (CUP) error are calculated based on the second position and the actual position; the target loss is calculated based on the classification error, prediction error, and CUP error; the preset neural network model is optimized using the target loss, and the optimized neural network model is used as the recognition model.

[0136] In this embodiment of the application, the analysis module 63 is used to select the lane change category with the highest probability value in the first lane change intention as the candidate lane change category and determine the number of candidate lane change categories; and to determine the candidate lane change category with the largest number as the second lane change intention.

[0137] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 7As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system).

[0138] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0139] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0140] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transient memory, such as at least one disk storage device, flash memory device, or other non-transient solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0141] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0142] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0143] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0144] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for recognizing a driver's lane-changing intention, characterized in that, The method includes: Acquire a set of driving images of the driver within the current time period, wherein the set of driving images includes multiple consecutive frames of driving images, and the driving images are images of the driver performing driving operations inside the target vehicle; Detect the first driving behavior feature corresponding to the driver in each frame of driving image, and based on the first driving behavior feature, identify the first lane change intention corresponding to the driving image; Analyze the driver's first lane change intention in each frame of the driving image to determine the driver's second lane change intention in the current time period; The step of detecting the first driving behavior feature corresponding to the driver in each frame of the driving image, and identifying the first lane change intention corresponding to the driving image based on the first driving behavior feature, includes: The driving image is input into a pre-trained recognition model, wherein the recognition model includes a first processing network and a first feature extraction network; The driving image is divided into multiple grids by the first processing network, a first grid is determined from the multiple grids, and the first grid is transmitted to the feature extraction network, wherein the first grid is used to predict the first position of the driver in the driving image; The first feature extraction network predicts the driver's first position in the driving image based on the first grid, extracts the first driving behavior feature based on the first position, and determines the probability value of the first driving behavior feature belonging to different lane change categories. The first lane change intention corresponding to the driving image is generated based on the lane change category and the probability value corresponding to the lane change category.

2. The method according to claim 1, characterized in that, The step of dividing the driving image into multiple grids through the first processing network, determining a first grid from the multiple grids, and transmitting the first grid to the feature extraction network includes: The driving image is divided into multiple grids using the first processing network; Detect the target grid that matches the center of the driver from multiple grids; The target grid is determined as a first grid for predicting the driver's first position in the driving image, and the first grid is transmitted to the feature extraction network.

3. The method according to claim 2, characterized in that, The step of predicting the driver's first position in the driving image based on the first grid using the first feature extraction network, extracting a first driving behavior feature based on the first position, and determining the probability value of the first driving behavior feature belonging to different lane change categories includes: The first feature extraction network uses the first grid to predict multiple bounding boxes corresponding to the driver and determines the first conditional probability corresponding to the bounding box, wherein the bounding box is used to characterize the driver's first position in the driving image; Extract the first driving behavior feature within the bounding box, and predict the second conditional probability that the first driving behavior feature belongs to different lane change categories; Based on the joint calculation of the first conditional probability and the second conditional probability, the probability values ​​of the first driving behavior feature belonging to different lane change categories are obtained.

4. The method according to claim 1, characterized in that, The training method for the recognition model includes: Acquire sample images, wherein the sample images include second driving behavior features triggered by the driver performing a lane change operation; Obtain the label information corresponding to the sample image, wherein the label information is used to mark the actual position of the driver in the sample image and the actual lane change category corresponding to the second driving behavior feature; The recognition model is obtained by training a preset neural network model using the sample images and the label information.

5. The method according to claim 4, characterized in that, The step of training a preset neural network model using the sample images and the label information to obtain the recognition model includes: The sample image and the label information are input into the preset neural network model, wherein the preset neural network model includes a second processing network and a second feature extraction network; The sample image is divided into multiple grids by the second processing network, a second grid is determined from the multiple grids, and the second grid is transmitted to the second feature extraction network, wherein the second grid is used to predict the second position of the driver in the sample image; The second feature extraction network predicts the driver's second position in the driving image based on the second grid, extracts second driving behavior features based on the second position, and determines the initial probability value of the second driving behavior features belonging to different lane change categories. The classification error is calculated based on the initial probability value of the lane change category and the actual lane change category; Calculate the prediction error and the cross-union ratio error based on the second position and the actual position; The target loss is calculated based on the classification error, the prediction error, and the intersection-union ratio error. The preset neural network model is optimized using the target loss, and the optimized neural network model is used as the recognition model.

6. The method according to claim 1, characterized in that, The analysis of the driver's first lane-changing intention in each frame of driving image to determine the driver's second lane-changing intention within the current time period includes: The lane change category with the highest probability value in the first lane change intention is selected as the candidate lane change category, and the number of the candidate lane change categories is determined. The candidate lane change category with the largest number is determined as the second lane change intention.

7. A driver's lane change intention recognition device, characterized in that, The device includes: The acquisition module is used to acquire a set of driving images of the driver within the current time period, wherein the set of driving images includes multiple consecutive frames of driving images, and the driving images are images of the driver performing driving operations inside the target vehicle; The detection module is used to detect the first driving behavior feature corresponding to the driver in each frame of driving image, and based on the first driving behavior feature, identify the first lane change intention corresponding to the driving image; The analysis module is used to analyze the driver's first lane change intention in each frame of driving image and determine the driver's second lane change intention in the current time period; The detection module includes: An input unit is used to input the driving image into a pre-trained recognition model, wherein the recognition model includes a first processing network and a first feature extraction network; The first processing unit is configured to divide the driving image into multiple grids through the first processing network, determine a first grid from the multiple grids, and transmit the first grid to the feature extraction network, wherein the first grid is used to predict the first position of the driver in the driving image; The second processing unit is used to predict the driver's first position in the driving image based on the first grid through the first feature extraction network, extract the first driving behavior feature based on the first position, and determine the probability value of the first driving behavior feature belonging to different lane change categories. The third processing unit is used to generate a first lane change intention corresponding to the driving image based on the lane change category and the probability value corresponding to the lane change category.

8. A computer device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 6.

Images