Front vehicle driving behavior recognition method and model training method, device, and electronic equipment

Abstract

The present invention relates to the field of intelligent driving, and specifically to a method for identifying driving behavior of a preceding vehicle and a model training method, device, and electronic device. Obtain at least two frames of driving images of a target preceding vehicle corresponding to the vehicle; input each frame of the driving image of the target preceding vehicle into a driving behavior identification model of the target preceding vehicle; the driving behavior identification model of the target preceding vehicle includes at least one driving behavior determination model of the target preceding vehicle and one driving behavior evaluation model of the target preceding vehicle; each driving behavior determination model of the target preceding vehicle identifies each frame of the driving image of the target preceding vehicle and determines the candidate driving behavior of the target preceding vehicle; the driving behavior evaluation model of the target preceding vehicle evaluates each candidate driving behavior of the preceding vehicle and determines the driving behavior of the target preceding vehicle corresponding to the target preceding vehicle. The accuracy of the driving behavior of the determined target preceding vehicle can be ensured, thereby avoiding the error of the driving behavior of the determined target preceding vehicle, which has a significant impact on the driving safety of the host vehicle.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving, and in particular to a method for identifying driving behavior of a preceding vehicle, a model training method, a device, and an electronic device. Background Art

[0002] According to statistics, a large number of traffic accidents are caused by the driver's misjudgment or operational errors, and most of these accidents occur during the lane change stage, causing significant casualties and property losses. According to statistics, more than 6% of traffic accidents are caused by lane changes. In addition, the vehicle's turn signal opening rate is not high, which brings potential dangers to the safety of vehicle lane changes. Therefore, advanced driver assistance systems need to accurately identify the behavior of the vehicle in front, issue warnings to the main vehicle, or adaptively switch the following objects to improve the driver's comfort and safety. In addition, accurately identifying the movement behavior of the vehicle in front helps autonomous vehicles to develop safe operating trajectories and reasonable decisions.

[0003] In the prior art, the driving behavior of the preceding vehicle is usually identified based on time series data. However, the model based on time series data requires several high-precision devices to be equipped on the vehicle, which faces compatibility issues with the devices. The entire deployment process is complex, time-consuming, and labor-intensive. Most importantly, the overall equipment cost is high, which limits the industrial application of the model.

[0004] Therefore, proposing an efficient, convenient and low-cost method for identifying the driving behavior of the preceding vehicle has become an urgent problem to be solved. Summary of the invention

[0005] In view of this, the present invention provides a preceding vehicle driving behavior recognition method and a model training method, device, and electronic device to solve the current problems.

[0006] In a first aspect, the present invention provides a method for identifying driving behavior of a preceding vehicle, the method comprising:

[0007] Acquire at least two frames of driving images of the target preceding vehicle corresponding to the vehicle;

[0008] Inputting each frame of the target front vehicle driving image into a target front vehicle driving behavior recognition model; the target front vehicle driving behavior recognition model includes at least one target front vehicle driving behavior determination model and a target front vehicle driving behavior evaluation model;

[0009] The driving behavior determination model of each target front vehicle recognizes the driving image of each frame of the target front vehicle and determines the candidate front vehicle driving behavior of the target front vehicle;

[0010] The target preceding vehicle driving behavior evaluation model evaluates the driving behaviors of each candidate preceding vehicle and determines the target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0011] The preceding vehicle driving behavior recognition method provided in the embodiment of the present application obtains at least two frames of the target preceding vehicle driving image corresponding to the target preceding vehicle of the present vehicle; each frame of the target preceding vehicle driving image is input into the target preceding vehicle driving behavior recognition model, and each target preceding vehicle driving behavior determination model in the target preceding vehicle driving behavior recognition model recognizes each frame of the target preceding vehicle driving image, determines the candidate preceding vehicle driving behavior of the target preceding vehicle, and ensures the accuracy of the determined candidate preceding vehicle driving behavior. The target preceding vehicle driving behavior evaluation model evaluates each candidate preceding vehicle driving behavior, determines the target preceding vehicle driving behavior corresponding to the target preceding vehicle, and ensures the accuracy of the determined target preceding vehicle driving behavior corresponding to the target preceding vehicle. In the above method, the candidate preceding vehicle driving behavior of the target preceding vehicle is determined based on at least one target preceding vehicle driving behavior determination model in the target preceding vehicle driving behavior recognition model, and then each candidate preceding vehicle driving behavior is evaluated to determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle. Instead of determining the target preceding vehicle's driving behavior through only a single model, the accuracy of the determined target preceding vehicle's driving behavior can be guaranteed, thereby avoiding errors in the determined target preceding vehicle's driving behavior, which would have a significant impact on the driving safety of the main vehicle.

[0012] In an optional implementation, obtaining at least two frames of driving images of a target preceding vehicle corresponding to the vehicle includes:

[0013] Obtain the original driving video of the target front vehicle;

[0014] According to a preset extraction rule, extract at least two frames of original front vehicle driving images corresponding to the target front vehicle from the original front vehicle driving video;

[0015] Recognize each frame of the original front vehicle driving image to determine the position of the target front vehicle in each frame of the original front vehicle driving image;

[0016] According to the position of the target front vehicle in each frame of the original front vehicle driving image, each frame of the original front vehicle driving image is cropped to obtain each frame of the target front vehicle driving image corresponding to the target front vehicle.

[0017] The method for identifying the driving behavior of a preceding vehicle provided in the embodiment of the present application obtains the original preceding vehicle driving video corresponding to the target preceding vehicle; according to the preset extraction rules, at least two frames of original preceding vehicle driving images corresponding to the target preceding vehicle are extracted from the original preceding vehicle driving video, ensuring that the extracted original preceding vehicle driving images can not only represent the driving trajectory of the preceding vehicle, but also make the number of extracted original preceding vehicle driving images appropriate, neither there will be too many extracted original preceding vehicle driving images, occupying too much memory, resulting in too long a time to determine the driving behavior of the target preceding vehicle based on the original preceding vehicle driving images, nor there will be too few extracted original preceding vehicle driving images, resulting in inaccurate determination of the driving behavior of the target preceding vehicle. Recognize each frame of the original preceding vehicle driving image, determine the position of the target preceding vehicle in each frame of the original preceding vehicle driving image, and according to the position of the target preceding vehicle in each frame of the original preceding vehicle driving image, crop each frame of the original preceding vehicle driving image to obtain each frame of the target preceding vehicle driving image corresponding to the target preceding vehicle, ensuring that each frame of the target preceding vehicle driving image can accurately represent the position information of the preceding vehicle, and ensuring that the obtained target preceding vehicle driving image occupies less memory.

[0018] In an optional implementation, inputting each frame of the target front vehicle driving image into the target front vehicle driving behavior recognition model includes:

[0019] Performing optical flow processing on the target front vehicle driving image of each frame to generate the target front vehicle optical flow image of each frame corresponding to the target front vehicle driving image of each frame;

[0020] For each frame of the target front vehicle optical flow image, the target front vehicle optical flow image is spliced ​​with the corresponding target front vehicle driving image to generate a target image to be processed;

[0021] Input the target image to be processed in each frame into the target front vehicle driving behavior recognition model.

[0022] The preceding vehicle driving behavior recognition method provided in the embodiment of the present application performs optical flow processing on each frame of the target preceding vehicle driving image, generates each frame of the target preceding vehicle optical flow image corresponding to each frame of the target preceding vehicle driving image, ensures the accuracy of the generated target preceding vehicle optical flow image, and then can determine the time characteristics of the target preceding vehicle based on the target preceding vehicle optical flow image. For each frame of the target preceding vehicle optical flow image, the target preceding vehicle optical flow image is spliced ​​with the corresponding target preceding vehicle driving image to generate a target to-be-processed image, ensuring that the generated target to-be-processed image can characterize both the time characteristics corresponding to the target preceding vehicle and the spatial characteristics corresponding to the target preceding vehicle. Each frame of the target preceding vehicle to-be-processed image is input into the target preceding vehicle driving behavior recognition model, ensuring that the time characteristics corresponding to the target preceding vehicle recognize each frame of the target preceding vehicle to be processed, and determining the accuracy of the target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0023] In an optional implementation, the candidate leading vehicle driving behaviors include three driving behaviors: a leading vehicle changing lanes to the left, a leading vehicle changing lanes to the right, and a leading vehicle going straight. The target leading vehicle driving behavior determination model includes a target feature extraction network and a target driving behavior determination network. Each target leading vehicle driving behavior determination model recognizes each frame of the target leading vehicle driving image to determine the candidate leading vehicle driving behaviors of the target leading vehicle, including:

[0024] The target feature extraction network is used to perform feature recognition on the target front vehicle optical flow image in each frame of the target to be processed image, and the feature representation in each frame of the target front vehicle optical flow image is extracted;

[0025] The time series analysis technology is used to calculate the feature representation corresponding to the optical flow image of the target vehicle in front of each frame, and the time feature corresponding to the optical flow image of the target vehicle in front of each frame is generated;

[0026] Combine the calculated time features in chronological order to form a time feature matrix;

[0027] The target feature extraction network performs feature recognition on the target front vehicle driving image in each frame of the target to be processed image, and generates a spatial feature matrix corresponding to the target front vehicle driving image in each frame;

[0028] The target feature extraction network fuses the temporal feature matrix and the spatial feature matrix to generate a target feature matrix;

[0029] The target driving behavior determination network in each target front vehicle driving behavior determination model performs encoding processing on the target feature matrix to generate an encoding matrix;

[0030] Each target driving behavior determination network identifies the coding matrix and outputs the probability values ​​corresponding to the front vehicle changing lanes to the left, the front vehicle changing lanes to the right, and the front vehicle going straight.

[0031] According to the probability values ​​corresponding to the front vehicle's lane-changing behavior to the left, the front vehicle's lane-changing behavior to the right, and the front vehicle's straight-ahead behavior, respectively, the driving behavior with the highest probability value is determined as the candidate front vehicle driving behavior of the target front vehicle.

[0032] The method for identifying the driving behavior of the preceding vehicle provided in the embodiment of the present application uses a target feature extraction network to perform feature recognition on the target preceding vehicle optical flow image in each frame of the target to be processed image, and extracts the feature representation in each frame of the target preceding vehicle optical flow image, thereby ensuring the accuracy of the extracted feature representation. The feature representation corresponding to each frame of the target preceding vehicle optical flow image is calculated using time series analysis technology to generate the time features corresponding to each frame of the target preceding vehicle optical flow image, thereby ensuring the accuracy of the generated time features. The calculated time features are combined together in time sequence to form a time feature matrix, thereby ensuring the accuracy of the generated time feature matrix. The target feature extraction network performs feature recognition on the target preceding vehicle driving image in each frame of the target to be processed image, and generates the spatial feature matrix corresponding to each frame of the target preceding vehicle driving image, thereby ensuring the accuracy of the generated spatial feature matrix. The target feature extraction network performs fusion processing on the time feature matrix and the spatial feature matrix to generate the target feature matrix, thereby ensuring the accuracy of the generated target feature matrix. The target driving behavior determination network in each target front vehicle driving behavior determination model encodes the target feature matrix to generate a coding matrix. Each target driving behavior determination network identifies the coding matrix and outputs the probability values ​​corresponding to the front vehicle changing lanes to the left, the front vehicle changing lanes to the right, and the front vehicle going straight, respectively, to ensure the accuracy of the output probability values. According to the output probability values ​​corresponding to the front vehicle changing lanes to the left, the front vehicle changing lanes to the right, and the front vehicle going straight, the driving behavior with the highest probability value is determined as the candidate front vehicle driving behavior of the target front vehicle, to ensure the accuracy of the determined candidate front vehicle driving behavior of the target front vehicle.

[0033] In an optional implementation, the target preceding vehicle driving behavior evaluation model evaluates each candidate preceding vehicle driving behavior to determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle, including:

[0034] The target preceding vehicle driving behavior evaluation model votes and counts each candidate preceding vehicle driving behavior output by each target preceding vehicle driving behavior determination model;

[0035] The candidate preceding vehicle driving behavior with the most votes is determined as the target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0036] In the preceding vehicle driving behavior identification method provided in the embodiment of the present application, the target preceding vehicle driving behavior evaluation model votes and counts each candidate preceding vehicle driving behavior output by each target preceding vehicle driving behavior determination model, and determines the candidate preceding vehicle driving behavior with the most votes as the target preceding vehicle driving behavior corresponding to the target preceding vehicle, thereby ensuring the accuracy of the determined target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0037] In a second aspect, the present invention provides a method for training a preceding vehicle driving behavior recognition model, the method comprising:

[0038] Obtaining a training set corresponding to the driving behavior of the preceding vehicle, the training set including a plurality of training image sets corresponding to the driving behaviors of the preceding vehicle and a preceding vehicle driving behavior label corresponding to each training image set; the preceding vehicle driving behavior label is used to represent the actual preceding vehicle driving behavior;

[0039] Input each training image set into an initial preceding vehicle driving behavior recognition network; the initial preceding vehicle driving behavior recognition network includes at least one initial preceding vehicle driving behavior determination network and an initial preceding vehicle driving behavior evaluation network;

[0040] Each initial preceding vehicle driving behavior determination network recognizes each training image set and outputs a first virtual preceding vehicle driving behavior corresponding to each training image set;

[0041] The initial front vehicle driving behavior evaluation network evaluates each first virtual front vehicle driving behavior corresponding to each training image set, and outputs a first target virtual front vehicle driving behavior corresponding to each training image set;

[0042] Inputting the first target virtual leading vehicle driving behavior and the real leading vehicle driving behavior corresponding to each training image set into the first loss function corresponding to the initial leading vehicle driving behavior recognition network, and calculating the first loss value;

[0043] Based on the first loss value, each initial preceding vehicle driving behavior determination network and the initial preceding vehicle driving behavior evaluation network are updated respectively, and this cycle is repeated until the first loss value meets the preset requirement, thereby generating a candidate preceding vehicle driving behavior recognition model;

[0044] A target preceding vehicle driving behavior recognition model is generated according to the candidate preceding vehicle driving behavior recognition model; the target preceding vehicle driving behavior recognition model is applied to the preceding vehicle driving behavior recognition method of the first aspect or any corresponding embodiment thereof.

[0045] The preceding vehicle driving behavior recognition method provided by the embodiment of the present application obtains the training set corresponding to the preceding vehicle driving behavior, inputs each training image set into the initial preceding vehicle driving behavior recognition network, each initial preceding vehicle driving behavior determination network recognizes each training image set, and outputs the first virtual preceding vehicle driving behavior corresponding to each training image set, respectively, thereby ensuring the accuracy of the first virtual preceding vehicle driving behavior output corresponding to each training image set. The initial preceding vehicle driving behavior evaluation network evaluates each first virtual preceding vehicle driving behavior corresponding to each training image set, and outputs the first target virtual preceding vehicle driving behavior corresponding to each training image set, thereby ensuring the accuracy of the first target virtual preceding vehicle driving behavior output corresponding to each training image set. The first target virtual preceding vehicle driving behavior corresponding to each training image set and the real preceding vehicle driving behavior are input into the first loss function corresponding to the initial preceding vehicle driving behavior recognition network, and the first loss value is calculated, thereby ensuring the accuracy of the calculated first loss value. Based on the first loss value, each initial preceding vehicle driving behavior determination network and the initial preceding vehicle driving behavior evaluation network are respectively updated, and the cycle is repeated until the first loss value meets the preset requirements, and a candidate preceding vehicle driving behavior recognition model is generated, thereby ensuring the accuracy of the generated candidate preceding vehicle driving behavior recognition model. According to the candidate leading vehicle driving behavior recognition model, a target leading vehicle driving behavior recognition model is generated, thereby ensuring the accuracy of the generated target leading vehicle driving behavior recognition model.

[0046] In an optional implementation, the candidate preceding vehicle driving behavior recognition model includes at least one candidate preceding vehicle driving behavior determination model and a target preceding vehicle driving behavior evaluation model. According to the candidate preceding vehicle driving behavior recognition model, a target preceding vehicle driving behavior recognition model is generated, including:

[0047] Inputting each training image set into the candidate preceding vehicle driving behavior recognition model again, each candidate preceding vehicle driving behavior determination model in the candidate preceding vehicle driving behavior recognition model recognizes the training image set and outputs a second virtual preceding vehicle driving behavior;

[0048] The target front vehicle driving behavior evaluation model counts votes for each second virtual front vehicle driving behavior, and determines the second virtual front vehicle driving behavior with the largest number of votes as the second target virtual front vehicle driving behavior corresponding to the training image set;

[0049] Comparing the second target virtual preceding vehicle driving behavior with each second virtual preceding vehicle driving behavior, and determining, from each candidate preceding vehicle driving behavior determination model, each candidate preceding vehicle driving behavior determination model to be trained whose output second virtual preceding vehicle driving behavior is inconsistent with the second target virtual preceding vehicle driving behavior;

[0050] Using the training image set, each candidate preceding vehicle driving behavior determination model to be trained is trained again to obtain a target preceding vehicle driving behavior determination model;

[0051] The target front vehicle driving behavior determination model and the target front vehicle driving behavior evaluation model are integrated to generate a target front vehicle driving behavior recognition model.

[0052] The preceding vehicle driving behavior recognition method provided in the embodiment of the present application inputs each training image set into the candidate preceding vehicle driving behavior recognition model again, each candidate preceding vehicle driving behavior determination model in the candidate preceding vehicle driving behavior recognition model recognizes the training image set, and outputs a second virtual preceding vehicle driving behavior; the target preceding vehicle driving behavior evaluation model votes and counts each second virtual preceding vehicle driving behavior, and determines that the second virtual preceding vehicle driving behavior with the largest number of votes is the second target virtual preceding vehicle driving behavior corresponding to the training image set, thereby ensuring the accuracy of the determined second target virtual preceding vehicle driving behavior. The second target virtual preceding vehicle driving behavior is compared with each second virtual preceding vehicle driving behavior, and each candidate preceding vehicle driving behavior determination model to be trained whose output second virtual preceding vehicle driving behavior is inconsistent with the second target virtual preceding vehicle driving behavior is determined from each candidate preceding vehicle driving behavior determination model; each candidate preceding vehicle driving behavior determination model to be trained is trained again using the training image set to obtain a target preceding vehicle driving behavior determination model; the target preceding vehicle driving behavior determination model and the target preceding vehicle driving behavior evaluation model are integrated to generate a target preceding vehicle driving behavior recognition model, thereby ensuring that the accuracy of each target preceding vehicle driving behavior determination model in the target preceding vehicle driving behavior recognition model is high, and avoiding the existence of a target preceding vehicle driving behavior determination model with low accuracy in the target preceding vehicle driving behavior recognition model, which in turn leads to a low accuracy of the target preceding vehicle driving behavior recognition model.

[0053] In a third aspect, the present invention provides a device for identifying driving behavior of a preceding vehicle, the device comprising:

[0054] The first acquisition module is used to acquire at least two frames of driving images of the target preceding vehicle corresponding to the vehicle;

[0055] The first input module is used to input each frame of the target front vehicle driving image into the target front vehicle driving behavior recognition model; the target front vehicle driving behavior recognition model includes at least one target front vehicle driving behavior determination model and a target front vehicle driving behavior evaluation model;

[0056] The first recognition module is used for each target front vehicle driving behavior determination model to recognize each frame of the target front vehicle driving image and determine the candidate front vehicle driving behavior of the target front vehicle;

[0057] The first judging module is used for the target preceding vehicle driving behavior judging model to judge each candidate preceding vehicle driving behavior and determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0058] The preceding vehicle driving behavior recognition device provided in the embodiment of the present application obtains at least two frames of the target preceding vehicle driving image of the target preceding vehicle corresponding to the vehicle; each frame of the target preceding vehicle driving image is input into the target preceding vehicle driving behavior recognition model, and each target preceding vehicle driving behavior determination model in the target preceding vehicle driving behavior recognition model recognizes each frame of the target preceding vehicle driving image, determines the candidate preceding vehicle driving behavior of the target preceding vehicle, and ensures the accuracy of the determined candidate preceding vehicle driving behavior. The target preceding vehicle driving behavior evaluation model evaluates each candidate preceding vehicle driving behavior, determines the target preceding vehicle driving behavior corresponding to the target preceding vehicle, and ensures the accuracy of the determined target preceding vehicle driving behavior corresponding to the target preceding vehicle. In the above device, the candidate preceding vehicle driving behavior of the target preceding vehicle is determined based on at least the target preceding vehicle driving behavior determination model in the target preceding vehicle driving behavior recognition model, and then each candidate preceding vehicle driving behavior is evaluated to determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle. Instead of determining the target preceding vehicle's driving behavior through only a single model, the accuracy of the determined target preceding vehicle's driving behavior can be guaranteed, thereby avoiding errors in the determined target preceding vehicle's driving behavior, which would have a significant impact on the driving safety of the main vehicle.

[0059] In a fourth aspect, the present invention provides a preceding vehicle driving behavior recognition model training device, the device comprising:

[0060] The second acquisition module is used to acquire a training set corresponding to the driving behavior of the preceding vehicle, wherein the training set includes a plurality of training image sets corresponding to the driving behaviors of the preceding vehicle and a preceding vehicle driving behavior label corresponding to each training image set; the preceding vehicle driving behavior label is used to represent the actual preceding vehicle driving behavior;

[0061] A second input module is used to input each training image set into an initial preceding vehicle driving behavior recognition network; the initial preceding vehicle driving behavior recognition network includes at least one initial preceding vehicle driving behavior determination network and an initial preceding vehicle driving behavior evaluation network;

[0062] The second recognition module is used for each initial front vehicle driving behavior determination network to recognize each training image set and output the first virtual front vehicle driving behavior corresponding to each training image set respectively;

[0063] The second judging module is used for the initial front vehicle driving behavior judging network to judge each first virtual front vehicle driving behavior corresponding to each training image set, and output the first target virtual front vehicle driving behavior corresponding to each training image set;

[0064] A calculation module, used for inputting the first target virtual leading vehicle driving behavior and the real leading vehicle driving behavior corresponding to each training image set into the first loss function corresponding to the initial leading vehicle driving behavior recognition network, and calculating the first loss value;

[0065] An updating module, used for updating each initial preceding vehicle driving behavior determination network and each initial preceding vehicle driving behavior evaluation network respectively based on the first loss value, and repeating this cycle until the first loss value meets a preset requirement, thereby generating a candidate preceding vehicle driving behavior recognition model;

[0066] A generation module is used to generate a target preceding vehicle driving behavior recognition model based on a candidate preceding vehicle driving behavior recognition model; the target preceding vehicle driving behavior recognition model is applied to the preceding vehicle driving behavior recognition method of the first aspect or any corresponding embodiment thereof.

[0067] The preceding vehicle driving behavior recognition device provided by the embodiment of the present application obtains the training set corresponding to the preceding vehicle driving behavior, inputs each training image set into the initial preceding vehicle driving behavior recognition network, each initial preceding vehicle driving behavior determination network recognizes each training image set, and outputs the first virtual preceding vehicle driving behavior corresponding to each training image set, respectively, thereby ensuring the accuracy of the first virtual preceding vehicle driving behavior output corresponding to each training image set. The initial preceding vehicle driving behavior evaluation network evaluates each first virtual preceding vehicle driving behavior corresponding to each training image set, and outputs the first target virtual preceding vehicle driving behavior corresponding to each training image set, thereby ensuring the accuracy of the first target virtual preceding vehicle driving behavior output corresponding to each training image set. The first target virtual preceding vehicle driving behavior corresponding to each training image set and the real preceding vehicle driving behavior are input into the first loss function corresponding to the initial preceding vehicle driving behavior recognition network, and the first loss value is calculated, thereby ensuring the accuracy of the calculated first loss value. Based on the first loss value, each initial preceding vehicle driving behavior determination network and the initial preceding vehicle driving behavior evaluation network are respectively updated, and the cycle is repeated until the first loss value meets the preset requirements, and a candidate preceding vehicle driving behavior recognition model is generated, thereby ensuring the accuracy of the generated candidate preceding vehicle driving behavior recognition model. According to the candidate leading vehicle driving behavior recognition model, a target leading vehicle driving behavior recognition model is generated, thereby ensuring the accuracy of the generated target leading vehicle driving behavior recognition model.

[0068] In a fifth aspect, the present invention provides an electronic device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the computer instructions to execute the preceding vehicle driving behavior identification method of the first aspect or any corresponding embodiment thereof and the preceding vehicle driving behavior identification model training method of the second aspect or any corresponding embodiment thereof.

[0069] In a sixth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the preceding vehicle driving behavior identification method of the first aspect or any corresponding embodiment thereof and the preceding vehicle driving behavior identification model training method of the second aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 is a flow chart of a method for identifying driving behavior of a preceding vehicle according to an embodiment of the present invention;

[0072] Figure 2 is a flow chart of another method for identifying driving behavior of a preceding vehicle according to an embodiment of the present invention;

[0073] Figure 3 is a flow chart of another method for identifying driving behavior of a preceding vehicle according to an embodiment of the present invention;

[0074] Figure 4 is a spatial and temporal variation diagram of a target front vehicle during driving according to an embodiment of the present invention;

[0075] Figure 5 is a flow chart of another method for identifying driving behavior of a preceding vehicle according to an embodiment of the present invention;

[0076] Figure 6 is a flow chart of a preceding vehicle driving behavior recognition model training method according to an embodiment of the present invention;

[0077] Figure 7 is a structural block diagram of a preceding vehicle driving behavior recognition device according to an embodiment of the present invention;

[0078] Figure 8 is a structural block diagram of a preceding vehicle driving behavior recognition model training device according to an embodiment of the present invention;

[0079] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0081] According to an embodiment of the present invention, an embodiment of a method for identifying driving behavior of a leading vehicle is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0082] It should be noted that the execution subject of the method for identifying the driving behavior of the preceding vehicle provided in the embodiment of the present application may be a device for identifying the driving behavior of the preceding vehicle, and the device for identifying the driving behavior of the preceding vehicle may be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware, wherein the electronic device may be a control terminal in an intelligent vehicle. In the following method embodiments, the execution subject is an electronic device as an example for explanation.

[0083] In this embodiment, a method for identifying driving behavior of a preceding vehicle is provided, which can be used in the above-mentioned electronic device. Figure 1 is a flow chart of a method for identifying driving behavior of a preceding vehicle according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0084] Step S101, obtaining at least two frames of driving images of a target preceding vehicle corresponding to the own vehicle.

[0085] Specifically, the electronic device can receive at least two frames of driving images of the target preceding vehicle corresponding to the own vehicle transmitted by the image acquisition device.

[0086] Among them, there is a certain time interval between at least two frames of target front vehicle driving images, and the duration of the time interval can be 0.1s, 0.2s, or 0.3s. This application does not specifically limit the time interval between at least two frames of target front vehicle driving images.

[0087] This step will be described in detail below.

[0088] Step S102: input each frame of the target front vehicle driving image into the target front vehicle driving behavior recognition model.

[0089] The target preceding vehicle driving behavior recognition model includes at least one target preceding vehicle driving behavior determination model and one target preceding vehicle driving behavior evaluation model.

[0090] Specifically, the electronic device may input each frame of the target front vehicle driving image into the target front vehicle driving behavior recognition model.

[0091] Among them, each target front vehicle driving behavior determination model can be a radial basis function (RBF) network, a feedforward neural network (FFNN), a convolutional neural network (CNN), a deconvolutional neural network (DN), a deep convolutional inverse graphics network (DCIGN), a generative adversarial network (GAN), a deep residual network (DRN) and an extreme learning machine (ELM) in combination with any one of Transfomer. The embodiment of the present application does not limit the target front vehicle driving behavior determination model.

[0092] The target preceding vehicle driving behavior evaluation model may be any one of a voting ensemble model, a stacking ensemble model, a boosting ensemble model, a random forest model, a gradient boosting tree model, etc. The embodiment of the present application does not limit the target preceding vehicle driving behavior evaluation model.

[0093] Step S103: each target front vehicle driving behavior determination model recognizes each frame of the target front vehicle driving image to determine a candidate front vehicle driving behavior of the target front vehicle.

[0094] Specifically, each target preceding vehicle driving behavior determination model extracts features from each frame of the target preceding vehicle driving image, and determines a candidate preceding vehicle driving behavior of the target preceding vehicle based on the extracted features.

[0095] This step will be described in detail below.

[0096] Step S104: the target preceding vehicle driving behavior evaluation model evaluates each candidate preceding vehicle driving behavior to determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0097] Specifically, the target preceding vehicle driving behavior evaluation model can collect statistics on the candidate preceding vehicle driving behaviors of the target preceding vehicles output by each target preceding vehicle driving behavior determination model, and then determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle based on the statistical results.

[0098] This step will be described in detail below.

[0099] The preceding vehicle driving behavior recognition method provided in the embodiment of the present application obtains at least two frames of the target preceding vehicle driving image corresponding to the target preceding vehicle of the present vehicle; each frame of the target preceding vehicle driving image is input into the target preceding vehicle driving behavior recognition model, and each target preceding vehicle driving behavior determination model in the target preceding vehicle driving behavior recognition model recognizes each frame of the target preceding vehicle driving image, determines the candidate preceding vehicle driving behavior of the target preceding vehicle, and ensures the accuracy of the determined candidate preceding vehicle driving behavior. The target preceding vehicle driving behavior evaluation model evaluates each candidate preceding vehicle driving behavior, determines the target preceding vehicle driving behavior corresponding to the target preceding vehicle, and ensures the accuracy of the determined target preceding vehicle driving behavior corresponding to the target preceding vehicle. In the above method, the candidate preceding vehicle driving behavior of the target preceding vehicle is determined based on at least one target preceding vehicle driving behavior determination model in the target preceding vehicle driving behavior recognition model, and then each candidate preceding vehicle driving behavior is evaluated to determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle. Instead of determining the target preceding vehicle's driving behavior through only a single model, the accuracy of the determined target preceding vehicle's driving behavior can be guaranteed, thereby avoiding errors in the determined target preceding vehicle's driving behavior, which would have a significant impact on the driving safety of the main vehicle.

[0100] In this embodiment, a method for identifying driving behavior of a preceding vehicle is provided, which can be used in the above-mentioned electronic device. Figure 2 is a flow chart of a method for identifying driving behavior of a preceding vehicle according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0101] Step S201, obtaining at least two frames of driving images of the target preceding vehicle corresponding to the own vehicle.

[0102] Specifically, the above step S201 may include the following steps:

[0103] Step S2011, obtaining the original preceding vehicle driving video corresponding to the target preceding vehicle.

[0104] Specifically, the electronic device can receive the original preceding vehicle driving video corresponding to the target preceding vehicle sent by the video acquisition device.

[0105] Step S2012: extracting at least two frames of original front vehicle driving images corresponding to the target front vehicle from the original front vehicle driving video according to a preset extraction rule.

[0106] Specifically, the electronic device may receive a preset extraction rule input by a user, or may set a preset extraction rule according to the data volume of the original front vehicle driving video. The embodiment of the present application does not specifically limit the manner in which the electronic device obtains the preset extraction rule.

[0107] The preset extraction rule may be to extract one frame of the original driving image of the target vehicle in front from every 5 frames of the original driving image of the target vehicle in front, or to extract one frame of the original driving image of the target vehicle in front from every 4 frames of the original driving image of the target vehicle in front, or other extraction rules. The embodiment of the present application does not specifically limit the preset extraction rule.

[0108] Exemplarily, when the data volume of the original front vehicle driving video is greater than 5G, the preset extraction rule may be to extract one frame of original front vehicle driving image corresponding to the target front vehicle from every 5 frames of original front vehicle driving image; when the data volume of the original front vehicle driving video is less than or equal to 5G, the preset extraction rule may be to extract one frame of original front vehicle driving image corresponding to the target front vehicle from every 4 frames of original front vehicle driving image.

[0109] Step S2013, identifying each frame of the original front vehicle driving image, and determining the position of the target front vehicle in each frame of the original front vehicle driving image.

[0110] Specifically, the electronic device can identify the target front vehicle in each frame of the original front vehicle driving image based on the target front vehicle recognition model, and determine the position of the target front vehicle in each frame of the original front vehicle driving image according to the recognition result.

[0111] Among them, the target front vehicle recognition model can be a front vehicle recognition model generated by combining a model based on manual features with Transfomer, wherein the model of manual features can be DPM (Deformable Parts Model), and the target front vehicle recognition model can also be a front vehicle recognition model generated by combining a model based on a convolutional neural network with Transfomer, wherein the model based on the convolutional neural network can be a YOLO (You Only Look Once) detector, R-CNN, (Region-based Convolutional Neural Networks, region-based convolutional neural network) model, SSD (Single Shot MultiBox, single shot multi-box) detector and Mask R-CNN (MaskRegion-based Convolutional Neural Networks, masked region-based convolutional neural network) model, etc. The embodiment of the present application does not specifically limit the target front vehicle recognition model.

[0112] Step S2014, cropping each frame of the original front vehicle driving image according to the position of the target front vehicle in each frame of the original front vehicle driving image to obtain each frame of the target front vehicle driving image corresponding to the target front vehicle.

[0113] Specifically, after the electronic device identifies each frame of the original front vehicle driving image and determines the position of the target front vehicle in each frame of the original front vehicle driving image, it can crop each frame of the original front vehicle driving image according to the position of the target front vehicle in each frame of the original front vehicle driving image, crop out other scenes around the target front vehicle in the original front vehicle driving image, and retain the target front vehicle in each frame of the original front vehicle driving image, thereby obtaining each frame of the target front vehicle driving image corresponding to the target front vehicle.

[0114] Exemplarily, the electronic device can preprocess the original front vehicle driving video. The video frame rate is 25 frames per second. In order to improve the recognition accuracy, training speed and model generalization ability, one frame is extracted from the original front vehicle driving video every 5 frames. If the length is not enough, it is necessary to fill in the blanks. Thus, at least two frames of original front vehicle driving images corresponding to the target front vehicle are extracted from the original front vehicle driving video. Each frame of the original front vehicle driving image is identified to determine the position of the target front vehicle in each frame of the original front vehicle driving image; according to the position of the target front vehicle in each frame of the original front vehicle driving image, each frame of the original front vehicle driving image is cropped to obtain each frame of the target front vehicle driving image corresponding to the target front vehicle. This can remove useless information in the original front vehicle driving image, improve the processing speed, and obtain a 299*299 square target front vehicle driving image after center cropping.

[0115] Step S202: input each frame of the target front vehicle driving image into the target front vehicle driving behavior recognition model.

[0116] The target preceding vehicle driving behavior recognition model includes at least one target preceding vehicle driving behavior determination model and one target preceding vehicle driving behavior evaluation model.

[0117] For details about this step, please refer to the introduction of step S102 in FIG. 1 , which will not be described in detail here.

[0118] Step S203: each target front vehicle driving behavior determination model recognizes each frame of the target front vehicle driving image to determine a candidate front vehicle driving behavior of the target front vehicle.

[0119] For details about this step, please refer to the introduction of step S103 in FIG1 , which will not be described in detail here.

[0120] Step S204: the target preceding vehicle driving behavior evaluation model evaluates each candidate preceding vehicle driving behavior to determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0121] For details about this step, please refer to the description of step S104 in FIG. 1 , which will not be described in detail here.

[0122] In this embodiment, a method for identifying driving behavior of a preceding vehicle is provided, which can be used in the above-mentioned electronic device. Figure 3is a flow chart of a method for identifying driving behavior of a preceding vehicle according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0123] Step S301, obtaining at least two frames of driving images of the target preceding vehicle corresponding to the own vehicle.

[0124] For more information about this step, see Figure 2 The introduction of step S201 is not repeated here.

[0125] Step S302: input each frame of the target front vehicle driving image into the target front vehicle driving behavior recognition model.

[0126] The target preceding vehicle driving behavior recognition model includes at least one target preceding vehicle driving behavior determination model and one target preceding vehicle driving behavior evaluation model.

[0127] Specifically, the above step S302 may include the following steps:

[0128] Step S3021, performing optical flow processing on each frame of the target front vehicle driving image to generate each frame of the target front vehicle optical flow image corresponding to each frame of the target front vehicle driving image.

[0129] Specifically, the electronic device can convert each frame of the target front vehicle driving image into an objective function of a two-dimensional signal, wherein the dependent variable of the objective function is the two-dimensional coordinate position (a, b) of each pixel point in the target front vehicle driving image.

[0130] Specifically, the electronic device can regard each frame of the target vehicle driving image as a function of the two-dimensional signal (the output image is a grayscale image), and the dependent variable is the two-dimensional coordinate position , if we use quadratic polynomial to approximate the target vehicle driving image for approximate modeling, we will get the following objective function:

[0131] (1)

[0132] Among them, B is a 2×2 matrix, z is a 2×1 matrix, and c is a coefficient. Therefore, after parameterization, the above formula 1 can be transformed into formula 2:

[0133] (2)

[0134] Among them, m1, m2, m3, m4, m5 and m6 are all parameters.

[0135] If the original (Cartesian coordinate system) two-dimensional signal space of the target vehicle driving image is converted to (1, a, b, a 2 ,b 2, ab) as the space of basis functions, it means that the target front vehicle driving image requires a six-dimensional vector as a parameter, and the positions of different pixels (a, b) are substituted to calculate the grayscale values ​​of different pixels in the target front vehicle driving image.

[0136] Therefore, for each pixel of the target vehicle driving image, the electronic device constructs a neighborhood of size (2n+1)² with the pixel as the center. For each pixel in the neighborhood, it is used as a sample point to construct a linear model containing six-dimensional parameters. Then, the least squares method is used to fit this linear model to obtain the six-dimensional parameters of the central pixel. The above steps are repeated for each pixel in the target vehicle driving image, and finally a six-dimensional vector representation of the entire target vehicle driving image can be obtained.

[0137] Gray value in a neighborhood The matrix is ​​split and rearranged in column-priority order to become The vector f. At the same time, it is known that (1, a, b, a 2 ,b 2 ,ab) as the dimension of the transformation matrix Z of the basis function is , the dimension of the common parameter vector m in the neighborhood is 6*1. So we have .

[0138] Then, when the electronic device uses the least squares method to solve, not every pixel sample error in the neighborhood has the same influence on the center point. The function uses a two-dimensional Gaussian distribution to assign weights to the influence. Therefore, the influence is weighted using a two-dimensional Gaussian distribution. In a matrix with a two-dimensional Gaussian distribution in the neighborhood, the matrix is ​​split and merged into a column-priority d vector. Therefore, the transformation matrix Z of the original basis function is transformed into the following formula:

[0139] (3)

[0140] By further solving, the parameter matrix m is obtained. The basis function matrix Z is transformed into a dual matrix G. The column vector of the transformed basis function matrix is, and the matrix G after the dual transformation is calculated as follows:

[0141] (4)

[0142] After dual transformation, we get the parameter vector m:

[0143] (5)

[0144] Where ✳ represents the cross-correlation process between two signals.

[0145] For example, Figure 4As shown in the figure, it is a graph of the space (reflected in the form of static images) and time (time is reflected in the form of optical flow) changes during the driving process of the target vehicle in front. Figure 4 The above is a static picture. Figure 4 Below is the corresponding optical flow map calculated by the optical flow algorithm.

[0146] Step S3022: for each frame of the target front vehicle optical flow image, the target front vehicle optical flow image is spliced ​​with the corresponding target front vehicle driving image to generate a target image to be processed.

[0147] Specifically, the electronic device splices the target front vehicle optical flow image with the corresponding target front vehicle driving image for each frame of the target front vehicle optical flow image to generate a target image to be processed.

[0148] Step S3023, inputting each frame of the target to-be-processed image into the target front vehicle driving behavior recognition model.

[0149] Specifically, the electronic device inputs each frame of the target image to be processed into a target front vehicle driving behavior recognition model.

[0150] Step S303: each target front vehicle driving behavior determination model recognizes each frame of the target front vehicle driving image to determine the candidate front vehicle driving behavior of the target front vehicle.

[0151] In an optional embodiment of the present application, the candidate leading vehicle driving behaviors include three driving behaviors: the leading vehicle changing lanes to the left, the leading vehicle changing lanes to the right, and the leading vehicle going straight. The target leading vehicle driving behavior determination model includes a target feature extraction network and a target driving behavior determination network.

[0152] Specifically, the above step S303 may include the following steps:

[0153] Step S3031, using the target feature extraction network to perform feature recognition on the target front vehicle optical flow image in each frame of the target image to be processed, and extracting the feature representation in each frame of the target front vehicle optical flow image.

[0154] Specifically, the electronic device can use the target feature extraction network to extract features from each frame of the target front vehicle optical flow image. Through the forward propagation process of the network, the target front vehicle optical flow image in each frame of the target to be processed image is input into the network to obtain the feature representation of the last layer of the network.

[0155] Step S3032, using time series analysis technology to calculate the feature representation corresponding to the optical flow image of the target vehicle in front of each frame, and generate the time feature corresponding to the optical flow image of the target vehicle in front of each frame.

[0156] Specifically, the electronic device can arrange the feature representations corresponding to each frame of the target vehicle optical flow image in chronological order to form time series data. Each frame of the target vehicle optical flow image corresponds to a time point, and the feature representation is the feature information of the time point. Then, the time series data is processed using time series analysis technology to calculate the time features corresponding to each frame of the target vehicle optical flow image. Commonly used time series analysis techniques include mean, variance, autocorrelation, sliding window, etc., and a suitable method can be selected for calculation according to specific needs.

[0157] Step S3033: combine the calculated time features in chronological order to form a time feature matrix.

[0158] Specifically, the electronic device combines the calculated time features in chronological order to form a time feature matrix.

[0159] Step S3034, the target feature extraction network performs feature recognition on the target front vehicle driving image in each frame of the target to be processed image, and generates a spatial feature matrix corresponding to the target front vehicle driving image in each frame.

[0160] Specifically, the electronic device uses the target feature extraction network to perform feature recognition on each frame of the target front vehicle driving image and extract the spatial features of the image. These features can be high-level representations extracted through models such as convolutional neural networks (CNNs), which can capture the visual information in the image. Then, the feature representation extracted by the electronic device from the convolutional network is a multidimensional array, which needs to be converted into a vector form for subsequent processing. This step can be achieved by flattening the feature map or through a fully connected layer. Then, the electronic device combines the spatial feature vectors extracted from each frame of the image in sequence to form a spatial feature matrix. Each row in the matrix represents the spatial feature vector corresponding to a frame of the image, and each column represents a feature dimension.

[0161] Step S3035: The target feature extraction network fuses the temporal feature matrix and the spatial feature matrix to generate a target feature matrix.

[0162] Optionally, the electronic device can splice the time feature matrix and the space feature matrix in a certain way to form a comprehensive feature matrix. This fusion method can be simply splicing the time feature matrix and the space feature matrix together in the row or column direction, or fusing the time feature matrix and the space feature matrix in a certain weighted manner.

[0163] Optionally, the electronic device can also use deep learning technology to build a fusion network, take the time feature matrix and the space feature matrix as input, and learn the mapping relationship of the target feature matrix through the neural network structure. This method can more flexibly fuse feature information of different dimensions.

[0164] Optionally, the electronic device can also perform weighted averaging on the time feature matrix and the space feature matrix, assign different weights to each according to their importance, and obtain the final target feature matrix. This method can adjust the weights of each feature matrix according to actual conditions to achieve the best fusion effect.

[0165] Step S3036: the target driving behavior determination network in each target front vehicle driving behavior determination model performs encoding processing on the target feature matrix to generate an encoding matrix.

[0166] Specifically, the electronic device can input the target feature matrix into the target driving behavior determination network in the target front vehicle driving behavior determination model, and map the target feature matrix to the encoding space through the forward propagation process of the target driving behavior determination network to generate a corresponding encoding matrix.

[0167] Step S3037: each target driving behavior determination network identifies the coding matrix and outputs probability values ​​corresponding to the preceding vehicle's lane-changing behavior to the left, the preceding vehicle's lane-changing behavior to the right, and the preceding vehicle's straight-ahead behavior, respectively.

[0168] Specifically, the electronic device can identify the coding matrix based on the network determination of each target driving behavior, and output the probability values ​​corresponding to the front vehicle changing lanes to the left, the front vehicle changing lanes to the right, and the front vehicle going straight.

[0169] Step S3038, based on the output probability values ​​corresponding to the front vehicle's lane changing behavior to the left, the front vehicle's lane changing behavior to the right, and the front vehicle's straight driving behavior, the driving behavior with the highest probability value is determined as the candidate front vehicle driving behavior of the target front vehicle.

[0170] Specifically, the electronic device may compare the probability values ​​corresponding to the output front vehicle's lane changing behavior to the left, the front vehicle's lane changing behavior to the right, and the front vehicle going straight, and determine the driving behavior with the highest probability value as the candidate front vehicle driving behavior of the target front vehicle.

[0171] Step S304: the target preceding vehicle driving behavior evaluation model evaluates each candidate preceding vehicle driving behavior to determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0172] Specifically, the above step S304 may include the following steps:

[0173] Step S3041: the target preceding vehicle driving behavior evaluation model votes and counts the candidate preceding vehicle driving behaviors output by each target preceding vehicle driving behavior determination model.

[0174] Specifically, the electronic device may utilize the target preceding vehicle driving behavior evaluation model to perform vote counting on each candidate preceding vehicle driving behavior output by each target preceding vehicle driving behavior determination model.

[0175] Step S3042: determine the candidate preceding vehicle driving behavior with the most votes as the target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0176] Specifically, the electronic device determines a candidate preceding vehicle driving behavior with the most votes as a target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0177] In an optional embodiment of the present application, illustratively, as Figure 5 As shown, the electronic device can extract the original front vehicle driving image from the original front vehicle driving video, and then crop the original front vehicle driving image to obtain each frame of the target front vehicle driving image corresponding to the target front vehicle. The electronic device performs optical flow processing on each frame of the target front vehicle driving image to generate each frame of the target front vehicle optical flow image corresponding to each frame of the target front vehicle driving image. For each frame of the target front vehicle optical flow image, the target front vehicle optical flow image is spliced ​​with the corresponding target front vehicle driving image to generate a target image to be processed.

[0178] The electronic device inputs each frame of the target to be processed image into multiple CNN neural network models respectively, and each target to be processed image extracts features from each frame of the target to be processed image to generate a feature matrix. Then, based on Transformer encoding, each candidate leading vehicle driving behavior is output through a fully connected layer and then a Softmax layer. Each candidate leading vehicle driving behavior is voted, and the candidate leading vehicle driving behavior with the most votes is determined as the target leading vehicle driving behavior corresponding to the target leading vehicle.

[0179] The preceding vehicle driving behavior recognition method provided in the embodiment of the present application performs optical flow processing on each frame of the target preceding vehicle driving image, generates each frame of the target preceding vehicle optical flow image corresponding to each frame of the target preceding vehicle driving image, ensures the accuracy of the generated target preceding vehicle optical flow image, and then can determine the time characteristics of the target preceding vehicle based on the target preceding vehicle optical flow image. For each frame of the target preceding vehicle optical flow image, the target preceding vehicle optical flow image is spliced ​​with the corresponding target preceding vehicle driving image to generate a target to-be-processed image, ensuring that the generated target to-be-processed image can characterize both the time characteristics corresponding to the target preceding vehicle and the spatial characteristics corresponding to the target preceding vehicle. Each frame of the target preceding vehicle to-be-processed image is input into the target preceding vehicle driving behavior recognition model, ensuring that the time characteristics corresponding to the target preceding vehicle recognize each frame of the target preceding vehicle to be processed, and determining the accuracy of the target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0180] The target feature extraction network is used to perform feature recognition on the target front vehicle optical flow image in each frame of the target to be processed image, and the feature representation in each frame of the target front vehicle optical flow image is extracted, thereby ensuring the accuracy of the extracted feature representation. The time series analysis technology is used to calculate the feature representation corresponding to the target front vehicle optical flow image in each frame, and the time features corresponding to the target front vehicle optical flow image in each frame are generated, thereby ensuring the accuracy of the generated time features. The calculated time features are combined in time order to form a time feature matrix, thereby ensuring the accuracy of the generated time feature matrix. The target feature extraction network performs feature recognition on the target front vehicle driving image in each frame of the target to be processed image, and generates the spatial feature matrix corresponding to the target front vehicle driving image in each frame, thereby ensuring the accuracy of the generated spatial feature matrix. The target feature extraction network fuses the temporal feature matrix and the spatial feature matrix to generate the target feature matrix, thereby ensuring the accuracy of the generated target feature matrix. The target driving behavior determination network in each target front vehicle driving behavior determination model encodes the target feature matrix to generate a coding matrix. Each target driving behavior determination network identifies the coding matrix and outputs the probability values ​​corresponding to the front vehicle changing lanes to the left, the front vehicle changing lanes to the right, and the front vehicle going straight, respectively, to ensure the accuracy of the output probability values. According to the output probability values ​​corresponding to the front vehicle changing lanes to the left, the front vehicle changing lanes to the right, and the front vehicle going straight, the driving behavior with the highest probability value is determined as the candidate front vehicle driving behavior of the target front vehicle, to ensure the accuracy of the determined candidate front vehicle driving behavior of the target front vehicle.

[0181] The target preceding vehicle driving behavior evaluation model votes and counts the candidate preceding vehicle driving behaviors output by the target preceding vehicle driving behavior determination models, and determines the candidate preceding vehicle driving behavior with the most votes as the target preceding vehicle driving behavior corresponding to the target preceding vehicle, thereby ensuring the accuracy of the determined target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0182] In this embodiment, a preceding vehicle driving behavior recognition model training method is provided, which can be used in the above-mentioned electronic device. Figure 6 is a flow chart of a preceding vehicle driving behavior recognition model training method according to an embodiment of the present invention. Figure 6 As shown, the process includes the following steps:

[0183] Step S401, obtaining a training set corresponding to the driving behavior of the preceding vehicle.

[0184] The training set includes training image sets corresponding to various preceding vehicle driving behaviors and preceding vehicle driving behavior labels corresponding to each training image set; the preceding vehicle driving behavior labels are used to characterize the actual preceding vehicle driving behaviors.

[0185] Specifically, the electronic device can receive a training set corresponding to the driving behavior of the preceding vehicle input by a user, and can also receive a training set corresponding to the driving behavior of the preceding vehicle sent by other devices, and can also receive a training set corresponding to the driving behavior of the preceding vehicle sent by other devices.

[0186] The embodiment of the present application does not specifically limit the manner in which the electronic device obtains the training set corresponding to the driving behavior of the preceding vehicle.

[0187] Step S402: input each training image set into the initial front vehicle driving behavior recognition network.

[0188] The initial preceding vehicle driving behavior recognition network includes at least one initial preceding vehicle driving behavior determination network and one initial preceding vehicle driving behavior evaluation network.

[0189] Specifically, the electronic device can input each training image set into the initial preceding vehicle driving behavior recognition network.

[0190] Step S403: each initial preceding vehicle driving behavior determination network identifies each training image set and outputs a first virtual preceding vehicle driving behavior corresponding to each training image set.

[0191] Specifically, the electronic device can use each initial preceding vehicle driving behavior determination network in the initial preceding vehicle driving behavior recognition network to identify each training image set, and output the first virtual preceding vehicle driving behavior corresponding to each training image set respectively.

[0192] Step S404: the initial preceding vehicle driving behavior evaluation network evaluates each first virtual preceding vehicle driving behavior corresponding to each training image set, and outputs a first target virtual preceding vehicle driving behavior corresponding to each training image set.

[0193] Specifically, the electronic device can use the initial preceding vehicle driving behavior judging network in the initial preceding vehicle driving behavior recognition network to judge each first virtual preceding vehicle driving behavior corresponding to each training image set, and output the first target virtual preceding vehicle driving behavior corresponding to each training image set.

[0194] Step S405: input the first target virtual leading vehicle driving behavior and the real leading vehicle driving behavior corresponding to each training image set into the first loss function corresponding to the initial leading vehicle driving behavior recognition network to calculate the first loss value.

[0195] Specifically, the electronic device inputs the first target virtual leading vehicle driving behavior and the real leading vehicle driving behavior corresponding to each training image set into the first loss function corresponding to the initial leading vehicle driving behavior recognition network, and calculates the first loss value.

[0196] Step S406, based on the first loss value, each initial preceding vehicle driving behavior determination network and each initial preceding vehicle driving behavior evaluation network are updated respectively, and this cycle is repeated until the first loss value meets the preset requirement, thereby generating a candidate preceding vehicle driving behavior recognition model.

[0197] Specifically, the electronic device updates each initial preceding vehicle driving behavior determination network and each initial preceding vehicle driving behavior evaluation network based on the first loss value, and repeats this cycle until the first loss value meets a preset requirement, thereby generating a candidate preceding vehicle driving behavior recognition model.

[0198] Step S407: generating a target leading vehicle driving behavior recognition model based on the candidate leading vehicle driving behavior recognition model.

[0199] Among them, the target preceding vehicle driving behavior recognition model is applied to the preceding vehicle driving behavior recognition method in any one of the above-mentioned implementation modes.

[0200] In an optional implementation of the present application, the candidate preceding vehicle driving behavior recognition model includes at least one candidate preceding vehicle driving behavior determination model and a target preceding vehicle driving behavior evaluation model. The above step S407 may include the following steps:

[0201] Step S4071, input each training image set into the candidate preceding vehicle driving behavior recognition model again, each candidate preceding vehicle driving behavior determination model in the candidate preceding vehicle driving behavior recognition model recognizes the training image set and outputs a second virtual preceding vehicle driving behavior.

[0202] Specifically, the electronic device may input each training image set into the candidate preceding vehicle driving behavior recognition model again, and each candidate preceding vehicle driving behavior determination model in the candidate preceding vehicle driving behavior recognition model recognizes the training image set and outputs a second virtual preceding vehicle driving behavior.

[0203] Step S4072: the target preceding vehicle driving behavior evaluation model counts votes for each second virtual preceding vehicle driving behavior, and determines the second virtual preceding vehicle driving behavior with the largest number of votes as the second target virtual preceding vehicle driving behavior corresponding to the training image set.

[0204] Specifically, the electronic device can use the target front vehicle driving behavior evaluation model to count votes for each second virtual front vehicle driving behavior, and determine that the second virtual front vehicle driving behavior with the largest number of votes is the second target virtual front vehicle driving behavior corresponding to the training image set.

[0205] Step S4073, comparing the second target virtual preceding vehicle driving behavior with each second virtual preceding vehicle driving behavior, and determining from each candidate preceding vehicle driving behavior determination model each candidate preceding vehicle driving behavior determination model to be trained whose output second virtual preceding vehicle driving behavior is inconsistent with the second target virtual preceding vehicle driving behavior.

[0206] Specifically, the electronic device can compare the second target virtual preceding vehicle driving behavior with each second virtual preceding vehicle driving behavior, and determine from each candidate preceding vehicle driving behavior determination model each candidate preceding vehicle driving behavior determination model to be trained whose output second virtual preceding vehicle driving behavior is inconsistent with the second target virtual preceding vehicle driving behavior.

[0207] Step S4074: train each candidate preceding vehicle driving behavior determination model again using the training image set to obtain a target preceding vehicle driving behavior determination model.

[0208] Specifically, the electronic device uses the training image set to train each candidate preceding vehicle driving behavior determination model to be trained again to obtain a target preceding vehicle driving behavior determination model.

[0209] Step S4075, the target front vehicle driving behavior determination model and the target front vehicle driving behavior evaluation model are integrated to generate a target front vehicle driving behavior recognition model.

[0210] Specifically, the electronic device combines the target front vehicle driving behavior determination model and the target front vehicle driving behavior evaluation model to generate a target front vehicle driving behavior recognition model.

[0211] The preceding vehicle driving behavior recognition method provided by the embodiment of the present application obtains the training set corresponding to the preceding vehicle driving behavior, inputs each training image set into the initial preceding vehicle driving behavior recognition network, each initial preceding vehicle driving behavior determination network recognizes each training image set, and outputs the first virtual preceding vehicle driving behavior corresponding to each training image set, respectively, thereby ensuring the accuracy of the first virtual preceding vehicle driving behavior output corresponding to each training image set. The initial preceding vehicle driving behavior evaluation network evaluates each first virtual preceding vehicle driving behavior corresponding to each training image set, and outputs the first target virtual preceding vehicle driving behavior corresponding to each training image set, thereby ensuring the accuracy of the first target virtual preceding vehicle driving behavior output corresponding to each training image set. The first target virtual preceding vehicle driving behavior corresponding to each training image set and the real preceding vehicle driving behavior are input into the first loss function corresponding to the initial preceding vehicle driving behavior recognition network, and the first loss value is calculated, thereby ensuring the accuracy of the calculated first loss value. Based on the first loss value, each initial preceding vehicle driving behavior determination network and the initial preceding vehicle driving behavior evaluation network are respectively updated, and the cycle is repeated until the first loss value meets the preset requirements, and a candidate preceding vehicle driving behavior recognition model is generated, thereby ensuring the accuracy of the generated candidate preceding vehicle driving behavior recognition model. Each training image set is input into the candidate leading vehicle driving behavior recognition model again, and each candidate leading vehicle driving behavior determination model in the candidate leading vehicle driving behavior recognition model recognizes the training image set and outputs the second virtual leading vehicle driving behavior; the target leading vehicle driving behavior evaluation model votes and counts each second virtual leading vehicle driving behavior, and determines that the second virtual leading vehicle driving behavior with the largest number of votes is the second target virtual leading vehicle driving behavior corresponding to the training image set, thereby ensuring the accuracy of the determined second target virtual leading vehicle driving behavior. The second target virtual preceding vehicle driving behavior is compared with each second virtual preceding vehicle driving behavior, and each candidate preceding vehicle driving behavior determination model to be trained whose output second virtual preceding vehicle driving behavior is inconsistent with the second target virtual preceding vehicle driving behavior is determined from each candidate preceding vehicle driving behavior determination model; each candidate preceding vehicle driving behavior determination model to be trained is trained again using the training image set to obtain a target preceding vehicle driving behavior determination model; the target preceding vehicle driving behavior determination model and the target preceding vehicle driving behavior evaluation model are integrated to generate a target preceding vehicle driving behavior recognition model, thereby ensuring that the accuracy of each target preceding vehicle driving behavior determination model in the target preceding vehicle driving behavior recognition model is high, and avoiding the existence of a target preceding vehicle driving behavior determination model with low accuracy in the target preceding vehicle driving behavior recognition model, which in turn leads to a low accuracy of the target preceding vehicle driving behavior recognition model.

[0212] In this embodiment, a preceding vehicle driving behavior recognition device and a preceding vehicle driving behavior recognition model training device are also provided. The above-mentioned devices are used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0213] This embodiment provides a device for identifying the driving behavior of a preceding vehicle. Figure 7 As shown, the device comprises:

[0214] The first acquisition module 501 is used to acquire at least two frames of driving images of the target preceding vehicle corresponding to the vehicle;

[0215] The first input module 502 is used to input each frame of the target front vehicle driving image into the target front vehicle driving behavior recognition model; the target front vehicle driving behavior recognition model includes at least one target front vehicle driving behavior determination model and a target front vehicle driving behavior evaluation model;

[0216] The first recognition module 503 is used for each target front vehicle driving behavior determination model to recognize each frame of the target front vehicle driving image and determine the candidate front vehicle driving behavior of the target front vehicle;

[0217] The first judging module 504 is used for the target preceding vehicle driving behavior judging model to judge each candidate preceding vehicle driving behavior and determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle.

[0218] This embodiment provides a preceding vehicle driving behavior recognition model training device, such as Figure 8 As shown, the device comprises:

[0219] The second acquisition module 601 is used to acquire a training set corresponding to the driving behavior of the preceding vehicle, wherein the training set includes a plurality of training image sets corresponding to the driving behaviors of the preceding vehicle and a preceding vehicle driving behavior label corresponding to each training image set; the preceding vehicle driving behavior label is used to represent the actual preceding vehicle driving behavior;

[0220] The second input module 602 is used to input each training image set into the initial preceding vehicle driving behavior recognition network; the initial preceding vehicle driving behavior recognition network includes at least one initial preceding vehicle driving behavior determination network and an initial preceding vehicle driving behavior evaluation network;

[0221] The second recognition module 603 is used for each initial front vehicle driving behavior determination network to recognize each training image set and output the first virtual front vehicle driving behavior corresponding to each training image set;

[0222] The second judging module 604 is used for the initial front vehicle driving behavior judging network to judge each first virtual front vehicle driving behavior corresponding to each training image set, and output the first target virtual front vehicle driving behavior corresponding to each training image set;

[0223] A calculation module 605 is used to input the first target virtual leading vehicle driving behavior and the real leading vehicle driving behavior corresponding to each training image set into a first loss function corresponding to the initial leading vehicle driving behavior recognition network to calculate a first loss value;

[0224] An updating module 606 is used to update each initial preceding vehicle driving behavior determination network and each initial preceding vehicle driving behavior evaluation network based on the first loss value, and repeat this cycle until the first loss value meets a preset requirement, thereby generating a candidate preceding vehicle driving behavior recognition model;

[0225] The generation module 607 is used to generate a target preceding vehicle driving behavior recognition model according to the candidate preceding vehicle driving behavior recognition model; the target preceding vehicle driving behavior recognition model is applied to the preceding vehicle driving behavior recognition method of any one of the above-mentioned implementation modes.

[0226] In this embodiment, the preceding vehicle driving behavior recognition device and the preceding vehicle driving behavior recognition model training device are presented in the form of functional units, where the units refer to ASIC circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0227] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0228] The embodiment of the present invention also provides an electronic device having the above Figure 7 The preceding vehicle driving behavior recognition device and Figure 8 The preceding vehicle driving behavior recognition model training device is shown.

[0229] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present invention, such as Fig. 9As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9 A processor 10 is taken as an example.

[0230] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0231] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0232] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of an electronic device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0233] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0234] The electronic device further comprises a communication interface 30 for the electronic device to communicate with other devices or a communication network.

[0235] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0236] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for identifying driving behavior of a preceding vehicle, characterized in that: The method comprises: Acquire at least two frames of driving images of the target preceding vehicle corresponding to the vehicle; Performing optical flow processing on each frame of the target front vehicle driving image to generate each frame of the target front vehicle optical flow image corresponding to each frame of the target front vehicle driving image; splicing the target front vehicle optical flow image with the target front vehicle driving image to generate a target image to be processed; inputting each frame of the target image to be processed into a target front vehicle driving behavior recognition model; the target front vehicle driving behavior recognition model includes multiple target front vehicle driving behavior determination models and a target front vehicle driving behavior evaluation model; wherein the target front vehicle driving behavior determination model includes a target feature extraction network and a target driving behavior determination network; Each target preceding vehicle driving behavior determination model identifies the target preceding vehicle driving image in each frame to determine the candidate preceding vehicle driving behavior of the target preceding vehicle; including: using the target feature extraction network to perform feature recognition on the target preceding vehicle optical flow image in each frame of the target to-be-processed image to extract the feature representation in each frame of the target preceding vehicle optical flow image; using the time series analysis technology to calculate the feature representation to generate the time feature corresponding to the target preceding vehicle optical flow image; combining each of the time features in time sequence to form a time feature matrix; performing feature recognition on the target preceding vehicle driving image in each frame of the target to-be-processed image to generate a spatial feature matrix corresponding to the target preceding vehicle driving image; fusing the time feature matrix and the spatial feature matrix to generate a target feature matrix; the target driving behavior determination network encodes the target feature matrix to generate an encoding matrix; identifying the encoding matrix, and outputting the probability values ​​corresponding to each candidate preceding vehicle driving behavior respectively; according to the corresponding probability values, determining the driving behavior with the highest probability value as the candidate preceding vehicle driving behavior; The target preceding vehicle driving behavior evaluation model evaluates each of the candidate preceding vehicle driving behaviors to determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle; including: counting votes for each of the candidate preceding vehicle driving behaviors; and determining the candidate preceding vehicle driving behavior with the most votes as the target preceding vehicle driving behavior.

2. The method according to claim 1, characterized in that: The step of acquiring at least two frames of driving images of a target preceding vehicle corresponding to the vehicle includes: Obtaining an original preceding vehicle driving video corresponding to the target preceding vehicle; According to a preset extraction rule, extracting at least two frames of original front vehicle driving images corresponding to the target front vehicle from the original front vehicle driving video; Identify the original front vehicle driving image of each frame to determine the position of the target front vehicle in the original front vehicle driving image of each frame; According to the position of the target front vehicle in each frame of the original front vehicle driving image, the original front vehicle driving image in each frame is cropped to obtain the target front vehicle driving image in each frame corresponding to the target front vehicle.

3. The method according to claim 1, characterized in that The candidate leading vehicle driving behaviors include three driving behaviors: the leading vehicle changing lanes to the left, the leading vehicle changing lanes to the right, and the leading vehicle going straight.

4. A method for training a preceding vehicle driving behavior recognition model, characterized in that: The method comprises: Acquire a training set corresponding to the driving behavior of the preceding vehicle, wherein the training set includes a plurality of training image sets corresponding to the driving behaviors of the preceding vehicle and a preceding vehicle driving behavior label corresponding to each of the training image sets; the preceding vehicle driving behavior label is used to characterize the actual preceding vehicle driving behavior; Inputting each of the training image sets into an initial preceding vehicle driving behavior recognition network; the initial preceding vehicle driving behavior recognition network includes a plurality of initial preceding vehicle driving behavior determination networks and an initial preceding vehicle driving behavior evaluation network; Each of the initial preceding vehicle driving behavior determination networks recognizes each of the training image sets, and outputs a first virtual preceding vehicle driving behavior corresponding to each of the training image sets; The initial preceding vehicle driving behavior evaluation network evaluates each of the first virtual preceding vehicle driving behaviors corresponding to each of the training image sets, and outputs a first target virtual preceding vehicle driving behavior corresponding to each of the training image sets; Inputting the first target virtual preceding vehicle driving behavior and the real preceding vehicle driving behavior corresponding to each of the training image sets into a first loss function corresponding to the initial preceding vehicle driving behavior recognition network, and calculating a first loss value; Based on the first loss value, each of the initial preceding vehicle driving behavior determination network and the initial preceding vehicle driving behavior evaluation network is updated respectively, and this cycle is repeated until the first loss value meets the preset requirement, thereby generating a candidate preceding vehicle driving behavior recognition model; A target preceding vehicle driving behavior recognition model is generated based on the candidate preceding vehicle driving behavior recognition model; the target preceding vehicle driving behavior recognition model is applied to the preceding vehicle driving behavior recognition method according to any one of claims 1 to 3.

5. The method according to claim 4, characterized in that The candidate preceding vehicle driving behavior recognition model includes a plurality of candidate preceding vehicle driving behavior determination models and a target preceding vehicle driving behavior evaluation model. The generating a target preceding vehicle driving behavior recognition model according to the candidate preceding vehicle driving behavior recognition model includes: Inputting each of the training image sets into the candidate preceding vehicle driving behavior recognition model again, each of the candidate preceding vehicle driving behavior determination models in the candidate preceding vehicle driving behavior recognition model recognizes the training image set and outputs a second virtual preceding vehicle driving behavior; The target front vehicle driving behavior evaluation model counts votes for each of the second virtual front vehicle driving behaviors, and determines that the second virtual front vehicle driving behavior with the largest number of votes is the second target virtual front vehicle driving behavior corresponding to the training image set; Comparing the second target virtual preceding vehicle driving behavior with each of the second virtual preceding vehicle driving behaviors, and determining, from each of the candidate preceding vehicle driving behavior determination models, each candidate preceding vehicle driving behavior determination model to be trained whose outputted second virtual preceding vehicle driving behavior is inconsistent with the second target virtual preceding vehicle driving behavior; Using the training image set to train each of the candidate preceding vehicle driving behavior determination models to be trained again to obtain a target preceding vehicle driving behavior determination model; The target preceding vehicle driving behavior determination model and the target preceding vehicle driving behavior evaluation model are integrated to generate the target preceding vehicle driving behavior recognition model.

6. A device for identifying driving behavior of a preceding vehicle, characterized in that: The device comprises: The first acquisition module is used to acquire at least two frames of driving images of the target preceding vehicle corresponding to the vehicle; The first input module is used to perform optical flow processing on each frame of the target front vehicle driving image to generate each frame of the target front vehicle optical flow image corresponding to each frame of the target front vehicle driving image; splicing the target front vehicle optical flow image with the target front vehicle driving image to generate a target image to be processed; inputting each frame of the target image to be processed into a target front vehicle driving behavior recognition model; the target front vehicle driving behavior recognition model includes multiple target front vehicle driving behavior determination models and a target front vehicle driving behavior evaluation model; wherein the target front vehicle driving behavior determination model includes a target feature extraction network and a target driving behavior determination network; The first recognition module is used for each target vehicle driving behavior determination model to recognize the target vehicle driving image in each frame and determine the candidate vehicle driving behavior of the target vehicle; including: using the target feature extraction network to perform feature recognition on the target vehicle optical flow image in each frame of the target image to be processed, and extracting the feature representation in each frame of the target vehicle optical flow image; using the time series analysis technology to calculate the feature representation and generate the time feature corresponding to the target vehicle optical flow image; combining each time feature in time order to form a time feature matrix; performing feature recognition on the target vehicle driving image in each frame of the target image to be processed, and generating a spatial feature matrix corresponding to the target vehicle driving image; fusing the time feature matrix and the spatial feature matrix to generate a target feature matrix; the target driving behavior determination network encodes the target feature matrix to generate an encoding matrix; identifying the encoding matrix, and outputting the probability value corresponding to each candidate vehicle driving behavior; according to the corresponding probability value, determining the driving behavior with the highest probability value as the candidate vehicle driving behavior; The first judging module is used for the target preceding vehicle driving behavior judging model to judge each of the candidate preceding vehicle driving behaviors and determine the target preceding vehicle driving behavior corresponding to the target preceding vehicle; including: counting votes for each of the candidate preceding vehicle driving behaviors; and determining the candidate preceding vehicle driving behavior with the most votes as the target preceding vehicle driving behavior.

7. A preceding vehicle driving behavior recognition model training device, characterized in that: The device comprises: A second acquisition module is used to acquire a training set corresponding to the driving behavior of the preceding vehicle, wherein the training set includes a plurality of training image sets corresponding to the driving behaviors of the preceding vehicle and a preceding vehicle driving behavior label corresponding to each of the training image sets; the preceding vehicle driving behavior label is used to characterize the actual preceding vehicle driving behavior; A second input module is used to input each of the training image sets into an initial preceding vehicle driving behavior recognition network; the initial preceding vehicle driving behavior recognition network includes a plurality of initial preceding vehicle driving behavior determination networks and an initial preceding vehicle driving behavior evaluation network; A second recognition module is used for each of the initial preceding vehicle driving behavior determination networks to recognize each of the training image sets and output a first virtual preceding vehicle driving behavior corresponding to each of the training image sets; A second judging module, used for the initial front vehicle driving behavior judging network to judge each of the first virtual front vehicle driving behaviors corresponding to each of the training image sets, and output a first target virtual front vehicle driving behavior corresponding to each of the training image sets; a calculation module, configured to input the first target virtual preceding vehicle driving behavior and the real preceding vehicle driving behavior corresponding to each of the training image sets into a first loss function corresponding to the initial preceding vehicle driving behavior recognition network, and calculate a first loss value; An updating module, configured to update each of the initial preceding vehicle driving behavior determination network and the initial preceding vehicle driving behavior evaluation network based on the first loss value, and repeat this cycle until the first loss value meets a preset requirement, thereby generating a candidate preceding vehicle driving behavior recognition model; A generation module is used to generate a target preceding vehicle driving behavior recognition model based on the candidate preceding vehicle driving behavior recognition model; the target preceding vehicle driving behavior recognition model is applied to the preceding vehicle driving behavior recognition method according to any one of claims 1-3.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the preceding vehicle driving behavior recognition method described in any one of claims 1 to 3 and the preceding vehicle driving behavior recognition model training method described in any one of claims 4-5 by executing the computer instructions.

Images