Method and device for recognizing passing intention based on behavior of surrounding vehicles

Abstract

The application relates to a passing intention recognition method and device based on behaviors of surrounding vehicles, wherein the method comprises the following steps: collecting multiple continuous frame images of a current traffic scene; obtaining a traffic scene dynamic map of a current road according to the multiple continuous frame images of the current traffic scene, and predicting driving behaviors of surrounding vehicles; and constructing a traffic scene space-time graph model by using the traffic scene dynamic map and the driving behaviors, and inferring an actual command intention of a command personnel on the current road based on the traffic scene space-time graph model. According to the application, the traffic scene space-time graph model can be obtained based on the traffic scene dynamic map and the predicted driving behaviors, so that the actual command intention of the command personnel can be inferred, the accuracy and safety of vehicle passing are improved, the intelligent level of the vehicle is improved, the driving and riding experience of the vehicle is ensured, and the vehicle is safe and reliable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, and in particular to a passing intention recognition method and device based on behaviors of surrounding vehicles. BACKGROUND

[0002] In related technologies, a computer vision method can be used to infer the active interaction intention of pedestrians in a traffic scene, especially the command intention of traffic police. In this method, the pose of the traffic police can be obtained through the computer vision method, and the command intention of the traffic police can be inferred based on the pose of the traffic police, so as to realize safe driving of the vehicle.

[0003] However, in related technologies, the actual command intention of the command personnel cannot be inferred in a complex and changeable traffic environment, which reduces the accuracy and safety of vehicle passing, reduces the intelligent level of the vehicle, and cannot meet the driving needs of users, which needs to be solved urgently. SUMMARY

[0004] The present application provides a passing intention recognition method and device based on behaviors of surrounding vehicles to solve the technical problem that the actual command intention of the command personnel cannot be inferred in a complex and changeable traffic environment in related technologies, which reduces the accuracy and safety of vehicle passing, reduces the intelligent level of the vehicle, and cannot meet the driving needs of users.

[0005] The first aspect of the present application provides a passing intention recognition method based on behaviors of surrounding vehicles, comprising the following steps: collecting a plurality of continuous frame images of a current traffic scene; obtaining a traffic scene dynamic map of a current road according to the plurality of continuous frame images of the current traffic scene, and predicting driving behaviors of surrounding vehicles; constructing a traffic scene spatio-temporal graph model using the traffic scene dynamic map and the driving behaviors, and inferring the actual command intention of command personnel on the current road based on the traffic scene spatio-temporal graph model.

[0006] According to the above technical means, the traffic scene spatio-temporal graph model can be obtained based on the traffic scene dynamic map and the predicted driving behaviors, so as to infer the actual command intention of the command personnel, improve the accuracy and safety of vehicle passing, improve the intelligence of the vehicle, ensure the driving experience of the vehicle, and be safe and reliable.

[0007] Optionally, in an embodiment of the present application, the obtaining the traffic scene dynamic map of the current road according to the plurality of continuous frame images of the current traffic scene comprises: inputting the plurality of continuous frame images into a pre-constructed target detection and target tracking network, outputting position information of surrounding vehicles in a spatial dimension, motion information in a time dimension and position information of a commander, and obtaining motion information of the current vehicle; and mapping the position information of the surrounding vehicles in the spatial dimension and the position information of the current vehicle into a preset map to obtain the traffic scene dynamic map.

[0008] According to the above technical means, the embodiment of the present application can obtain the traffic scene dynamic map according to the motion information of the vehicle, thereby effectively improving the intelligent level of the vehicle, and having stronger anti-interference ability in a complex traffic environment, and effectively improving the accuracy of intention reasoning.

[0009] Optionally, in an embodiment of the present application, the predicting the driving behavior of the surrounding vehicle comprises: inputting the image, position information and motion information of the target detection frame of each surrounding vehicle obtained from the plurality of continuous frame images and the motion information of the current vehicle into a pre-constructed multi-branch network architecture to obtain the driving behavior of the surrounding vehicle.

[0010] According to the above technical means, the embodiment of the present application can input the motion information of the vehicle into the multi-branch network architecture, and obtain the driving behavior of the surrounding vehicle, thereby effectively improving the accuracy of vehicle control and meeting the driving demand of the user.

[0011] Optionally, in an embodiment of the present application, the constructing a traffic scene spatio-temporal graph model by using the traffic scene dynamic map comprises: taking the vehicles in the traffic scene dynamic map as nodes of the graph model, taking the interaction relationship between the vehicles as edges of the graph model, and taking the image information, position information, motion information and corresponding driving behavior of all the vehicles as node features to generate the traffic scene spatio-temporal graph model.

[0012] According to the above technical means, the embodiment of the present application can infer the traffic scene spatio-temporal graph model by inferring the node features of the surrounding vehicles in the case of missing visual information of the commander in the vehicle-mounted perspective, thereby effectively improving the executability of the commander intention reasoning and improving the automation level of the vehicle.

[0013] Optionally, in an embodiment of the present application, the inferring the actual command intention of the commander on the current road based on the traffic scene spatio-temporal graph model comprises: inputting the traffic scene spatio-temporal graph model into a preset spatio-temporal graph convolution network to obtain global features of the spatio-temporal graph model; and inferring the actual command intention by using the global features of the spatio-temporal graph model.

[0014] According to the technical means described above, the embodiment of the present application can infer the actual command intention of the command personnel by using the spatio-temporal graph model global feature, thereby effectively improving the accuracy and safety of vehicle passing, improving the intelligent level of the vehicle, and improving the driving experience of the user.

[0015] The second aspect embodiment of the present application provides a passing intention recognition device based on surrounding vehicle behavior, comprising: an acquisition module for acquiring a plurality of continuous frame images of a current traffic scene; a prediction module for obtaining a traffic scene dynamic map of a current road according to the plurality of continuous frame images of the current traffic scene, and predicting the driving behavior of surrounding vehicles; and an inference module for constructing a traffic scene spatio-temporal graph model by using the traffic scene dynamic map and the driving behavior, and inferring the actual command intention of the command personnel on the current road based on the traffic scene spatio-temporal graph model.

[0016] Optionally, in an embodiment of the present application, the prediction module comprises: a first acquisition unit for inputting the plurality of continuous frame images into a pre-constructed target detection and target tracking network, outputting the position information of the spatial dimension of the surrounding vehicles, the motion information of the time dimension, and the position information of the command personnel, and obtaining the motion information of the current vehicle; and a second acquisition unit for mapping the position information of the spatial dimension of the surrounding vehicles and the position information of the current vehicle into a pre-set map to obtain the traffic scene dynamic map.

[0017] Optionally, in an embodiment of the present application, the prediction module further comprises: a prediction unit for inputting the image, position information and motion information of the target detection frame of each surrounding vehicle obtained from the plurality of continuous frame images, and the motion information of the current vehicle into a pre-constructed multi-branch network architecture to obtain the driving behavior of the surrounding vehicles.

[0018] Optionally, in an embodiment of the present application, the inference module is further used for taking the vehicles in the traffic scene dynamic map as the nodes of the graph model, taking the interaction relationship between the vehicles as the edges of the graph model, and taking the image information, position information, motion information and corresponding driving behavior of all vehicles as the node features to generate the traffic scene spatio-temporal graph model.

[0019] Optionally, in an embodiment of the present application, the inference module further comprises: an acquisition unit for inputting the traffic scene spatio-temporal graph model into a pre-set spatio-temporal graph convolution network to obtain the spatio-temporal graph model global feature; and an inference unit for inferring the actual command intention by using the spatio-temporal graph model global feature.

[0020] The third aspect of the present application provides a vehicle, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the passing intention recognition method based on behaviors of surrounding vehicles as described in the above embodiments.

[0021] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the passing intention recognition method based on behaviors of surrounding vehicles as described above.

[0022] The present application has the following beneficial effects:

[0023] (1) The embodiments of the present application can input the motion information of the vehicle into the multi-branch network architecture, and obtain the driving behavior of the surrounding vehicle, thereby effectively improving the accuracy of vehicle control and meeting the driving needs of users.

[0024] (2) The embodiments of the present application can infer the node features of the surrounding vehicle, and thus infer the traffic scene spatiotemporal graph model in the case of missing visual information of the commander in the vehicle-mounted perspective, thereby effectively improving the executability of the intention inference of the commander and improving the automation level of the vehicle.

[0025] (3) The embodiments of the present application can obtain the traffic scene spatiotemporal graph model based on the traffic scene dynamic map and the predicted driving behavior, thereby inferring the actual command intention of the commander, improving the accuracy and safety of passing, improving the intelligence of the vehicle, ensuring the driving experience of the vehicle, and being safe and reliable.

[0026] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0027] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, taken in conjunction with the accompanying drawings, in which:

[0028] Figure 1 A flowchart of a passing intention recognition method based on behaviors of surrounding vehicles according to an embodiment of the present application is provided;

[0029] Figure 2 A principle diagram of a passing intention recognition method based on behaviors of surrounding vehicles according to an embodiment of the present application is provided;

[0030] Figure 3 A diagram of a multi-branch GRU (Gated Recurrent Unit) network architecture according to an embodiment of the present application is provided;

[0031] Figure 4 Structure diagram of passing intention reasoning based on surrounding vehicle behavior for a specific embodiment of the present application;

[0032] Figure 5 Structure diagram of passing intention recognition device based on surrounding vehicle behavior according to an embodiment of the present application;

[0033] Figure 6 Structure diagram of a vehicle provided for an embodiment of the application.

[0034] Wherein, 10 - passing intention recognition device based on surrounding vehicle behavior; 100 - acquisition module, 200 - prediction module, 300 - reasoning module; 601 - memory, 602 - processor, 603 - communication interface. DETAILED DESCRIPTION

[0035] The embodiments of the present application will be described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0036] The passing intention recognition method and device based on surrounding vehicle behavior of the embodiments of the present application are described below with reference to the accompanying drawings. In view of the problem that the related art mentioned in the above background technology cannot reason the actual command intention of the command personnel in a complex and variable traffic environment, reduces the accuracy and safety of vehicle passing, and at the same time, reduces the intelligent level of the vehicle, and cannot meet the driving needs of users, the present application provides a passing intention recognition method based on surrounding vehicle behavior, in which method, a plurality of continuous frame images of the current traffic scene can be acquired, and a traffic scene dynamic map of the current road can be obtained, the driving behavior of surrounding vehicles can be predicted, and a traffic scene space-time graph model can be constructed using the traffic scene dynamic map and the driving behavior, so as to reason the actual command intention of the command personnel on the current road, thereby effectively improving the accuracy and safety of vehicle passing, and at the same time, improving the intelligent level of the vehicle, ensuring the driving experience of the vehicle, and being safe and reliable. Thus, the technical problem that the related art cannot reason the actual command intention of the command personnel in a complex and variable traffic environment, reduces the accuracy and safety of vehicle passing, and at the same time, reduces the intelligent level of the vehicle, and cannot meet the driving needs of users is solved.

[0037] Specifically, Figure 1 A flowchart of a passing intention recognition method based on surrounding vehicle behavior provided by an embodiment of the present application.

[0038] As Figure 1As shown, the passing intention recognition method based on the behaviors of surrounding vehicles includes the following steps:

[0039] In step S101, multiple continuous frame images of the current traffic scene are collected.

[0040] It can be understood that the embodiments of the present application can collect multiple continuous frame images of the current traffic scene, for example, collect continuous frame images of the current traffic scene through a vehicle-mounted camera, so that when the vehicle is in a complex and variable scene, the driving intention of the commander can be inferred by inferring the behaviors of surrounding vehicles, thereby improving the accuracy of intention inference and effectively improving the driving experience of users.

[0041] In step S102, a traffic scene dynamic map of the current road is obtained according to the multiple continuous frame images of the current traffic scene, and the driving behaviors of surrounding vehicles are predicted.

[0042] It can be understood that the embodiments of the present application can obtain a traffic scene dynamic map of the current road according to the multiple continuous frame images of the current traffic scene collected in the following steps, for example, in a complex and variable scene, the specific pose of a traffic participant such as a car can be obtained through a vehicle-mounted camera, and the driving behavior of surrounding vehicles can be predicted, thereby effectively improving the intelligent level of the vehicle and improving the safety and reliability of the vehicle.

[0043] Optionally, in an embodiment of the present application, obtaining a traffic scene dynamic map of the current road according to the multiple continuous frame images of the current traffic scene includes: inputting the multiple continuous frame images into a pre-constructed target detection and target tracking network, outputting the position information of the spatial dimension of the surrounding vehicles, the motion information of the time dimension, and the position information of the commander, and obtaining the motion information of the current vehicle; mapping the position information of the spatial dimension of the surrounding vehicles and the position information of the current vehicle to a preset map to obtain the traffic scene dynamic map.

[0044] For example, as shown, Figure 2 the embodiments of the present application can input the continuous frame images into a target detection network Yolov5 and a target tracking network DeepSort in sequence, detect and track the surrounding vehicles and traffic police in the continuous frame images, and obtain the position coordinates of the surrounding vehicles and traffic police in the images and (u pi ,v pi ), and the visual information and IMGP i in the corresponding target detection frame, where i represents the i-th frame image, n represents the number of recognized surrounding vehicles, and the vehicle motion information y synchronized with the image is obtained through an OBD (On-Board-Diagnostics, vehicle-mounted automatic diagnostic system) sensor installed on the vehiclei wherein the vehicle motion information comprises a vehicle speed, a heading and a GPS coordinate, the position coordinates of the surrounding vehicles and the traffic police in the image are converted into global coordinates in the high-precision map coordinate system through coordinate conversion, that is,

[0045]

[0046] wherein (u, v) is the coordinates of the target in the image, (x w ,y w ) is the coordinates of the target in the high-precision map, and K is a coordinate conversion matrix from the image coordinates to the high-precision map coordinates.

[0047] Therefore, the embodiment of the application can obtain the position coordinates of the surrounding vehicles and the traffic police in the high-precision map, and take the position coordinates as the spatial feature information of the spatial dimension of the surrounding vehicles and the traffic police In addition, the position coordinates of the surrounding vehicles, the traffic police and the vehicle can be mapped into the high-precision map in real time in combination with the vehicle motion information y i provided by the OBD sensor, to form a traffic scene dynamic map DMAP.

[0048] As shown in the traffic scene dynamic map DMAP, the black vehicle represents the vehicle, the white vehicle is the surrounding vehicle in the traffic scene, and the five-point star represents the detected traffic police. Figure 2 The application can calculate the time feature information of the corresponding time dimension according to the spatial feature information of the surrounding vehicles that is,

[0049]

[0050] wherein t represents a time interval.

[0051] In summary, the embodiment of the application can obtain the spatial feature information and the time feature information of the surrounding vehicles in the high-precision map. For the surrounding vehicles, a traffic scene space-time graph model G i (N, E) is constructed with the surrounding vehicles as nodes N and the interaction influence between the surrounding vehicles as edges E, so as to effectively improve the intelligent level of the vehicle, and has stronger anti-interference ability in a complex traffic environment, and effectively improves the accuracy of intention reasoning.

[0052] It should be noted that the preset map is set by a person skilled in the art according to the actual situation, and is not specifically limited here.

[0053] Optionally, in an embodiment of the present application, the driving behavior of the surrounding vehicle is predicted, including: inputting the image, position information and motion information of the target detection frame of each surrounding vehicle obtained from multiple continuous frame images, and the motion information of the current vehicle into a pre-constructed multi-branch network architecture to obtain the driving behavior of the surrounding vehicle.

[0054] For example, in combination with Figure 2 and Figure 3 , the embodiment of the present application can construct a multi-branch GRU network architecture, wherein the multi-branch GRU network architecture includes four GRU branches, and the image information of the surrounding vehicle target detection frame obtained from the continuous frame images spatial feature information temporal feature information and the motion information y of the current vehicle obtained by the OBD sensor i and other multi-source data information F D are respectively input into different branch networks for feature encoding to form multi-source features F_H i , that is:

[0055]

[0056] Among them, the image information is extracted by a lightweight network Mobilenetv3 before being input into GRU1 to obtain the image feature F_I of the surrounding vehicle i , that is:

[0057]

[0058] Then, the embodiment of the present application can use an attention module to fuse the features of different branches to finally generate multi-dimensional fusion features F_A i , that is:

[0059] F_A i =Attention3(Attention2(Attention1(F_H i (2),F_H i (3)),F_H i (1)),F_H i (1))

[0060] Among them, Attention(g) represents the attention module, and F_H i (m) represents the mth dimension of the multi-dimensional feature.

[0061] Further, the fused features F_A iThe input is into the FC (Fully Connected layers), and the final surrounding vehicle behavior prediction result is obtained, that is:

[0062] V i n = FC (F_A i )

[0063] wherein V i n is the surrounding vehicle behavior prediction result, i represents the i-th frame image, and n represents the n-th surrounding vehicle.

[0064] The vehicle behavior prediction result can include left turn, right turn, U-turn, lane change and the like, thereby effectively improving the accuracy of the passing intention reasoning, improving the intelligent level of the vehicle, and ensuring the driving experience of the vehicle.

[0065] In some embodiments, as shown in Figure 3 , the multi-branch GRU network architecture can include four GRU branches respectively receiving the image of the surrounding vehicle target detection frame, the position information of the surrounding vehicle, the motion information of the surrounding vehicle, and the motion information of the vehicle, and encoding the data from different sources respectively, wherein for the image information, the image features can be extracted by using the convolution network first, and then the image features are encoded by using the GRU, and the attention module is used to gradually fuse the GRU encoding features of different branches, finally, the fused features are input into the fully connected layer network to obtain the prediction result of the surrounding vehicle behavior, thereby effectively improving the accuracy of vehicle control and meeting the driving needs of users.

[0066] In step S103, a traffic scene spatio-temporal graph model is constructed using the traffic scene dynamic map and the driving behavior, and the actual command intention of the command personnel on the current road is inferred based on the traffic scene spatio-temporal graph model.

[0067] It can be understood that the embodiments of the present application can construct a traffic scene spatio-temporal graph model using the traffic scene dynamic map and the driving behavior in the following steps, and infer the actual command intention of the command personnel on the current road based on the traffic scene spatio-temporal graph model in the following steps. By identifying the actual command intention, the movement of the current vehicle can be accurately controlled, thereby effectively improving the intelligent level of the vehicle, improving the safety and reliability of the vehicle, and meeting the use requirements of users.

[0068] Optionally, in an embodiment of the present application, the traffic scene spatio-temporal graph model is constructed using the traffic scene dynamic map, which includes: taking the vehicles in the traffic scene dynamic map as the nodes of the graph model, taking the interaction relationship between the vehicles as the edges of the graph model, and taking the image information, position information, motion information and corresponding driving behavior of all vehicles as node features to generate the traffic scene spatio-temporal graph model.

[0069] For example, as shown in the following formula (1), embodiments of the present application can take a vehicle in a traffic scene dynamic map as a node v of a graph model, take an interaction relationship between vehicles as an edge e of the graph model, and take image information, position information, motion information of the vehicle, and a driving behavior of the vehicle predicted through a multi-branch GRU network as a node feature, thereby generating a traffic scene spatio-temporal graph model, i.e.: Figure 2

[0070] G i (v,e)

[0071] wherein i represents an i-th frame image, v represents a node of the graph model, and e represents an edge of the graph model.

[0072] Therefore, embodiments of the present application can be able to infer a traffic scene spatio-temporal graph model by inferring node features of surrounding vehicles in the case of missing visual information of a commander in a vehicle visual field, thereby effectively improving the executability of commander intention inference and improving the automation level of the vehicle.

[0073] Optionally, in an embodiment of the present application, inferring an actual command intention of the commander on the current road based on the traffic scene spatio-temporal graph model comprises: inputting the traffic scene spatio-temporal graph model into a preset spatio-temporal graph convolution network to obtain a spatio-temporal graph model global feature; and inferring the actual command intention by using the spatio-temporal graph model global feature.

[0074] In actual execution, as shown in the following formula (2), embodiments of the present application input the traffic scene spatio-temporal graph model into a preset spatio-temporal graph convolution network, for example, can input the traffic scene spatio-temporal graph model G i (v,e) into a ST-GCN (Spatial Temporal Graph Convolutional Networks, spatio-temporal graph convolution network), and obtain a traffic scene spatio-temporal graph model global feature through propagation and aggregation of node features, and infer the command intention of the commander through the spatio-temporal graph model global feature, thereby effectively improving the accuracy and safety of vehicle passing, improving the intelligent level of the vehicle, and improving the driving experience of the user. Figure 2 For example, as shown in the following formula (3), embodiments of the present application can input visual information IMGP i of a traffic police target detection box recognized in "target detection + target tracking" into a human pose detection network OpenPose, and obtain pose information POSE i of the traffic police, wherein the pose information POSE i of the traffic police mainly includes position coordinates of human key points.

[0075] Figure 2 ​​​and motion information wherein i represents the i-th image frame, j represents the j-th key point, and

[0076]

[0077] wherein t represents a time interval.

[0078]

[0079] wherein Concatenate(g) is a feature concatenation function.

[0080] Subsequently, the acquired traffic police pose information POSE i is input into a fully connected network FC to acquire the pose feature P_F i of the traffic police, i.e.:

[0081] P_F i = FC(POSE i )

[0082] Further, the embodiment of the present application can use a light network Mobilenetv3 and a onehot encoder to respectively extract and encode the image information of the surrounding vehicles and the vehicle behavior V i predicted by the multi-branch GRU network. n

[0083]

[0084]

[0085] wherein I_f i n is the visual feature of the surrounding vehicles, C_f i n is the encoded feature of the behavior of the surrounding vehicles C_f i n , i represents the i-th image frame, and n represents the n-th surrounding vehicle.

[0086] Subsequently, the visual feature I_f i n of the surrounding vehicles, the encoded feature C_f i n of the behavior, and the spatial feature information and the temporal feature information T i n can be fused by using a feature concatenation function Concatenate, and the fused feature is taken as the node feature H t of the traffic scene spatio-temporal graph model, i.e.:

[0087]

[0088] The spatiotemporal graph model G of the traffic scene i (v,e) and node features H of the spatiotemporal graph model of the traffic scene t The input is fed into a spatiotemporal graph convolutional network to achieve the propagation and aggregation of node features, that is:

[0089]

[0090] Among them, H t Let σ be the node feature, and σ be the activation function. For degree matrix, A is the adjacency matrix, I is the identity matrix, W is the first parameter matrix, and H′ is the node feature after propagation and aggregation.

[0091] Secondly, the propagation and aggregation process of node features enables the interaction of node information in the spatiotemporal graph model of the traffic scene. The propagated and aggregated node features H′ are input into the fully connected layer network to obtain the global features FG of the spatiotemporal graph model of the traffic scene. i The global features FG of the spatiotemporal graph model of traffic scene i and the posture characteristics of traffic police P_F i The input is fed into the attention module for feature fusion to generate the fused feature F. i ,Right now:

[0092] F i =Attention4(FG) i ,P_F i )

[0093] Among them, Attention4(g) is the attention module.

[0094] The final fusion feature F i The input is fed into an MLP (Multi-Layer Perception) to infer the traffic police officer's command intention P_I. i ,Right now:

[0095] P_I i =Softmax(MLP(F i ))

[0096] Softmax(·) is the softmax function.

[0097] In summary, the embodiment of the present application can identify the motion state of surrounding vehicles in the traffic scene based on the visual depth network, and infer the command intention of the traffic police by combining the motion state of the surrounding vehicles and the motion state of the vehicle, thereby effectively improving the accuracy of the passing intention inference and the safety of vehicle driving, improving the intelligent level of the vehicle, and improving the driving experience of the user.

[0098] For example, as shown in Figure 4 the structure diagram of the passing intention inference based on the behavior of surrounding vehicles in one specific embodiment of the present application, wherein the vehicle-mounted perception system includes a vehicle-mounted camera, a high-precision map, an OBD sensor, etc., and can be used for perception elements and image processing algorithm processors to obtain images and coordinate positions, input the traffic police image provided by the vehicle-mounted perception system into the posture detection network to obtain the posture of the traffic police, and use the real-time position information of the surrounding vehicles, the vehicle and the traffic police provided by the vehicle-mounted perception system to construct a traffic scene dynamic map, infer the behavior of the surrounding vehicles by using the traffic scene dynamic map, and finally, perform feature fusion on the behavior of the surrounding vehicles and the posture of the traffic police, infer the command intention of the traffic police based on the fused features, thereby effectively improving the accuracy of the passing intention inference and improving the safety of vehicle driving.

[0099] According to the vehicle control method provided by the embodiment of the present application, a plurality of continuous frame images of the current traffic scene can be collected, a traffic scene dynamic map of the current road can be obtained, the driving behavior of the surrounding vehicles can be predicted, and a traffic scene space-time graph model can be constructed by using the traffic scene dynamic map and the driving behavior, thereby inferring the actual command intention of the command personnel on the current road, and effectively improving the accuracy and safety of vehicle passing, improving the intelligent level of the vehicle, ensuring the driving experience of the vehicle, and being safe and reliable. Thus, the technical problem that the actual command intention of the command personnel cannot be inferred in a complex and changeable traffic environment in the related art, the accuracy and safety of vehicle passing are reduced, the intelligent level of the vehicle is reduced, and the driving needs of the user cannot be met is solved.

[0100] Secondly, the passing intention recognition device based on the behavior of surrounding vehicles according to the embodiment of the present application is described with reference to the accompanying drawings.

[0101] Figure 5 is a structure example diagram of the passing intention recognition device based on the behavior of surrounding vehicles in the embodiment of the present application.

[0102] As shown in Figure 5 the passing intention recognition device based on the behavior of surrounding vehicles 10 includes an acquisition module 100, a prediction module 200 and an inference module 300.

[0103] The acquisition module 100 is configured to acquire a plurality of continuous frame images of the current traffic scene.

[0104] The prediction module 200 is configured to acquire a traffic scene dynamic map of a current road according to a plurality of continuous frame images of a current traffic scene, and predict driving behaviors of surrounding vehicles.

[0105] The reasoning module 300 is configured to construct a traffic scene spatio-temporal graph model by using the traffic scene dynamic map and the driving behaviors, and reason an actual command intention of a command personnel on the current road based on the traffic scene spatio-temporal graph model.

[0106] Optionally, in an embodiment of the present application, the prediction module 200 comprises a first acquisition unit and a second acquisition unit.

[0107] The first acquisition unit is configured to input the plurality of continuous frame images into a pre-constructed target detection and target tracking network, output position information in a spatial dimension of the surrounding vehicles, motion information in a time dimension, and position information of the command personnel, and acquire motion information of the current vehicle.

[0108] The second acquisition unit is configured to map the position information in the spatial dimension of the surrounding vehicles and the position information of the current vehicle into a preset map to obtain the traffic scene dynamic map.

[0109] Optionally, in an embodiment of the present application, the prediction module 200 further comprises a prediction unit.

[0110] The prediction unit is configured to input the image, the position information, and the motion information of the target detection frame of each surrounding vehicle obtained from the plurality of continuous frame images, and the motion information of the current vehicle into a pre-constructed multi-branch network architecture to obtain the driving behaviors of the surrounding vehicles.

[0111] Optionally, in an embodiment of the present application, the reasoning module 300 is further configured to take the vehicles in the traffic scene dynamic map as nodes of a graph model, take the interaction relationships between the vehicles as edges of the graph model, and take the image information, the position information, the motion information, and the corresponding driving behaviors of all the vehicles as node features to generate the traffic scene spatio-temporal graph model.

[0112] Optionally, in an embodiment of the present application, the reasoning module 300 comprises an acquisition unit and a reasoning unit.

[0113] The acquisition unit is configured to input the traffic scene spatio-temporal graph model into a preset spatio-temporal graph convolution network to acquire global features of the spatio-temporal graph model.

[0114] The reasoning unit is configured to reason the actual command intention by using the global features of the spatio-temporal graph model.

[0115] It should be noted that the foregoing explanation of the embodiment of the method for identifying the passing intention based on the behaviors of surrounding vehicles is also applicable to the embodiment of the device for identifying the passing intention based on the behaviors of surrounding vehicles, and will not be repeated here.

[0116] The device for identifying the passing intention based on the behaviors of surrounding vehicles according to the embodiment of the present application can collect multiple continuous frame images of the current traffic scene, obtain a traffic scene dynamic map of the current road, predict the driving behaviors of surrounding vehicles, and construct a traffic scene space-time graph model by using the traffic scene dynamic map and the driving behaviors, so as to infer the actual command intention of the command personnel on the current road, thereby effectively improving the accuracy and safety of vehicle passing, improving the intelligent level of the vehicle, ensuring the driving experience of the vehicle, and ensuring safety and reliability. Thus, the technical problem that the actual command intention of the command personnel cannot be inferred in a complex and changeable traffic environment in the related art, the accuracy and safety of vehicle passing are reduced, the intelligent level of the vehicle is reduced, and the driving needs of users cannot be met is solved.

[0117] Figure 6 The vehicle according to the embodiment of the present application is provided with a structural schematic diagram. The vehicle can include:

[0118] The memory 601, the processor 602, and the computer program stored in the memory 601 and executable on the processor 602.

[0119] The processor 602 implements the method for identifying the passing intention based on the behaviors of surrounding vehicles provided in the above embodiments when executing the program.

[0120] Further, the vehicle further includes:

[0121] The communication interface 603 is used for communication between the memory 601 and the processor 602.

[0122] The memory 601 is used to store the computer program executable on the processor 602.

[0123] The memory 601 can include a high-speed RAM memory, and can also include a non-volatile memory such as at least one disk memory.

[0124] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected with each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 6 Only one thick line is used to represent the bus in the figure, but it does not mean that there is only one bus or only one type of bus.

[0125] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can complete communication between each other through an internal interface.

[0126] The processor 602 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the present application.

[0127] The embodiment also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the passing intention recognition method based on behaviors of surrounding vehicles.

[0128] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0129] Moreover, the terms "first", "second", "third", etc. are used herein only to describe different steps or categories of steps. Thus, the use of the term "first" does not imply that different steps must be in a time sequence. Nor is it implied that a "first" step must precede a "second" step, that a "second" step, etc. must follow a "first" step, etc. Furthermore, when a process or method is described herein with several steps or several categories of steps, it should be understood that these are merely illustrative of the steps that can be employed in the process or method. Not all of the steps can be required, and in some cases, additional steps can be employed. The order of the steps can be varied, and some of the steps can be performed simultaneously. The steps can be performed in an order different than that described herein. The steps can be performed in any order, unless otherwise specified.

[0130] Any process or method described in flowcharts or otherwise described herein can be understood as representing a module, segment, or portion of code that includes one or N executable instructions for implementing the specified logical function or process. It will be understood that the scope of the preferred embodiments of the present application encompasses additional implementations that can not be expressly shown or described herein, including implementations that can be performed in an order different than that shown or discussed, including substantially simultaneously, or in reverse order, as appropriate, depending on the functionality involved.

[0131] The logic and / or steps represented in flowcharts or otherwise described herein, for example, can be considered as a list of executable instructions for implementing the logic function, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a product of the manufacturing and / or processing, and can be a machine-readable storage medium (alternatively, computer-readable storage medium). The computer-readable medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or N wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical), and a portable compact disc read-only memory (CDROM). Note that the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for example, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0132] It should be understood that portions of the application can be realized with a combination of hardware, software, firmware, or their combination. In the above-described embodiments, the N steps or methods can be realized with software or firmware stored in a memory and executed by a suitable instruction execution system. As in another embodiment, if realized with hardware, any one or their combination of the following technologies known in the art can be used: discrete logic circuit with logic gate circuit for implementing logic functions on data signals, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA), etc.

[0133] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by a program instructing the relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0134] In addition, each functional unit in each embodiment of the present application can be integrated into one processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0135] The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A method for recognizing traffic intentions based on the behavior of surrounding vehicles, characterized in that, Includes the following steps: Collect multiple consecutive frames of images of the current traffic scene; A dynamic map of the traffic scene on the current road is obtained from multiple consecutive frames of the current traffic scene, and the driving behavior of surrounding vehicles is predicted. as well as A traffic scene spatiotemporal graph model is constructed using the dynamic traffic scene map and the driving behavior, and the actual command intention of the person directing traffic on the current road is inferred based on the traffic scene spatiotemporal graph model. The step of obtaining a dynamic map of the current road traffic scene based on multiple consecutive frame images of the current traffic scene includes: The multiple consecutive frame images are input into a pre-constructed target detection and target tracking network, which outputs the spatial position information of surrounding vehicles, the temporal motion information, and the position information of the command personnel, and obtains the current vehicle's motion information. The spatial location information of the surrounding vehicles and the location information of the current vehicle are both mapped onto a preset map to obtain the dynamic map of the traffic scene. The prediction of driving behavior of surrounding vehicles includes: The target detection bounding boxes, position information, and motion information of each surrounding vehicle obtained from the multiple consecutive frame images, along with the motion information of the current vehicle, are input into a pre-constructed multi-branch network architecture to obtain the driving behavior of the surrounding vehicles.

2. The method according to claim 1, characterized in that, The construction of a spatiotemporal map model of a traffic scene using the dynamic traffic scene map includes: The vehicles in the dynamic map of the traffic scene are used as nodes of the graph model, and the interaction relationships between vehicles are used as edges of the graph model. The image information, location information, motion information and corresponding driving behavior of all vehicles are used as node features to generate the spatiotemporal graph model of the traffic scene.

3. The method according to claim 1, characterized in that, The reasoning of the actual command intentions of the personnel directing traffic on the current road based on the spatiotemporal graph model of the traffic scene includes: The spatiotemporal graph model of the traffic scene is input into a preset spatiotemporal graph convolutional network to obtain the global features of the spatiotemporal graph model; The actual command intent is inferred using the global features of the spatiotemporal graph model.

4. A device for recognizing traffic intentions based on the behavior of surrounding vehicles, characterized in that, include: The acquisition module is used to acquire multiple consecutive frames of images of the current traffic scene; The prediction module is used to obtain a dynamic map of the traffic scene of the current road based on multiple consecutive frame images of the current traffic scene, and to predict the driving behavior of surrounding vehicles. as well as The reasoning module is used to construct a traffic scene spatiotemporal graph model using the traffic scene dynamic map and the driving behavior, and to infer the actual command intention of the person directing traffic on the current road based on the traffic scene spatiotemporal graph model. The prediction module includes: The first acquisition unit is used to input the multiple consecutive frame images into a pre-constructed target detection and target tracking network, output the spatial position information of surrounding vehicles, the temporal motion information and the position information of the command personnel, and acquire the motion information of the current vehicle. The second acquisition unit is used to map the spatial location information of the surrounding vehicles and the location information of the current vehicle onto a preset map to obtain the dynamic map of the traffic scene. The prediction module also includes: The prediction unit is used to input the target detection bounding box image, position information and motion information of each surrounding vehicle obtained from the multiple consecutive frame images, along with the motion information of the current vehicle, into a pre-constructed multi-branch network architecture to obtain the driving behavior of the surrounding vehicles.

5. The apparatus according to claim 4, characterized in that, The reasoning module is further used to take the vehicles in the dynamic map of the traffic scene as nodes of the graph model, take the interaction relationship between the vehicles as edges of the graph model, and take the image information, location information, motion information and corresponding driving behavior of all vehicles as node features to generate the spatiotemporal graph model of the traffic scene.

6. The apparatus according to claim 4, characterized in that, The reasoning module also includes: The acquisition unit is used to input the traffic scene spatiotemporal graph model into a preset spatiotemporal graph convolutional network to acquire the global features of the spatiotemporal graph model; The reasoning unit is used to infer the actual command intent using the global features of the spatiotemporal graph model.

7. A vehicle, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the traffic intention recognition method based on the behavior of surrounding vehicles as described in any one of claims 1-3.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the traffic intention recognition method based on the behavior of surrounding vehicles as described in any one of claims 1-3.

Images