Behavior data processing method, autonomous driving vehicle control method, and autonomous driving vehicle

Abstract

The present disclosure provides a behavioral data processing method, an autonomous vehicle control method, an apparatus, a device, a storage medium, a computer program product, and an autonomous vehicle, relating to the fields of artificial intelligence technology, particularly autonomous driving, big data, and deep learning. A specific implementation scheme comprises: acquiring historical driving data, including lane-level navigation data; performing data mining on the historical driving data to obtain driving feature information, including at least one of the following: lane change position characteristics, driving speed characteristics, and driving path characteristics. By mining historical driving data and assisting in generating autonomous driving decision information, the safety of autonomous driving is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the field of autonomous driving, big data and deep learning technology, and in particular to a behavior data processing method, a control method for an autonomous driving vehicle, an apparatus, a device, a storage medium, a computer program product and an autonomous driving vehicle. Background Art

[0002] With the increasing adoption of artificial intelligence (AI) and fifth-generation mobile communications, autonomous driving technology has rapidly developed, and its adoption in vehicles is increasing. Typically, autonomous driving relies on foundational technologies such as machine vision, radar positioning, satellite positioning, and intelligent control. Ensuring and continuously improving autonomous driving safety remains a key concern in the autonomous driving field. Summary of the Invention

[0003] The present disclosure provides a behavior data processing method, an autonomous driving vehicle control method, an apparatus, a device, a storage medium, a computer program product, and an autonomous driving vehicle, thereby improving the safety of autonomous driving.

[0004] According to one aspect of the present disclosure, a behavior data processing method is provided, comprising: acquiring historical driving data, the historical driving data including lane-level navigation data; performing data mining on the historical driving data to obtain driving feature information, the driving feature information including at least one of the following: lane change position feature, driving speed feature, and driving path feature.

[0005] According to another aspect of the present disclosure, a control method for an autonomous driving vehicle is provided, comprising: obtaining driving characteristic information, wherein the driving characteristic information is obtained by the above-mentioned behavior data processing method; obtaining vehicle-side perception information, wherein the vehicle-side perception information includes environmental information and positioning information; generating driving decision information based on the driving characteristic information and the vehicle-side perception information; and controlling the autonomous driving vehicle based on the driving decision information.

[0006] According to another aspect of the present disclosure, a behavior data processing device is provided, including: an acquisition module, configured to acquire historical driving data, the historical driving data including lane-level navigation data; a mining module, configured to perform data mining on the historical driving data to obtain driving feature information, the driving feature information including at least one of the following: lane change position feature, driving speed feature, and driving path feature.

[0007] According to another aspect of the present disclosure, a control device for an autonomous driving vehicle is provided, comprising: a first acquisition module configured to acquire driving characteristic information, wherein the driving characteristic information is obtained by the above-mentioned behavior data processing device; a second acquisition module configured to acquire perception and positioning information of the vehicle side; a decision module configured to generate driving decision information based on the driving characteristic information and the perception and positioning information; and a control module configured to control the autonomous driving vehicle based on the driving decision information.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned behavior data processing method or the control method of an autonomous driving vehicle.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the above-mentioned computer instructions are used to enable the above-mentioned computer to execute the above-mentioned behavior data processing method or the control method of the autonomous driving vehicle.

[0010] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements the above-mentioned behavior data processing method or autonomous driving vehicle control method when executed by a processor.

[0011] According to another aspect of the present disclosure, an autonomous driving vehicle is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the control method of the autonomous driving vehicle.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0014] Figure 1 is an exemplary system architecture diagram in which the present disclosure may be applied;

[0015] Figure 2 is a flow chart of an embodiment of a method for processing behavioral data according to the present disclosure;

[0016] Figure 3is a flowchart of another embodiment of the behavior data processing method according to the present disclosure;

[0017] Figure 4 is a flowchart of another embodiment of the behavior data processing method according to the present disclosure;

[0018] Figure 5 is a flowchart of an embodiment of a method for controlling an autonomous vehicle according to the present disclosure;

[0019] Figure 6 is a schematic diagram of a process for generating driving decision information according to the present disclosure;

[0020] Figure 7 is a structural diagram of an embodiment of a behavior data processing device according to the present disclosure;

[0021] Figure 8 is a schematic structural diagram of an embodiment of a control device for an autonomous driving vehicle according to the present disclosure;

[0022] Figure 9 It is a block diagram of an electronic device used to implement the behavior data processing method or the control method of an autonomous driving vehicle according to the embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] Figure 1 An exemplary system architecture 100 is shown to which an embodiment of the behavior data processing method or the control method of an autonomous driving vehicle disclosed herein can be applied.

[0025] like Figure 1 As shown, system architecture 100 may include an autonomous vehicle 101, a conventional vehicle 102, a terminal device 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between autonomous vehicle 101, conventional vehicle 102, terminal device 103, and server 105. Network 103 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0026] Autonomous vehicle 101, standard vehicle 102, and terminal device 103 can interact with server 105 via network 104. Autonomous vehicle 101 can be equipped with various intelligent vehicle-side applications, such as intelligent driving and navigation applications; standard vehicle 102 can be equipped with driving assistance systems and navigation application modules; and terminal device 103 can be equipped with navigation applications and map applications. Server 105 can provide various behavioral data processing services. For example, server 105 can obtain driving or navigation path information from autonomous vehicle 101, standard vehicle 102, and terminal device 103, as well as image data captured by onboard cameras of autonomous vehicle 101 and standard vehicle 102. It can then mine this data to obtain driving feature information and transmit it to autonomous vehicle 101. Autonomous vehicle 101 can generate driving decision information based on the received driving feature information and execute autonomous driving behaviors based on this decision information.

[0027] It should be noted that the terminal device 103 can be hardware or software. When the terminal device 103 is hardware, it can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc. When the terminal device 103 is software, it can be installed in the above-mentioned electronic devices. It can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here. The server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, for providing distributed services), or as a single software or software module. No specific limitation is made here.

[0028] It should be understood that Figure 1 The number of autonomous vehicles, ordinary vehicles, terminal devices, networks, and servers in the present invention is merely illustrative. Any number of autonomous vehicles, ordinary vehicles, terminal devices, networks, and servers may be used as needed.

[0029] It should be noted that the behavior data processing method provided in the embodiments of the present disclosure is generally executed by the server 105, and accordingly, the behavior data processing device is generally installed in the server 105. In addition, in various embodiments of the present disclosure, the behavior data to be processed by the behavior data processing method can be the behavior data generated by the vehicle during previous driving processes, that is, historical driving data.

[0030] Continue to refer Figure 2 , which shows a process 200 of an embodiment of a behavior data processing method according to the present disclosure. The method includes the following steps:

[0031] Step 201: Acquire historical driving data, which includes lane-level navigation data.

[0032] In this embodiment, the execution subject of the behavior data processing method (eg Figure 1 The server 105 shown can first obtain historical driving data, wherein historical driving data refers to driving data generated by various types of vehicles during previous driving processes. This data mainly includes lane-level navigation data, and can also include high-precision map data. In addition to the content of ordinary maps, high-precision maps also store a large amount of driving auxiliary information as structured data. This information can include road data, such as lane information such as the location, type, width, slope and curvature of lane lines, and can also include fixed object information around the lane, such as traffic signs, traffic lights, lane height limits, sewer entrances, obstacles and other road details, as well as infrastructure information such as overhead objects, guardrails, number, road edge type, roadside landmarks, etc. Lane-level navigation data includes navigation trajectory information generated based on high-precision maps.

[0033] In this embodiment, the source of historical driving data is not limited to autonomous vehicles and can also come from navigation applications in ordinary vehicles or terminal devices. For example, after completing each navigation or driving event, the autonomous vehicle can send the generated navigation data to the server as historical driving data. The navigation module or navigation application integrated in ordinary vehicles, as well as the navigation application in the terminal device, can also send the generated navigation data to the server as historical driving data after completing each navigation task.

[0034] Step 202: Perform data mining on historical driving data to obtain driving feature information, where the driving feature information includes at least one of the following: lane change position feature, driving speed feature, and driving path feature.

[0035] In this embodiment, after obtaining historical driving data, the execution subject of the behavior data processing method needs to further mine the historical driving data to obtain the required driving feature information. Specifically, due to the relatively large amount of historical driving data obtained, big data analysis and mining methods, such as classification, regression analysis, clustering, association rules, feature analysis, change and deviation analysis methods, etc., can be used to mine the historical driving data from different perspectives to obtain one or more of the following: lane change position characteristics, driving speed characteristics, and driving path characteristics as components of the driving feature information. Among them, the lane change position characteristics can be used to characterize the lane change position selected by the vehicle when changing lanes, the driving speed characteristics can be used to characterize the vehicle's speed during driving, and the driving path characteristics can be used to characterize the selectable paths between two geographical locations.

[0036] In some optional implementations of this embodiment, the historical driving data also includes vehicle-mounted image data; and the driving feature information includes at least one of the following: dangerous scene features, dynamic event features, and road condition features.

[0037] Specifically, the on-board image data can be image data captured by an on-board camera installed on an autonomous vehicle or an ordinary vehicle while the vehicle is moving. By mining historical driving data including on-board image data, one or more of dangerous scene features, dynamic event features, and road condition features can be further obtained, which are also part of the driving feature information. Among them, dangerous scene features can be used to characterize scenes where accidents and other unexpected events are relatively prone to occur during driving, such as winding mountain road scenes. Dynamic event features can be used to characterize temporary events that will affect the driving process, such as traffic accidents. Road condition features can be used to characterize road surface information such as the material of the traffic road surface that affects the perception of the autonomous driving vehicle, such as a slippery road surface. By mining on-board image data, the comprehensiveness of driving feature information can be further improved.

[0038] The behavioral data processing method provided in the disclosed embodiments first acquires historical driving data, including lane-level navigation data. It then performs data mining on this historical driving data to obtain driving characteristic information. This driving characteristic information includes at least one of the following: lane change location characteristics, driving speed characteristics, and driving path characteristics. By fully mining this historical driving data, accurate and comprehensive driving characteristic information can be obtained.

[0039] Further reference Figure 3 , which shows a process 300 of another embodiment of the behavior data processing method according to the present disclosure. The method includes the following steps:

[0040] Step 301: Acquire historical driving data, which includes lane-level navigation data.

[0041] In this embodiment, the specific operation of step 301 has been Figure 2 Step 201 is described in detail in the illustrated embodiment and will not be repeated here.

[0042] In this embodiment, the driving characteristic information obtained may include one or more of lane change position characteristics, driving speed characteristics, and travel path characteristics. If steps 302-304 are executed, the driving characteristic information obtained may include lane change position characteristics; if steps 305-306 are executed, the driving characteristic information obtained may include driving speed characteristics; if steps 307-308 are executed, the driving characteristic information obtained may include travel path characteristics. The behavioral data processing method of this embodiment may include steps 302-308 simultaneously, or may include one or more of steps 302-304, 305-306, and 307-308.

[0043] Step 302: Acquire lane change data for multiple lane change scenarios from the lane-level navigation data, where the lane change scenarios include intersection lane change scenarios.

[0044] In this embodiment, after obtaining the lane-level navigation data, the execution subject of the behavior data processing method can first filter out lane change data under multiple lane change scenarios from the navigation data. Lane change scenarios can include lane change scenarios at intersections and can also include straight lane change scenarios. Intersections can include crossroads, T-junctions, entrances and exits, U-turns, etc., thereby covering the lane change situations of vehicles in scenarios such as intersections, entrances and exits, and highway ramps. Straight lane change scenarios include situations where vehicles change lanes in straight lanes. Lane change data can include vehicle trajectory information within a predetermined time or predetermined distance before and after the lane change.

[0045] Step 303: Classify the lane change data in each lane change scenario according to the traffic flow level to obtain multiple groups of lane change data.

[0046] In this embodiment, after obtaining lane change data under multiple lane change scenarios, the execution subject of the behavior data processing method can process the lane change data under each lane change scenario separately. Specifically, the lane change data under a lane change scenario can be classified according to the traffic flow level. Among them, the traffic flow level can include three levels: high flow, medium flow, and low flow. Specifically, the daily average traffic volume PV value of the road where the vehicle is located can be used as the basis for classification. For example, the lane change data generated on the road with a PV value greater than 200 can be classified into the high flow group, the lane change data generated on the road with a PV value less than 40 can be classified into the low flow group, and the remaining lane change data can be classified into the medium flow group. In this way, multiple groups of lane change data corresponding to multiple flow levels can be obtained.

[0047] Step 304: Cluster the lane change positions of each set of lane change data to obtain multiple lane change position features.

[0048] In this embodiment, each lane change position feature corresponds to a traffic flow level in a lane change scenario. Specifically, after obtaining multiple lane change data sets, the execution subject of the behavioral data processing method can perform a clustering operation on each data set. Specifically, a common clustering algorithm (such as K-means or hierarchical clustering algorithm) can be used for this clustering operation. The clustering results are then used as lane change position features, thereby obtaining multiple lane change position features. Since each set of lane change data corresponds to a traffic flow level in a lane change scenario, the resulting lane change position features can also be differentiated according to the lane change scenario and traffic flow level.

[0049] By dividing lane-level navigation data according to lane-changing scenarios and traffic flow levels, the resulting lane-changing position features are more targeted and adaptable to various lane-changing situations.

[0050] Step 305: Acquire driving speed data at different geographical locations from the lane-level navigation data.

[0051] In this embodiment, after obtaining lane-level navigation data, the execution subject of the behavior data processing method can obtain the driving speed data of each vehicle at various geographic locations based on the vehicle's driving trajectory and driving speed in the lane-level navigation data. In this way, multiple driving speed data can be obtained at a single geographic location. The geographic location can be a specific coordinate point in a high-precision map.

[0052] Step 306 : Cluster the driving speed data at each geographical location to obtain average speed values, average acceleration values, and extreme speed values ​​at different geographical locations as driving speed features.

[0053] In this embodiment, the execution body of the behavior data processing method can use a clustering algorithm to cluster the driving speed data at each geographical location. Specifically, a density clustering algorithm in machine learning, such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise), can be used for clustering. The obtained clustering results may include average speed values, average acceleration values, and extreme speed values ​​at different geographical locations. The average speed value may be the average speed of multiple vehicles traveling at the geographical location, the average acceleration value may be the average acceleration used when multiple vehicles accelerate or decelerate at the geographical location, and the extreme speed value may be the maximum speed and minimum speed of multiple vehicles traveling at the geographical location. The average speed value, average acceleration value, and extreme speed value at each geographical location can be collectively used as driving speed features.

[0054] By clustering the driving speed data at different geographical locations, the obtained driving speed features can be accurately mapped to the coordinates of each geographical location, thereby improving the reliability of the driving speed features.

[0055] Step 307: Acquire driving path data from the lane-level navigation data.

[0056] In this embodiment, the execution entity of the behavioral data processing method can determine the vehicle's driving trajectory during each navigation session from the lane-level navigation data, and based on the driving trajectory, determine the path the vehicle traveled from the navigation starting point to the navigation end point. The driving path between the two geographic locations thus obtained can be used as the acquired driving path data. It is understood that the driving path is affected by factors such as road traffic conditions during navigation and the preferences of the navigation user. Therefore, the driving path between two specific geographic locations may include a variety of road combinations.

[0057] Step 308: Classify the driving path data to obtain recommended path features and dangerous path features as driving path features.

[0058] In this embodiment, after acquiring the driving path data, the execution subject of the behavior data processing method can use statistical analysis methods in statistics to classify the driving path data, or input the driving path data into a pre-trained classification model for classification. After classifying the driving path data, recommended path features and dangerous path features can be obtained. Among them, the recommended path feature can be used to characterize the road combination pattern that has a high number of driving times between two geographical locations and is often used as a planned path for driving decisions; the dangerous path feature can be used to characterize the road combination pattern that has a low number of driving times between two geographical locations and is often used for dangerous driving behaviors (such as driving against traffic, illegal lane changes, etc.). After classification, for those driving path features that cannot be classified as recommended path features or dangerous path features, other path features can also be formed as part of the driving path features.

[0059] By classifying the driving path data, the obtained driving path features may include recommended path features and dangerous path features, thereby improving the reliability and safety of the driving path features.

[0060] Further reference Figure 4 , which shows a process 400 of another embodiment of a behavior data processing method according to the present disclosure. The processing method includes the following steps:

[0061] Step 401: Acquire historical driving data, which includes vehicle-mounted image data.

[0062] In this embodiment, the specific operation of step 401 has been Figure 2In the embodiment shown, step 201 is described in detail and will not be repeated here.

[0063] In this embodiment, the historical driving data may include vehicle-mounted image data, and may also include lane-level navigation data and high-precision maps.

[0064] The driving characteristic information obtained in this embodiment may include one or more of dangerous scene characteristics, dynamic event characteristics, and road condition characteristics. If the following steps 402-404 are executed, the driving characteristic information obtained may include dangerous scene characteristics; if the following steps 405-406 are executed, the driving characteristic information obtained may include dynamic event characteristics; if the following steps 407-408 are executed, the driving characteristic information obtained may include road condition characteristics. The behavior data processing method of this embodiment may include steps 402-408 at the same time, or may include one or more of 402-404, 405-406, and 407-408. It should be noted that steps 402-408 and Figure 3 In some optional implementations of this embodiment, the behavior data processing method may include one or more of steps 302-304, 305-306, 307-308, 402-404, 405-406, and 407-408.

[0065] Step 402: Extract driving scene data from the vehicle-mounted image data.

[0066] In this embodiment, the execution entity of the behavior data processing method can first convert the video data in the vehicle-mounted image data into image data. For example, the image frames representing the scene in the video data can be retained and the remaining frames can be deleted. Similar filtering can also be performed on the image data in the vehicle-mounted image data. The resulting driving scene data includes multiple images that primarily display the driving scene.

[0067] Step 403: Classify the driving scene data to obtain multiple dangerous scenes.

[0068] In this embodiment, the execution subject of the behavior data processing method can use the image classification method to classify the multiple images included in the driving scene data to obtain multiple dangerous scenes. Among them, dangerous scenes refer to scenes where accidents and other unexpected events are relatively likely to occur during driving, and specifically include curve scenes and lane scenes. Curve scenes can include curve scenes in urban roads, and can also include curve scenes in mountain roads or highways. Lane scenes mainly correspond to scenes where the road is extremely narrow and there are many obstacles on both sides. It can be understood that the examples of specific dangerous scenes mentioned above do not constitute a limitation to this embodiment. Scenes that may affect driving safety, such as bumpy roads and unpaved roads, can all be regarded as dangerous scenes in this embodiment.

[0069] Step 404: Use the image features of each dangerous scene as dangerous scene features.

[0070] In this embodiment, the image classification method in the above step 403 is based on image features. Therefore, after classification, image features corresponding to each dangerous scene can be obtained, and these image features can be used as dangerous scene features.

[0071] In some optional implementations of this embodiment, the executor of the behavior data processing method can directly input the obtained vehicle-mounted image data into an image classification model based on deep learning. Since the model mainly focuses on image features related to dangerous scenes during the previous training process, it can directly classify the vehicle-mounted image data and output dangerous scene features.

[0072] By classifying and identifying driving scenes in vehicle-mounted image data, accurate dangerous scene features can be obtained, improving the comprehensiveness of driving feature information.

[0073] Step 405: Identify dynamic events in the vehicle-mounted image data.

[0074] In this embodiment, dynamic events include at least one of the following: construction events, accident events. Specifically, dynamic events may refer to non-fixed events that affect vehicle travel, such as some temporary events occurring on the road, including construction events, accident events, traffic control events, etc. The executor of the behavioral data processing method in this embodiment, when identifying dynamic events, can use deep learning algorithms to identify information such as accidents and construction signs in images to mine dynamic events. Specifically, the vehicle-mounted image data can be input into a deep learning convolutional neural network algorithm model to perform dynamic event recognition.

[0075] Step 406: Use the recognition result as a dynamic event feature.

[0076] In this embodiment, since the model used in step 405 mainly focuses on image features related to dynamic events during the previous training process, the recognition results output by the model can be directly used as dynamic event features.

[0077] By identifying dynamic events in vehicle-mounted image data, the obtained dynamic event features can be used as part of the driving feature information, further improving the comprehensiveness of the driving feature information.

[0078] Step 407: Extract road surface image data from the vehicle-mounted image data.

[0079] In this embodiment, the executor of the behavior data processing method can first extract road surface image data related to the road surface conditions from the vehicle-mounted image data. For example, the part of the picture where the road surface is located can be cut out from the video or image taken by the vehicle-mounted camera as the road surface image data.

[0080] Step 408: Identify the road surface image data to obtain road surface condition characteristics.

[0081] In this embodiment, the road condition characteristics include at least one of the following: road wear characteristics, road slippery characteristics, road pothole characteristics, and road obstacle characteristics. The execution subject of the behavior data processing method can use a deep learning algorithm to process the road image data to obtain the road condition characteristics.

[0082] In some optional implementations of this embodiment, the executor of the behavior data processing method can directly input the obtained vehicle-mounted image data into an image classification model based on deep learning. Since the model mainly focused on image features related to road conditions during the previous training process, it can directly identify the vehicle-mounted image data and output road condition features.

[0083] By identifying the road conditions in the vehicle-mounted image data, the obtained road condition characteristics can be used as part of the driving characteristic information, further improving the comprehensiveness of the driving characteristic information.

[0084] Further reference Figure 5 , which shows a process 500 of an embodiment of a control method for an autonomous driving vehicle according to the present disclosure, the control method includes the following steps:

[0085] Step 501: Acquire driving characteristic information.

[0086] In this embodiment, the execution subject of the control method (eg Figure 1 The autonomous driving vehicle 101 shown in FIG. 101 may obtain driving characteristic information from the server. The driving characteristic information may be obtained by the server according to Figure 2-Figure 4The driving characteristic information obtained by the behavior data processing method of any embodiment may include one or more of lane change position characteristics, driving speed characteristics, driving path characteristics, dangerous scene characteristics, dynamic event characteristics and road condition characteristics.

[0087] Step 502: Obtain vehicle-side perception and positioning information.

[0088] In this embodiment, the execution subject of the above-mentioned control method can also obtain vehicle-side perception information through sensors set on the vehicle side. Among them, the sensor on the vehicle side can be a point cloud sensor or an image sensor. The point cloud sensor is a sensor that can collect point cloud data, generally a 3D (3-dimension) sensor. The point cloud sensor includes a laser detection and ranging (Light detection and ranging, Lidar) sensor and a radio detection and ranging (Radio detection and ranging, Radar) sensor. The image sensor is a sensor that can collect images, generally a 2D (2-dimension) sensor, such as a camera sensor. After obtaining the image and point cloud data, the perception positioning information can be obtained through the perception positioning algorithm model. The perception positioning information can include static traffic elements such as road markings and obstacles, and dynamic elements such as vehicles and pedestrians.

[0089] Step 503: Generate driving decision information based on the driving feature information and the perception positioning information.

[0090] In this embodiment, the control method executes a multimodal information fusion operation on the driving feature information and the sensor positioning information, and generates driving decision information based on the fused information. The driving decision information can be used to instruct the autonomous vehicle to generate a driving task, perform driving path planning, or perform exception handling.

[0091] In some optional implementations, driving feature information and sensory positioning information can be fused based on pre-defined driving decision rules to generate driving decision information. In other optional implementations, driving feature information and sensory positioning information can be input into a deep learning multimodal information fusion model, and driving decision information can be derived through model inference.

[0092] Step 504: Control the autonomous driving vehicle based on the driving decision information.

[0093] In this embodiment, the execution subject of the control method can control the autonomous driving vehicle to perform autonomous driving tasks, perform path planning, and handle abnormal events based on the generated driving decision information.

[0094] In some optional implementations of this embodiment, the control method 600 also includes obtaining vehicle wireless communication V2X information; the above-mentioned generating driving decision information based on driving characteristic information and perception positioning information includes: generating driving decision information based on driving characteristic information, perception positioning information and V2X information.

[0095] In this embodiment, the execution subject of the control method may also obtain V2X information. V2X (vehicle to X or vehicle to everything) refers to vehicle-to-everything wireless communication technology, also known as vehicle-to-everything communication. It enables vehicles to obtain a series of traffic information such as real-time road conditions, road information, and pedestrian information, thereby improving driving safety, reducing congestion, and improving traffic efficiency. V represents the vehicle, and X represents any object that interacts with the vehicle. Currently, X primarily includes vehicles (V2V), pedestrians (V2P), roadside infrastructure (V2I), and networks (V2N). V2X information may include information about the surrounding environment, such as surrounding infrastructure, other vehicles, and pedestrians.

[0096] When generating driving decision information, the driving feature information, sensor positioning information, and V2X information can be combined to generate the driving decision information, perform multimodal information fusion, and generate the driving decision information based on the fused information. The specific generation method can be referred to above in step 603 and will not be repeated here.

[0097] from Figure 5 As can be seen in the figure, the control method for an autonomous vehicle in this embodiment first obtains driving characteristic information and vehicle-side sensory positioning information. Then, based on the driving characteristic information and sensory positioning information, driving decision information is generated. Finally, based on the driving decision information, the autonomous vehicle is controlled. By using driving characteristic information as the basis for generating driving decision information, driving decision information can be obtained based on historical driving data, thereby improving the safety of autonomous driving.

[0098] For ease of understanding, Figure 6 FIG. 1 is a schematic diagram showing a process for generating driving decision information that can implement an embodiment of the present disclosure. Figure 6As shown, the server mines historical driving data to extract driving characteristics, including lane change location, driving speed, driving path, dangerous scene, dynamic event, and road condition characteristics. Autonomous vehicles can retrieve all these mined driving characteristics from the server and obtain vehicle-side perception information through on-board sensors and vehicle-to-everything (V2X) information through road test equipment. They then perform a multimodal fusion of driving characteristics, vehicle-side perception information, and V2X information to generate driving decision-making information, including mission decision-making, trajectory planning, and exception handling.

[0099] Further references Figure 7 , as a response to the above Figure 2-Figure 4 The present disclosure provides an embodiment of a behavior data processing device. Figure 2-Figure 4 Corresponding to the method embodiment shown, the device can be specifically applied in a server.

[0100] like Figure 7 As shown, the behavior data processing device 700 of this embodiment may include an acquisition module 701 and a mining module 702. The acquisition module 701 is configured to acquire historical driving data, including lane-level navigation data; the mining module 702 is configured to perform data mining on the historical driving data to obtain driving feature information, including at least one of the following: lane change position feature, driving speed feature, and driving path feature.

[0101] In this embodiment, in the behavior data processing device 700, the specific processing of the acquisition module 701 and the mining module 702 and the technical effects thereof can be referred to in Figure 2 The relevant descriptions of steps 201-202 in the corresponding embodiment are not repeated here.

[0102] In some optional implementations of this embodiment, the historical driving data also includes vehicle-mounted image data; the driving feature information includes at least one of the following: dangerous scene features, dynamic event features, and road condition features.

[0103] In some optional implementations of this embodiment, the driving feature information includes lane change position features, and the mining module 702 includes: a first acquisition unit, configured to obtain lane change data in multiple lane change scenarios from lane-level navigation data, and the lane change scenarios include intersection lane change scenarios; a traffic classification unit, configured to classify the lane change data in each lane change scenario according to the traffic flow level to obtain multiple groups of lane change data; a position clustering unit, configured to perform lane change position clustering on each group of lane change data to obtain multiple lane change position features, wherein each lane change position feature corresponds to a type of traffic flow level in a lane change scenario.

[0104] In some optional implementations of this embodiment, the driving feature information includes a driving speed feature, and the mining module 702 includes: a second acquisition unit, configured to obtain driving speed data at different geographical locations from the lane-level navigation data; a speed clustering unit, configured to cluster the driving speed data at each geographical location respectively, and obtain the average speed value, average acceleration value and extreme speed value at different geographical locations as the driving speed feature.

[0105] In some optional implementations of this embodiment, the driving feature information includes driving path features, and the mining module 702 includes: a third acquisition unit, configured to obtain driving path data from lane-level navigation data; a path classification unit, configured to classify the driving path data to obtain recommended path features and dangerous path features as driving path features.

[0106] In some optional implementations of this embodiment, the driving feature information includes dangerous scene features, and the mining module 702 includes: a fourth acquisition unit, configured to extract driving scene data from the on-board image data; a scene classification unit, configured to classify the driving scene data to obtain multiple dangerous scenes, and the multiple dangerous scenes include curve scenes and alley scenes; a feature determination unit, configured to use the image features under each dangerous scene as dangerous scene features.

[0107] In some optional implementations of this embodiment, the driving feature information includes dynamic event features, and the mining module 702 includes: an event recognition unit, configured to recognize dynamic events in the vehicle-mounted image data, and the dynamic events include at least one of the following: construction events, accident events; and an event determination unit, configured to use the recognition results as dynamic event features.

[0108] In some optional implementations of this embodiment, the driving feature information includes road condition features, and the mining module 702 includes: a fifth acquisition unit, configured to extract road image data from the vehicle-mounted image data; a road recognition unit, configured to recognize the road image data to obtain road condition features, and the road condition features include at least one of the following: road wear features, road wet features, road pothole features, and road obstacle features.

[0109] Further references Figure 8 , as a response to the above Figure 5 The present disclosure provides an embodiment of a control device for an autonomous vehicle. Figure 5 Corresponding to the method embodiment shown, the device can be specifically applied to an autonomous driving vehicle.

[0110] like Figure 8As shown, the control device 800 of this embodiment may include a first acquisition module 801, a second acquisition module 802, a decision module 803 and a control module 804. The first acquisition module 801 is configured to acquire driving feature information, wherein the driving feature information is obtained by Figure 7 The behavior data processing device shown is obtained; the second acquisition module 802 is configured to obtain the perception and positioning information of the vehicle side; the decision module 803 is configured to generate driving decision information based on the driving feature information and the perception and positioning information; the control module 804 is configured to control the autonomous driving vehicle based on the driving decision information.

[0111] In this embodiment, the specific processing of the first acquisition module 801, the second acquisition module 802, the decision module 803 and the control module 804 and the technical effects thereof can be referred to in the respective Figure 5 The relevant descriptions of steps 501-504 in the corresponding embodiment are not repeated here.

[0112] The control device 800 also includes: a third acquisition module, configured to obtain vehicle wireless communication V2X information; the decision module 803 includes: a decision unit, configured to generate driving decision information based on driving feature information, perception positioning information and V2X information.

[0113] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, a computer program product and an autonomous driving vehicle.

[0114] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0115] like Figure 9As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0116] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0117] The computing unit 901 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the behavioral data processing method or the control method of an autonomous vehicle. For example, in some embodiments, the behavioral data processing method or the control method of an autonomous vehicle can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the behavioral data processing method or the control method of an autonomous vehicle described above can be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the behavioral data processing method or the control method of the autonomous driving vehicle in any other appropriate manner (e.g., by means of firmware).

[0118] The autonomous driving vehicle provided by the present disclosure may include: Figure 9 The above-mentioned electronic device shown can implement the control method of the automatic driving vehicle described in the above embodiment when its processor executes it.

[0119] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0120] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0121] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0123] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0124] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a server in a distributed system or a server integrated with blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server may be a server in a distributed system or a server integrated with blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0125] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0126] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0127] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for processing behavioral data, the method comprising: Acquiring historical driving data including lane-level navigation data and high-precision map data, wherein the high-precision map data includes road data and information about fixed objects around the lanes; Data mining is performed on the historical driving data to obtain driving feature information, including: Acquiring lane change data for multiple lane change scenarios from the lane-level navigation data, the lane change scenarios including intersection lane change scenarios, the lane change data including vehicle trajectory information within a predetermined time or predetermined distance before and after the lane change; classifying the lane change data for each lane change scenario according to a traffic flow level to obtain multiple groups of lane change data; clustering each group of lane change data by lane change position to obtain multiple lane change position features, wherein each lane change position feature corresponds to a type of traffic flow level for each lane change scenario; Acquiring driving speed data at different geographical locations from the lane-level navigation data; clustering the driving speed data at each geographical location to obtain average speed values, average acceleration values, and extreme speed values ​​at the different geographical locations as driving speed features; Acquiring driving path data from the lane-level navigation data; classifying the driving path data to obtain recommended path features and dangerous path features as driving path features, wherein the recommended path features are used to characterize a road combination pattern that has been frequently traveled between two geographic locations and is frequently used as a historically planned route for driving decisions; and the dangerous path features are used to characterize a road combination pattern that has been rarely traveled between two geographic locations and is frequently used as a route for dangerous driving behaviors; the driving feature information includes: the lane change position feature, the driving speed feature, and the driving path feature. Obtain vehicle-side perception information through sensors installed on the vehicle side; obtain vehicle wireless communication V2X information.

2. The method according to claim 1, wherein The driving characteristic information also includes at least one of the following: dangerous scene characteristics, dynamic event characteristics, and road condition characteristics.

3. The method according to claim 2, wherein: The data mining of the historical driving data to obtain driving feature information includes: Extract driving scene data from vehicle-mounted image data; The driving scene data is classified to obtain a plurality of dangerous scenes, wherein the plurality of dangerous scenes include curve scenes and lane scenes; and the image features of each dangerous scene are used as dangerous scene features.

4. The method according to claim 2, wherein: The driving characteristic information includes the dynamic event characteristics, and the data mining of the historical driving data to obtain the driving characteristic information includes: Identifying dynamic events in vehicle-mounted image data, wherein the dynamic events include at least one of the following: a construction event and an accident event; The recognition results are used as dynamic event features.

5. The method according to claim 2, wherein: The driving characteristic information includes the road condition characteristics, and the data mining of the historical driving data to obtain the driving characteristic information includes: Extracting road surface image data from vehicle-mounted image data; The road surface image data is identified to obtain road surface condition characteristics, where the road surface condition characteristics include at least one of the following: road surface wear characteristics, road surface slippery characteristics, road surface pothole characteristics, and road surface obstacle characteristics.

6. A method for controlling an autonomous vehicle, the method comprising: Acquiring driving characteristic information, wherein the driving characteristic information is obtained by the method according to any one of claims 1 to 5; Obtain vehicle-side perception and positioning information; generating driving decision information based on the driving feature information and the perception positioning information; Based on the driving decision information, the autonomous driving vehicle is controlled.

7. The method according to claim 6, further comprising: Obtain vehicle wireless communication V2X information; The generating of driving decision information based on the driving feature information and the perception positioning information includes: Driving decision information is generated based on the driving characteristic information, perception positioning information and V2X information.

8. A behavioral data processing device, comprising: an acquisition module configured to acquire historical driving data including lane-level navigation data and high-precision map data, wherein the high-precision map data includes road data and fixed object information around the lane; The mining module is configured to perform data mining on the historical driving data to obtain driving feature information, including: a first acquisition unit configured to acquire lane change data for a plurality of lane change scenarios from the lane-level navigation data, the lane change scenarios including intersection lane change scenarios, the lane change data including driving trajectory information of a vehicle within a predetermined time or predetermined distance before and after the lane change; a traffic flow classification unit configured to classify the lane change data for each lane change scenario according to a traffic flow level to obtain a plurality of groups of lane change data; and a position clustering unit configured to cluster lane change positions for each group of lane change data to obtain a plurality of lane change position features, wherein each lane change position feature corresponds to a type of traffic flow level for a lane change scenario; a second acquisition unit configured to acquire driving speed data at different geographical locations from the lane-level navigation data; cluster the driving speed data at each geographical location to obtain average speed values, average acceleration values, and extreme speed values ​​at the different geographical locations as driving speed features; a third acquisition unit configured to acquire driving path data from the lane-level navigation data; a path classification unit configured to classify the driving path data to obtain recommended path features and dangerous path features as driving path features, wherein the recommended path features are used to characterize road combinations that are frequently traveled between two geographic locations and are frequently used as historically planned routes for driving decisions; and the dangerous path features are used to characterize road combinations that are rarely traveled between two geographic locations and are frequently used for dangerous driving behaviors. The driving feature information includes: the lane change position feature, the driving speed feature, and the driving path feature. The sixth acquisition unit is configured to obtain vehicle-side perception information through sensors set on the vehicle side; and obtain vehicle wireless communication V2X information.

9. The device according to claim 8, wherein The driving characteristic information also includes at least one of the following: dangerous scene characteristics, dynamic event characteristics, and road condition characteristics.

10. The device according to claim 9, wherein The driving feature information includes the dangerous scene feature, and the mining module includes: A fourth acquisition unit is configured to extract driving scene data from the vehicle-mounted image data; a scene classification unit configured to classify the driving scene data to obtain a plurality of dangerous scenes, wherein the plurality of dangerous scenes include a curve scene and a lane scene; The feature determination unit is configured to use the image features in each dangerous scene as dangerous scene features.

11. The device according to claim 9, wherein the driving feature information includes the dynamic event feature, and the mining module comprises: An event recognition unit is configured to recognize dynamic events in the vehicle-mounted image data, wherein the dynamic events include at least one of the following: a construction event and an accident event; The event determination unit is configured to use the recognition result as a dynamic event feature.

12. The apparatus according to claim 9, wherein the driving characteristic information includes the road condition characteristics, and the mining module comprises: a fifth acquisition unit configured to extract road surface image data from the vehicle-borne image data; The road surface recognition unit is configured to recognize the road surface image data to obtain road surface condition characteristics, wherein the road surface condition characteristics include at least one of the following: road surface wear characteristics, road surface slippery characteristics, road surface pothole characteristics, and road surface obstacle characteristics.

13. A control device for an autonomous driving vehicle, the device comprising: A first acquisition module is configured to acquire driving characteristic information, wherein the driving characteristic information is obtained by the device according to any one of claims 8 to 12; A second acquisition module is configured to acquire the perception and positioning information of the vehicle side; a decision module, configured to generate driving decision information based on the driving feature information and the perception positioning information; The control module is configured to control the autonomous driving vehicle based on the driving decision information.

14. The apparatus according to claim 13, further comprising: A third acquisition module is configured to acquire vehicle wireless communication V2X information; The decision module includes: The decision unit is configured to generate driving decision information based on the driving feature information, the perception positioning information and the V2X information.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

18. An autonomous driving vehicle comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of claim 6 or 7.

Images