Method and system for mining automatic driving perception data under any condition

Abstract

The invention relates to an arbitrary condition automatic driving perception data mining method and system, and the method comprises the following steps: S1, taking a perception data file collected by an automatic driving vehicle as an input, carrying out the further splitting of perception object information based on an arbitrary condition query purpose, and processing a plurality of labels, taking the automatic driving vehicle, the sensing acquisition time and the data corresponding to the plurality of labels as a record, and generating an automatic driving sensing result detail table; s2, loading an automatic driving perception result detail list into an analysis type database which is stored in a determinant manner based on an automatic driving vehicle, perception acquisition time and a plurality of labels; and S3, inputting a general query model of any condition to query the analysis type database based on column storage, and obtaining required automatic driving perception scene data. The method has the beneficial effect of flexible and efficient automatic driving perception data mining.

Description

[Technical field]

[0001] The present invention relates to the field of computer database technology, and in particular to a method and system for mining perception data of autonomous driving under arbitrary conditions. [Background technology]

[0002] Figure 1 This is a schematic diagram of the location of the cloud perception data mining system in the process of autonomous driving development. Figure 1 As shown in the figure, in the process of autonomous driving R&D, the cloud perception data mining system is at the center. It is crucial to mine scene data sets covering various road conditions and driving situations based on the autonomous driving perception (data) results. It provides sufficient perception scene data support for key R&D links such as algorithms, simulations and tests in the downstream R&D. Typical perception scenes that need to be mined based on autonomous driving perception (data) results include:

[0003] Scenario 1: There is a child within 20 meters of the vehicle.

[0004] Scenario 2: There are 10 pedestrians within 30 meters in front of the vehicle;

[0005] Scenario 3: There are more than 5 and less than 10 electric vehicles with a speed exceeding 5 m / s within a 45° to 90° range behind the vehicle.

[0006] …

[0007] Figure 2 This is a schematic diagram of the traditional cloud perception data mining system for autonomous driving perception data mining. Figure 2 As shown in the figure, the traditional cloud perception data mining system performs autonomous driving perception data mining by processing some fixed data tags through scripting languages ​​such as Python by data users (such as algorithm developers, test developers, or simulation scenario library developers). Then the data developers query the autonomous driving perception database based on the processed fixed tags and the conditions that the fixed tags need to meet to obtain the autonomous driving perception scene data required by the data users. For example, for the perception scene data requirements of the above scenarios 1 to 3, the data user will develop the following fixed tags and write them into the "Perception Tag Table":

[0008] Label 1: The number of children within 20 meters of the vehicle;

[0009] Label 2: the number of pedestrians within 30 meters in front of the vehicle;

[0010] Label 3: The number of electric vehicles with a speed exceeding 5m / s in the range of 45° to 90° from the side and rear of the vehicle;

[0011] …

[0012] Based on these fixed tags, data users can perform autonomous driving perception data mining by setting corresponding conditions to obtain the corresponding autonomous driving perception scenarios:

[0013] Scenario 1: Tag 1>0;

[0014] Scenario 2: Tag 2>10;

[0015] Scenario 3: Tag3>=5and Tag3<=10;

[0016] …

[0017] The above method of mining and processing fixed labels based on autonomous driving perception data, the scene is divided into the relative fixed amount of perception objects + the number of perception objects, the perception object refers to the obstacles other than the road surface perceived by the vehicle's sensors during autonomous driving, and the relative fixed amount of perception objects refers to the physical parameters of the perception object relative to the vehicle; when processing labels, only the relative fixed amount of perception objects is processed to obtain fixed labels, and the corresponding perception database is counted based on the relative fixed amount of perception objects, which can store perception data covering the total number of perception objects. The conditions entered during the query are: fixed label (relative fixed amount of perception objects) + specific number of perception objects, and the required perception scene data is obtained. The biggest problem with this method is that it has very poor flexibility. If the R&D personnel are concerned about the mining conditions from "there are children within 20 meters around the vehicle" to "there are children within 30 meters around the vehicle", then a new label needs to be processed to support this scenario:

[0018] Label 4: the number of children within 30 meters of the vehicle;

[0019] Scenario 4: Tag 4>0.

[0020] This is a very inflexible approach that can only support data mining under fixed conditions (relatively fixed amounts of specific data labels for perceived objects), and is unlikely to support the need for massive data sets covering a variety of scenarios during the development of autonomous driving.

[0021] The present invention aims to solve the technical problem that the existing autonomous driving perception data mining has poor query flexibility by processing fixed tags, and makes technical improvements to the autonomous driving perception data mining method and system. [Summary of the invention]

[0022] The purpose of the present invention is to provide a flexible and efficient autonomous driving perception data mining method.

[0023] To achieve the above object, the technical solution adopted by the present invention is a method for mining perception data of autonomous driving under arbitrary conditions, comprising the following steps:

[0024] S1. Taking the perception data file collected by the autonomous driving vehicle as input, further split the perception object information based on any condition query purpose, process several tags, and take the autonomous driving vehicle, perception collection time and several tags corresponding data as one record to generate a detailed table of autonomous driving perception results;

[0025] S2. Load the detailed table of autonomous driving perception results into an analytical database that performs column storage based on the autonomous driving vehicle, perception collection time, and a number of tags;

[0026] S3. Input a general query model with any conditions to query the analytical database based on column storage to obtain the required autonomous driving perception scene data.

[0027] Preferably, the perception objects are various obstacles around the autonomous driving vehicle identified by the perception model.

[0028] Preferably, the several labels for splitting and processing the perceived object information in step S1 include perceived object id, type, subtype, speed, distance, and azimuth; the perceived object id label uniquely identifies the id of a perceived object, the type label identifies the type of the perceived object, the subtype label further subdivides the type into specific categories, the speed label identifies the movement speed of the perceived object, the distance label identifies the distance between the perceived object and the autonomous driving vehicle, and the azimuth label identifies the position of the perceived object in the autonomous driving vehicle.

[0029] Preferably, in the column storage in step S2, data is stored according to columns as the basic logical storage unit, and data in a column exists in the storage medium in a continuous storage form.

[0030] Preferably, the general query model of step S3 is: type in (input type), and subtype in (input subtype), and speed∈input speed range, and distance∈input distance range, and azimuth∈input azimuth range, and count(*)∈input quantity range; the user inputs the type, subtype, speed range, distance range, azimuth range, and quantity range of the required autonomous driving perception scene to obtain the required autonomous driving perception scene data.

[0031] Preferably, the label processed in step S2 also includes a travel direction and a predicted collision time; the travel direction label identifies the travel direction of the perceived object, and the predicted collision time label identifies the predicted collision time between the perceived object and the autonomous driving vehicle.

[0032] Another object of the present invention is to provide a flexible and efficient autonomous driving perception data mining system.

[0033] To achieve the above-mentioned yet another objective, the technical solution adopted by the present invention is an arbitrary condition autonomous driving perception data mining system, which is used to execute the above-mentioned arbitrary condition autonomous driving perception data mining method.

[0034] Preferably, the arbitrary condition autonomous driving perception data mining system is a cloud-based autonomous driving perception data mining system deployed on the cloud.

[0035] Preferably, the perception data file is a scene data set based on the perception data results uploaded to the cloud by the autonomous driving vehicle, covering various road conditions and driving situations.

[0036] Preferably, the required autonomous driving perception scene data is used for the algorithm, simulation and testing links downstream of autonomous driving research and development.

[0037] The beneficial effects of the method and system for mining autonomous driving perception data under arbitrary conditions of the present invention are as follows: by directly querying the detailed data of the autonomous driving perception results, the data mining needs of arbitrary conditions are met; at the same time, the introduction of an analytical database based on column storage can enable the query of massive data to respond in seconds; thereby making the mining of autonomous driving perception data both flexible and efficient.

Brief Description of the Drawings

[0038] Figure 1 This is a schematic diagram of the position of the cloud perception data mining system in the process of autonomous driving development.

[0039] Figure 2 This is a schematic diagram of the traditional cloud-based perception data mining system for autonomous driving perception data mining.

[0040] Figure 3 It is a step diagram of the method for mining autonomous driving perception data under arbitrary conditions.

[0041] Figure 4 It is a schematic diagram of the structure of an arbitrary condition autonomous driving perception data mining system. [Specific implementation method]

[0042] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In the detailed description below, many specific details are proposed to provide a comprehensive understanding of the present invention. But it is obvious to those skilled in the art that the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention. The present invention is by no means limited to any specific configuration and algorithm proposed below, but covers any modification, replacement and improvement of elements, parts and algorithms without departing from the concept of the present invention. In the accompanying drawings and the following description, known structures and technologies are not shown to avoid unnecessary ambiguity to the present invention.

[0043] Example 1

[0044] This embodiment implements a method for mining perception data for autonomous driving under arbitrary conditions.

[0045] The method of this embodiment mainly uses a high-performance analytical database to store ultra-large-scale autonomous driving perception target data, thereby realizing the mining of arbitrary conditions of perception scenes. Its flexibility is far superior to traditional autonomous driving data mining based on fixed labels.

[0046] Explanation of important terms in this embodiment method:

[0047] Perception data mining under fixed conditions: The autonomous driving scene is divided into the relative fixed quantity of perception objects + the number of perception objects. Perception objects refer to physical bodies other than the road surface perceived by the vehicle's sensors during autonomous driving. Perception objects refer to the relative fixed quantity of the perception objects relative to the vehicle's physical parameters. When processing labels, only the relative fixed quantity of the perception objects is used to obtain fixed labels. The corresponding perception database is counted based on the relative fixed quantity of the perception objects, and can store perception data covering the total number of perception objects. The conditions entered during the query are: fixed labels (relative fixed quantity of perception objects) + the specific number of perception objects to obtain the required autonomous driving perception scene data.

[0048] The above data mining is based on fixed labels and can only achieve limited-dimensional autonomous driving perception data mining. For example, "the number of pedestrians within 20 meters around" can only mine the number of pedestrians within 20 meters around. If you want to limit the investigation to within 10 meters or 30 meters around, you can only do it by adding perception labels such as "the number of pedestrians within 10 meters around" or "the number of pedestrians within 300 meters around". Adding perception labels is a process that consumes R&D resources, so autonomous driving perception data mining based on fixed labels is a very inefficient method.

[0049] Perception data mining under arbitrary conditions: further split the perception objects and relative fixed quantities of the autonomous driving perception scenes during the autonomous driving process, so as to realize the query of autonomous driving perception data under arbitrary conditions and flexibly obtain the required autonomous driving data scene data.

[0050] The method of this embodiment abandons fixed perception data labels, directly obtains a perception result list through autonomous driving perception data, and queries the perception result list. The obstacle category, speed, direction, distance and other parameters of interest can be arbitrarily input to achieve flexible autonomous driving perception scene data mining.

[0051] Column storage: Compared with the row storage used in traditional relational databases, column storage is often used in analytical databases. In row storage, data is stored based on row data as the basic logical storage unit, and a row of data exists in the storage medium in a continuous storage form. In column storage, data is stored based on column as the basic logical storage unit, and the data in a column exists in the storage medium in a continuous storage form.

[0052] Row storage is suitable for OLTP scenarios: most operations are based on entities, that is, most operations involve adding, deleting, modifying, and querying an entire row of records. Obviously, storing a row of data in a physically adjacent location is a good choice.

[0053] Column storage is suitable for OLAP scenarios: a typical query needs to traverse certain columns of the entire table to perform grouping, sorting, aggregation and other operations, so column storage can greatly reduce the amount of data involved in data query. At the same time, since the data repetition rate of the same data column is very high, columnar data has great advantages in compression.

[0054] Transactional databases (OLTP) and analytical databases (OLAP) are two major branches of the database. The former is good at processing online real-time business (represented by Oracle and MySQL), and the latter is good at processing offline analytical business (represented by Teradata, Greenplum, ClickHouse, Aliyun ADB, etc.).

[0055] Figure 3 This is a step diagram of the method for mining autonomous driving perception data under arbitrary conditions. Figure 3 As shown in the figure, it mainly includes three steps: generating a detailed table of perception results, an analytical database based on column storage of an analytical database, and arbitrary condition query based on a general query model:

[0056] S1. Taking the perception data file collected by the autonomous driving vehicle as input, further split the perception object information based on any condition query purpose, process several tags, and take the autonomous driving vehicle, perception collection time and several tags corresponding data as one record to generate a detailed table of autonomous driving perception results;

[0057] S2. Load the detailed table of autonomous driving perception results into an analytical database based on column storage;

[0058] S3. Input a general query model with any conditions to query the analytical database based on column storage to obtain the required autonomous driving perception scene data.

[0059] 1. Perception result details table: used to store various obstacles around the autonomous vehicle (such as pedestrians, vehicles, traffic facilities, green belts, etc.) identified by the perception model. It takes the perception result file collected by the autonomous vehicle and stored in the cloud as input, and processes the following fields through Python scripts:

[0060] Vehicle id: identifies which vehicle this is.

[0061] Time: The timestamp of the moment the perception occurs, which can be accurate to seconds or milliseconds.

[0062] Perception object information: All obstacle information within the perception range around the vehicle at that moment, which may include several to dozens of records. Specifically, it includes the perception object ID, type, subtype, speed, distance, and azimuth.

[0063] Perception object id: an id that uniquely identifies an object.

[0064] Type: The type of object, such as pedestrians, vehicles, low obstacles, etc.

[0065] Subtype: A further subdivided type. For example, pedestrians can be divided into specific categories such as children, elderly, and adults; vehicles can be divided into cars, trucks, and construction vehicles.

[0066] Speed: The speed of the perceived object's movement.

[0067] Distance: The distance between the perceived object and the vehicle.

[0068] Azimuth: The orientation of the perceived object relative to the vehicle.

[0069] This table can also be expanded to include other information such as the heading (direction of travel) of the perceived object, the predicted collision time between the perceived object and the vehicle, etc.

[0070] 2. Analytical database based on column storage. The "Perception Result Detail Table" will be loaded into an analytical database (such as Presto, ClickHouse, ADB, etc. commonly used in the industry). This database is based on column storage and can carry tens of billions or even hundreds of billions of massive data. At the same time, it provides query speeds of seconds or even sub-seconds, providing fast response for scenario mining.

[0071] 3. General query mode. In traditional perception scene mining, the fields of the "perception tag table" to be queried for each scene are different; however, the query based on the "perception result detail table" table can realize a general query mode:

[0072] type in (input type) and subtype in (input subtype)

[0073] and speed∈input speed range

[0074] and distance∈input distance range

[0075] and azimuth ∈ input azimuth range

[0076] and count(*)∈input quantity range

[0077] The user only needs to enter the following content for the scenario he is concerned about:

[0078] Input Type

[0079] Input Subtype

[0080] Input speed range

[0081] Input distance range

[0082] Input azimuth range

[0083] Input quantity range

[0084] For example, for the complex scenario "Scenario 3: There are more than 5 and less than 10 electric vehicles with a speed exceeding 5 m / s within a 45° to 90° range behind the vehicle", you only need to enter:

[0085] Input Type: Vehicle

[0086] Input Subtype: Electric Vehicle

[0087] Input speed range: [5, +∞)

[0088] Input distance range: No need to input, default is (-∞,+∞)

[0089] Input azimuth range: [-90, -45]

[0090] Input quantity range: [5,10]

[0091] The above general query pattern can generate corresponding query statements, thereby mining the corresponding scenarios. If the user's needs become: "Scenario 4: There are more than 10 electric vehicles with a speed less than 3m / s within 45° to 90° in front of the vehicle", then only the corresponding input data needs to be modified, without the need to develop an additional perception tag, thereby greatly improving the flexibility of the system and realizing perception data mining under any conditions.

[0092] The method of this embodiment uses the "perception result list" and the general query mode to input the obstacle category, speed, direction, distance and other parameters of interest to realize scene mining of arbitrary conditions, thereby greatly improving the flexibility of the perception data mining system.

[0093] The method of this embodiment stores the perception result detail table through an analytical database based on column storage, which can achieve a response to data queries in seconds, thereby greatly accelerating the speed of data mining.

[0094] Example 2

[0095] This embodiment implements an arbitrary condition autonomous driving perception data mining system.

[0096] Figure 4 This is a schematic diagram of the structure of an arbitrary condition autonomous driving perception data mining system. Figure 4 As shown, the system of this embodiment includes a cloud-based perception data mining system deployed on the cloud, and the cloud-based perception data mining system is used to execute an arbitrary condition autonomous driving perception data mining method described in Example 1.

[0097] In the process of autonomous driving research and development, the cloud-based perception data mining system of this embodiment is at the center. Based on the autonomous driving perception (data) results, scenario data sets covering various road conditions and driving situations are mined. Vehicle information, planning data, chassis data, etc. are uploaded to the cloud to generate perception result files, which are input into the cloud-based perception data mining system. The key research and development links such as downstream algorithms, simulations, and tests are queried under any conditions based on the cloud-based perception data mining system of this embodiment to obtain sufficient autonomous driving perception scenario data support that meets the needs.

[0098] The system of this embodiment introduces an analytical database based on column storage by using a perception result detail table, and realizes arbitrary condition mining of perception scenarios through a general query mode, greatly improving the flexibility and response speed of the system.

[0099] Those skilled in the art will appreciate that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0100] The above is only a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and supplements without departing from the principle of the present invention. These improvements and supplements should also be regarded as the scope of protection of the present invention.

Claims

1. A method for mining perception data of autonomous driving under arbitrary conditions, characterized in that The following steps are involved: S1. Taking the perception data file collected by the autonomous driving vehicle as input, further split the perception object information based on any condition query purpose, process several tags, and take the autonomous driving vehicle, perception collection time and several tags corresponding data as one record to generate a detailed table of autonomous driving perception results; S2. Load the detailed table of autonomous driving perception results into an analytical database that performs column storage based on the autonomous driving vehicle, perception collection time, and a number of tags; S3. Input a general query model with any conditions to query the analytical database based on column storage to obtain the required autonomous driving perception scene data.

2. The method for mining perception data of autonomous driving under arbitrary conditions according to claim 1, characterized in that: The perception objects are various obstacles around the autonomous driving vehicle identified by the perception model.

3. The method for mining perception data of autonomous driving under arbitrary conditions according to claim 2, characterized in that: In step S1, several labels for splitting and processing the perceived object information include perceived object id, type, subtype, speed, distance, and azimuth; the perceived object id label uniquely identifies the id of a perceived object, the type label identifies the type of the perceived object, the subtype label further subdivides the type into specific categories, the speed label identifies the movement speed of the perceived object, the distance label identifies the distance between the perceived object and the autonomous driving vehicle, and the azimuth label identifies the position of the perceived object in the autonomous driving vehicle.

4. The method for mining perception data of autonomous driving under arbitrary conditions according to claim 3, characterized in that: In the column storage described in step S2, data is stored based on columns as logical storage units, and data in a column exists in a continuous storage form in the storage medium.

5. The method for mining perception data of autonomous driving under arbitrary conditions according to claim 4, characterized in that The general query model of step S3 is: type in (input type), and subtype in (input subtype), and speed∈input speed range, and distance∈input distance range, and azimuth∈input azimuth range, and count(*)∈input quantity range; the user inputs the type, subtype, speed range, distance range, azimuth range, and quantity range according to the required autonomous driving perception scene to obtain the required autonomous driving perception scene data.

6. The method for mining perception data of autonomous driving under arbitrary conditions according to claim 5, characterized in that: The labels processed in step S2 also include a travel direction and a predicted collision time; the travel direction label identifies the travel direction of the perceived object, and the predicted collision time label identifies the predicted collision time between the perceived object and the autonomous driving vehicle.

7. An arbitrary condition autonomous driving perception data mining system, characterized by: Used to execute an arbitrary condition autonomous driving perception data mining method as described in any one of claims 1 to 5.

8. The arbitrary condition autonomous driving perception data mining system according to claim 7, characterized in that: It is a cloud-based autonomous driving perception data mining system deployed on the cloud.

9. The arbitrary condition autonomous driving perception data mining system according to claim 8, characterized in that: The perception data file is a scene data set based on the perception data results uploaded by the autonomous driving vehicle to the cloud, covering various road conditions and driving situations.

10. The arbitrary condition autonomous driving perception data mining system according to claim 9, characterized in that: The required autonomous driving perception scenario data is used for the algorithm, simulation and testing links downstream of autonomous driving research and development.

Images