A self-identification method and storage medium for autonomous driving scenes

Abstract

The present invention relates to a method for self-identification of autonomous driving scenarios and a storage medium. The method includes: data collection and storage, data cleaning, event and scenario identification and tracking, data cutting, classification and labeling, format conversion and data use. The present invention automatically identifies and extracts all collected vehicle network signals and sensor raw signal data, converts the format, and manages the labeled classification and storage. The scenario library established based on the hierarchical classification of real vehicle data covers natural traffic flow scenarios, autonomous driving failure conditions, driver takeover data, vehicle driving data, conditions outside the operating area, traffic accidents, traffic participant interaction data, and human-computer interaction data. It can provide big data support for autonomous vehicle product definition, function planning, demand analysis, software and hardware development, system integration, simulation and real vehicle verification, rapid iteration and update of functions and performance, and formulation of autonomous driving standards and regulations, thereby improving the utilization efficiency and reuse rate of data.

Description

Technical Field

[0001] The present invention belongs to the technical field of data acquisition and analysis, and specifically relates to a self-identification method and storage medium for autonomous driving scenarios. Background Art

[0002] The diversity, complexity, uncertainty, and lack of full reproducibility of traffic scene elements such as road types, lane markings, signs, and obstacles make it difficult to test and verify autonomous driving functions in all traffic scenarios, or even incur significant costs. This is also a major constraint on achieving true mass production of autonomous driving. Continuously iterating and updating software functionality and performance through data-driven development is undoubtedly the most cost-effective, fastest-paced, and most self-sustaining approach.

[0003] To ensure the safety of autonomous driving functions, any software optimization and upgrade requires hundreds or even thousands of kilometers of simulation and real-world testing in a single scenario. Large-scale data collection and automatic scenario classification, ensuring data support for subsequent problem analysis, simulation scenario construction, deep learning network optimization, and iterative upgrades of autonomous driving models, have become pressing challenges in the development of autonomous vehicles.

[0004] Currently, there are three main sources of data for autonomous driving test scenarios: the first type is virtual road traffic scenarios designed based on commercial software such as Prescan, Carsim, and VTD. These virtual scenarios are quite different from actual traffic scenarios, and it is difficult to characterize the actual performance of on-board sensors and execution. This type of data is usually used to verify whether the functional logic of software modules is normal; the second type of data comes from the laboratory, using physical sensors, but the surrounding environment is not the actual road traffic environment, and the deviation of this type of data is also large; the third type of data comes from the data generated during the actual operation of the vehicle, which reproduces the vehicle status, surrounding road environment, and driver behavior at any moment. However, the diversity and complexity of real road traffic scenarios, and the large amount of original images and point cloud data require that to truly realize data-driven development, it is necessary to solve the problems of efficient processing of real vehicle collected data cleaning, event and scene extraction, and labeled classification management. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a method and storage medium for self-identification of autonomous driving scenarios, so as to avoid the low feasibility of traditional real-vehicle data collection methods when facing positioning and analysis problems, and to achieve the effect of improving data utilization efficiency and reuse rate.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A method for self-identifying an autonomous driving scene, comprising the following steps:

[0008] S1: Data acquisition and storage, uploading vehicle network signals and sensor raw data from acquisition vehicles, test vehicles, production vehicles, and third-party platforms to the cloud storage server;

[0009] S2: Data cleaning: Check the integrity and continuity of the data in the cloud storage server in step S1, and remove damaged and invalid data;

[0010] S3: Event and scene recognition and tracking. Based on the logical relationship of vehicle network bus signals, it automatically identifies various events and traffic scenarios during driving and tracks the events and scenes of interest.

[0011] S4: Data cutting: Cut the data file according to the location of the buried points in step S3, and only retain the data with complete information within 5 seconds before and after the accident;

[0012] S5: Classification and labeling: automatically label the data retained after step S4 according to the classification and labeling rules of events and scenes;

[0013] S6: Format conversion, performing data conversion on the data file after labeling processing in step S5;

[0014] S7: Data usage: Use the data file obtained in step S6 for problem analysis, simulation scenario construction and testing, back-injection testing, AI training and testing, and map data production and updating.

[0015] To further improve the above technical solution, in step S1, the vehicle network signal includes the normal state of the vehicle, the sensor target signal, the GPS signal and the intermediate variable; the sensor raw data includes the video and the radar point cloud;

[0016] The data collected and stored in step S1 also includes the vehicle frame number, software and hardware version numbers, and the time and place of collection.

[0017] Furthermore, the step S2 further includes:

[0018] S201: extracting the data uploaded in step S1 from the cloud storage server;

[0019] S202: Check the integrity of the data in step S201. If the data is complete, execute S203. If the data is incomplete, notify the data administrator to remove the data with lost frames or missing signals on the cloud storage server.

[0020] S203: Check the collection time, vehicle frame number, and software and hardware version number of the complete data and determine whether they are valid. If the data is valid, execute S3; if the data is invalid, notify the data administrator to remove redundant, repeated, or outdated data on the cloud storage server.

[0021] Furthermore, the step S3 further includes:

[0022] S301: Establish an event and scenario tag library based on data content, map production requirements, AI training requirements, and testing requirements;

[0023] S302: Describing the events and scenes in the event and scene tag library in detail;

[0024] S303: defining rules that can be used for machine self-identification, and using corresponding algorithm functions to make the rules correspond one-to-one with the scene label library through single signals, logical combinations of multiple signals, and events;

[0025] S304: Develop event and scenario self-identification software, and identify and track the data files cleaned in step S2 based on the defined rules in the software.

[0026] Furthermore, in step S4, the data file after identification and embedding in step S3 is parsed, and the data file is cut according to the location of the embedding point, and the chassis number, time, software and hardware version number and identification mark are added to each cut file.

[0027] Furthermore, in step S5, after the automatic labeling process is completed, the data files are hierarchically classified and saved to the cloud storage server.

[0028] Furthermore, in step S6, the classified and layered data files are converted into data files of different formats to adapt to different data usage environments and data homology. The different formats include: BLF format, Open drive format, MF4 format and compressed and annotated image format.

[0029] The present invention also relates to a storage medium, which stores one or more programs. When the one or more programs are run by a processor, the steps of the above-mentioned method for self-identification of autonomous driving scenes are executed.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] A self-identification method for autonomous driving scenarios of the present invention automatically identifies and extracts, formats, and labels, classifies, stores, and manages all collected vehicle network signals and sensor raw signal data; wherein, the scenario library established based on the hierarchical classification of real vehicle data covers natural traffic flow scenarios, autonomous driving failure conditions, driver takeover data, vehicle driving data, conditions outside the operating area, traffic accidents, traffic participant interaction data, and human-computer interaction data, which can provide big data support for automatic vehicle product definition, function planning, demand analysis, software and hardware development, system integration, simulation and real vehicle verification, rapid iteration and update of functions and performance, transportation infrastructure construction, and formulation of autonomous driving standards and regulations, thereby improving data utilization efficiency and reuse rate; and the sensor raw video and point cloud data, and intermediate variable data can provide training and test data sets for visual semantic segmentation, target detection and tracking, lane line recognition, drivable area recognition, game decision-making, self-learning path planning, and driver behavior simulation learning, continuously improving the stability and reliability of autonomous driving performance and gradually expanding the boundaries of autonomous driving scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a logic block diagram of a method for self-identification of an autonomous driving scene according to an embodiment;

[0033] Figure 2 This is a flow chart of data cleaning in the present invention;

[0034] Figure 3 This is a flowchart of event, scene recognition and tracking in the present invention;

[0035] Figure 4 This is a flow chart of data cutting in the present invention;

[0036] Figure 5 This is a flow chart of format conversion in the present invention. DETAILED DESCRIPTION

[0037] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0038] See Figure 1-Figure 5 A method for self-identifying an autonomous driving scene in a specific embodiment includes the following steps:

[0039] S1: Data acquisition and storage, uploading vehicle network signals and sensor raw data from acquisition vehicles, test vehicles, production vehicles, and third-party platforms to the cloud storage server;

[0040] S2: Data cleaning: Check the integrity and continuity of the data in the cloud storage server in step S1, and remove damaged and invalid data;

[0041] Wherein, step S2 further includes:

[0042] S201: extracting the data uploaded in step S1 from the cloud storage server;

[0043] S202: Check the integrity of the data in step S201. If the data is complete, execute S203. If the data is incomplete, notify the data administrator to remove the data with lost frames or missing signals on the cloud storage server.

[0044] S203: Check the data collection time, vehicle frame number, and software and hardware version numbers to determine whether the data is valid. If the data is valid, execute S3; if the data is invalid, notify the data administrator to remove redundant, duplicate, or outdated data on the cloud storage server;

[0045] S3: Event and scene recognition and tracking. Based on the logical relationship of vehicle network bus signals, it automatically identifies various events and traffic scenarios during driving and tracks the events and scenes of interest.

[0046] During implementation, the events during driving include: false braking, missed braking, and driver takeover; the traffic scenarios during driving include: tunnels, lane line types, traffic lights, ramps, roundabouts, intersections, and congestion.

[0047] Wherein, step S3 further includes:

[0048] S301: Establish an event and scenario tag library based on data content, ODD (design operation area), potential failure modes, map production requirements, AI training requirements, and testing requirements;

[0049] S302: Describing the events and scenes in the event and scene tag library in detail;

[0050] S303: defining rules that can be used for machine self-identification, and using corresponding algorithm functions to make the rules correspond one-to-one with the scene label library through single signals, logical combinations of multiple signals, and events;

[0051] S304: Develop event and scenario self-recognition software, and identify and track data files cleaned in step S2 based on the defined rules in the software;

[0052] S4: Data cutting: Cut the data file according to the location of the buried points in step S3, and only retain the data with complete information within 5 seconds before and after the accident;

[0053] S5: Classification and labeling: automatically label the data retained after step S4 according to the classification and labeling rules of events and scenes;

[0054] S6: Format conversion, performing data conversion on the data file after labeling processing in step S5;

[0055] S7: Data usage: Use the data file obtained in step S6 for problem analysis, simulation scenario construction and testing, back-injection testing, AI training and testing, and map data production and updating.

[0056] An embodiment of a method for self-identification of autonomous driving scenarios automatically identifies, extracts, formats, and labels, classifies, stores, and manages all collected vehicle network signals and sensor raw signal data; wherein, a scenario library established based on hierarchical classification of real vehicle data covers natural traffic flow scenarios, autonomous driving failure conditions, driver takeover data, vehicle driving data, conditions outside the operating area, traffic accidents, traffic participant interaction data, and human-computer interaction data, which can provide big data support for autonomous vehicle product definition, function planning, demand analysis, software and hardware development, system integration, simulation and real vehicle verification, rapid iteration and update of functions and performance, transportation infrastructure construction, and formulation of autonomous driving standards and regulations, thereby improving data utilization efficiency and reuse rate.

[0057] During implementation, please refer to the following table for some of the events and scenario tag libraries:

[0058]

[0059]

[0060] During implementation, the event and scenario tag library are described in detail, as shown in the table below:

[0061]

[0062] It can be understood here that the above two tables are only examples of the event and scene label library and its detailed description described in the present invention. The content of the label library and its detailed description in specific situations should be far more than the content in the above table. They should be established and formulated based on the actual vehicle test conditions, but they should all be included in the scope of protection of the present invention.

[0063] During implementation, the expressions for logical combination of the signals mentioned in step S303 are shown in the following table:

[0064]

[0065]

[0066] Here is an explanation of the aforementioned "S303: define rules that can be used for machine self-identification, and use corresponding algorithm functions to make the rules correspond one-to-one with the scene label library through single signals, logical combinations of multiple signals and events": for example, when the ACC identification signal jumps from the activated state to the cancelled state and the vehicle deceleration is greater than 3m / s2 for one second, it is judged as an ACC missed braking event (secondary label) in the abnormal event (first-level label).

[0067] Please continue to see Figure 1-Figure 5 , wherein, in the step S1, the vehicle network signal includes the normal state of the vehicle, the sensor target signal, the GPS signal and the intermediate variable; the sensor raw data includes the video and the radar point cloud;

[0068] The data collected and stored in step S1 also includes the vehicle frame number, software and hardware version numbers, and the time and place of collection.

[0069] This makes it easier to trace and reproduce data. The sensor's raw video and point cloud data can provide training and test data sets for visual semantic segmentation, target detection and tracking, lane line recognition, drivable area recognition, game decision-making, self-learning path planning, and driver behavior simulation learning, continuously improving the stability and reliability of autonomous driving performance and gradually expanding the boundaries of autonomous driving scenarios.

[0070] During implementation, data from the collection vehicle and test vehicle are uploaded to the cloud storage server via a mobile hard drive; for the data from the mass-produced vehicle, when the vehicle identifies a general accident, serious accident, or abnormal event with a first-level label, the data 30 seconds before and after the event will be automatically uploaded to the cloud storage server via the vehicle terminal (T-BOX); data from third-party platforms are uploaded to the cloud storage server via a dedicated network. Third-party platforms include but are not limited to cloud platforms of government enterprises and institutions such as Alibaba Cloud, Tencent Cloud, Baidu Cloud, and supplier cloud platforms.

[0071] Among them, in step S4, the data file after identification and embedding in step S3 is parsed, and the data file is cut according to the location of the embedding point, and the chassis number, time, software and hardware version number and identification mark are added to each cut file.

[0072] In step S5, after the automatic labeling process is completed, the data files are hierarchically classified and saved to the cloud storage server.

[0073] During implementation, the files cut in step S4 are automatically labeled according to the event, scene classification and labeling rules.

[0074] In step S6, the classified and layered data files are converted into data files of different formats to adapt to different data usage environments and data homology. The different formats include: BLF format, Open drive format, MF4 format and compressed and annotated image format.

[0075] like Figure 5 As shown, the BLF format is used for back-injection testing; the Open drive format is used for map data production and updating; the MF4 format is used for problem analysis; the compressed and annotated image format is used for AI training and testing; all stored data types and formats are used for simulation scenario construction and testing.

[0076] The present invention also provides a storage medium, which stores one or more programs. When the one or more programs are executed by a processor, the steps of the above-mentioned method for self-identification of autonomous driving scenes are executed.

[0077] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for self-identification of autonomous driving scenarios, characterized by: The specific steps include: S1: Data acquisition and storage, uploading vehicle network signals and sensor raw data from acquisition vehicles, test vehicles, production vehicles, and third-party platforms to the cloud storage server; S2: Data cleaning: Check the integrity and continuity of the data in the cloud storage server in step S1, and remove damaged and invalid data; S3: Event and scene recognition and tracking. Based on the logical relationship of vehicle network bus signals, it automatically identifies various events and traffic scenarios during driving and tracks the events and scenes of interest. The specific process includes: S301: Establish an event and scenario tag library based on data content, map production requirements, AI training requirements, and testing requirements; S302: Describing the events and scenes in the event and scene tag library in detail; S303: defining rules that can be used for machine self-identification, and using corresponding algorithm functions to make the rules correspond one-to-one with the scene label library through single signals, logical combinations of multiple signals, and events; S304: Develop event and scenario self-recognition software, and identify and track data files cleaned in step S2 based on the defined rules in the software; S4: Data cutting: Cut the data file according to the location of the buried points in step S3, and only retain the data with complete information within 5 seconds before and after the accident; S5: Classification and labeling: automatically label the data retained after step S4 according to the classification and labeling rules of events and scenes; S6: Format conversion, performing data conversion on the data file after labeling processing in step S5; S7: Data usage: Use the data file obtained in step S6 for problem analysis, simulation scenario construction and testing, back-injection testing, AI training and testing, and map data production and updating.

2. The method for self-identification of an autonomous driving scene according to claim 1, characterized in that: In step S1, the vehicle network signal includes the normal state of the vehicle, the sensor target signal, the GPS signal and the intermediate variable; the sensor raw data includes the video and the radar point cloud; The data collected and stored in step S1 also includes the vehicle frame number, software and hardware version numbers, and the time and place of collection.

3. The method for self-identification of an autonomous driving scene according to claim 1, characterized in that: The step S2 further includes: S201: extracting the data uploaded in step S1 from the cloud storage server; S202: Check the integrity of the data in step S201. If the data is complete, execute S203. If the data is incomplete, notify the data administrator to remove the data with lost frames or missing signals on the cloud storage server. S203: Check the collection time, vehicle frame number, and software and hardware version number of the complete data and determine whether they are valid. If the data is valid, execute S3; if the data is invalid, notify the data administrator to remove redundant, repeated, or outdated data on the cloud storage server.

4. The method for self-identification of an autonomous driving scene according to claim 1, characterized in that: In step S4, the data file after identification and embedding in step S3 is parsed, and the data file is cut according to the location of the embedding point, and the chassis number, time, software and hardware version number and identification mark are added to each cut file.

5. The method for self-identification of an autonomous driving scene according to claim 1, characterized in that: In step S5, after the automatic labeling process is completed, the data files are hierarchically classified and saved to the cloud storage server.

6. The method for self-identification of an autonomous driving scene according to claim 5, characterized in that: In step S6, the classified and layered data files are converted into data files of different formats to adapt to different data usage environments and data homology. The different formats include: BLF format, Open drive format, MF4 format and compressed and annotated image format.

7. A storage medium, characterized in that: The storage medium stores one or more programs, and when the one or more programs are executed by the processor, the steps of the autonomous driving scene self-identification method according to any one of claims 1 to 6 are executed.

Images