Automatic driving scene mining method and device, equipment and storage medium

Abstract

The present disclosure provides a mining method and device for an autonomous driving scene, equipment and a storage medium, and relates to the technical field of data processing, in particular to the technical field of autonomous driving, cloud computing, deep learning and the like. The specific implementation scheme is as follows: according to a received mining task, determining autonomous driving data to be mined and a target autonomous driving scene; the autonomous driving data comprises a plurality of data segments with labels; according to the target autonomous driving scene, at least determining a label variable and a logic variable in a preset mining architecture; according to the label variable and the logic variable, generating a mining logic based on the preset mining architecture; and according to the mining logic, mining a target data segment corresponding to the target autonomous driving scene from the plurality of data segments. According to the present disclosure, the mining logic can be conveniently and quickly generated based on the target autonomous driving scene, and the target data segment corresponding to the target autonomous driving scene can be accurately mined from the autonomous driving data based on the mining logic.

Description

[0001] This application is a divisional application of a Chinese invention entitled “Method, Apparatus, Device and Storage Medium for Discovering Autonomous Driving Scenes”, application number “202211630717.5”, filed on December 19, 2022. Technical Field

[0002] This disclosure relates to the field of data processing technology, and in particular to the fields of autonomous driving, cloud computing, and deep learning. Background Technology

[0003] The approach of using data to drive the upgrade and iteration of autonomous driving systems is a widely accepted solution in the industry. However, extracting high-value data from massive amounts of autonomous driving data has always been a challenge for the industry. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, and storage medium for mining autonomous driving scenarios.

[0005] According to one aspect of this disclosure, a method for mining autonomous driving scenarios is provided, comprising:

[0006] Based on the received mining task, the autonomous driving data to be mined and the target autonomous driving scenario are determined; among them, the autonomous driving data includes multiple data fragments with labels;

[0007] Based on the target autonomous driving scenario, at least the label variables and logical variables in the preset mining architecture should be determined;

[0008] Based on the tag variables and logical variables, and using a pre-defined mining architecture, mining logic is generated; and

[0009] Based on the mining logic, target data segments corresponding to the target autonomous driving scenario are extracted from multiple data segments.

[0010] According to another aspect of this disclosure, an apparatus for identifying autonomous driving scenarios is provided, comprising:

[0011] The first determining module is used to determine the autonomous driving data to be mined and the target autonomous driving scenario based on the received mining task; wherein, the autonomous driving data includes multiple data fragments with labels;

[0012] The second determination module is used to determine at least the label variables and logical variables in the preset mining architecture based on the target autonomous driving scenario.

[0013] The generation module is used to generate mining logic based on a preset mining architecture, using tag variables and logical variables; and

[0014] The mining module is used to extract target data segments corresponding to the target autonomous driving scenario from multiple data segments based on mining logic.

[0015] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0016] At least one processor; and

[0017] The memory is communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0019] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0020] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0021] According to the technology disclosed herein, mining logic can be generated conveniently and quickly based on the target autonomous driving scenario, and the target data fragment corresponding to the target autonomous driving scenario can be accurately mined from the autonomous driving data based on the mining logic.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0023] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0024] Figure 1 This is a flowchart illustrating a method for mining autonomous driving scenarios according to an embodiment of the present disclosure;

[0025] Figure 2 This is a schematic diagram of the autonomous driving data segmentation method according to the autonomous driving scenario mining method of this disclosure embodiment;

[0026] Figure 3 This is a flowchart illustrating a method for mining autonomous driving scenarios according to another embodiment of the present disclosure;

[0027] Figure 4This is a schematic diagram of the preset label dimensions of the autonomous driving scenario mining method according to an embodiment of the present disclosure;

[0028] Figure 5 This is a schematic diagram of an excavation device in an autonomous driving scenario according to an embodiment of the present disclosure;

[0029] Figure 6 This is a block diagram of an electronic device used to implement the autonomous driving scenario discovery method of the present disclosure embodiments. Detailed Implementation

[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0031] like Figure 1 As shown, this disclosure provides a method for mining autonomous driving scenarios, including:

[0032] Step S101: Based on the received mining task, determine the autonomous driving data to be mined and the target autonomous driving scenario. The autonomous driving data includes multiple data segments with labels.

[0033] Step S102: Based on the target autonomous driving scenario, at least determine the label variables and logical variables in the preset mining architecture.

[0034] Step S103: Based on the label variables and logical variables, generate the mining logic according to the preset mining architecture.

[0035] Step S104: Based on the mining logic, extract the target data segment corresponding to the target autonomous driving scenario from multiple data segments.

[0036] According to the embodiments of this disclosure, it should be noted that:

[0037] Autonomous driving data can be understood as the data collected by autonomous vehicles during the process of driving with autonomous driving functions.

[0038] A target autonomous driving scenario can be understood as an autonomous driving scenario that needs to be found from the autonomous driving data to be mined. For example, a target autonomous driving scenario could be a scenario where the main vehicle continuously changes right lanes, a scenario where the main vehicle is going straight and an obstacle vehicle is changing lanes, a scenario where the main vehicle is passing through an intersection and a pedestrian is crossing, etc. In one example, if the target autonomous driving scenario is a scenario where the main vehicle continuously changes right lanes, then it is necessary to mine the data segments corresponding to the continuous right lane changes from the autonomous driving data to be mined.

[0039] The rules for dividing autonomous driving data into multiple data segments can be selected and adjusted as needed. After splicing and combining multiple data segments, complete autonomous driving data can be obtained. Autonomous driving data includes multiple data segments with labels, such as... Figure 2 As shown, this can be understood as dividing the data on the behavior of the main vehicle in autonomous driving data into multiple data segments and labeling them accordingly. Similarly, the data on the behavior of obstacles (e.g., pedestrians, vehicles, non-motorized vehicles, etc.) in autonomous driving data can be divided into multiple data segments and labeled accordingly. It should be noted that the lengths of the multiple data segments can be unequal. Each data segment can have one or more labels, depending on the label dimension. For example, during the main vehicle's straight-ahead movement from 0 seconds to 5 seconds, the data segment corresponding to this time period can have a label for the main vehicle's straight-ahead behavior. Simultaneously, if an obstacle interacts with the main vehicle during this time period, the data segment corresponding to this time period can also have a label for the vehicle decelerating. Furthermore, the data segment corresponding to the obstacle during this time period can have a label for the obstacle passing from the left front of the main vehicle.

[0040] Label variables can be understood as tags used to match data segments when performing scene mining on autonomous driving data. For example, in the target autonomous driving scenario where the main vehicle first changes lanes to the left and then immediately changes lanes to the right, the label variables are "main vehicle changes lanes to the left" and "main vehicle changes lanes to the right". Based on these label variables, all data segments labeled "main vehicle changes lanes to the left" and all data segments labeled "main vehicle changes lanes to the right" can be found from multiple data segments.

[0041] Logical variables can be understood as the computational logic used to further process data fragments obtained based on label variables when performing scene mining on autonomous driving data. For example, in the target autonomous driving scenario of continuous right lane changes by the main vehicle, based on the label variable (main vehicle right lane change), all data fragments with the label "main vehicle right lane change" are obtained from multiple data fragments in the autonomous driving data. Then, the logical variable (two main vehicle right lane changes are time-continuous) is used to process all the obtained data fragments with the "main vehicle right lane change" label to determine whether there are two time-continuous data fragments with the "main vehicle right lane change" label. If found, the data fragment corresponding to the continuous right lane change scenario of the main vehicle has been mined from the autonomous driving data.

[0042] A preset mining framework can be understood as the inherent basic structure of the mining logic. Only after adding the necessary label variables and logic variables to the preset mining framework can complete mining logic be generated. Because the preset mining framework is an inherent basic structure, it does not need to be adjusted when mining different target driving scenarios; only the label variables and logic variables within the preset mining framework need to be changed to generate different mining logic.

[0043] For example, the preset mining framework is:

[0044] $_name_ = "XX";

[0045] $_brief_ = "XX";

[0046] $_express_ = {XX};

[0047] Among them, “$_name_” and “$_brief_” are used to fill in the name variables of the mining logic; “$_express_” is used to fill in the logic variables and label variables.

[0048] Once the required label variables and logical variables are added to the preset mining framework, the preset mining framework constitutes the mining logic, for example:

[0049] $name="continuous_right_change_Lanes";

[0050] $_brief_="continuous_right_change_lanes";

[0051] $_express_={

[0052] $right_change1=right_change>>1;

[0053] $right_change2=right_change;

[0054] $matched=$right_change1<<? =$right_change2;

[0055] $last_res=merge($matched);

[0056] };

[0057] In this context, "continuous_right_change_Lanes" is the name variable for the mining logic; "$right_change1 = right_change" and "$right_change2 = right_change" are the label variables for the main vehicle's right lane change; "$right_change1 = right_change >> 1" is the logic variable that delays the main vehicle's right turn label time by 1 second; "$matched = $right_change1 << ? = $right_change2" is the logic variable for comparing the right lane change times of the two main vehicles, which is used to determine whether the right lane changes of the two main vehicles are consecutive; and "$last_res = merge($matched)" is the calculation result.

[0058] A target data segment can contain multiple data segments with labels. For example, if the target driving scenario is a series of right lane changes by the main vehicle, and two data segments showing the main vehicle changing right lanes in consecutive time are found based on the mining logic, then these two data segments showing the main vehicle changing right lanes in consecutive time constitute a target data segment.

[0059] According to the technology disclosed herein, mining logic can be conveniently and quickly generated based on a target autonomous driving scenario, and the target data fragments corresponding to the target autonomous driving scenario can be accurately extracted from autonomous driving data based on the mining logic. Since the preset mining architecture is an inherent basic structure, it does not need to be adjusted when mining different target driving scenarios; only the label variables and logical variables in the preset mining framework need to be changed to quickly customize the mining logic for different autonomous driving scenarios. This not only saves the time required to develop different mining logics for different target autonomous driving scenarios, but also allows for the mining of autonomous driving scenarios of varying complexity based on different combinations of label variables and logical variables. Because the label variables and logical variables can be freely changed according to the mining requirements of the autonomous driving scenario, the mining logic has strong scalability, enabling continuous upgrades and optimizations.

[0060] The method disclosed herein can quickly generate the required mining logic, thereby rapidly mining the target autonomous driving scenario and enabling the rapid mining of high-value data from massive amounts of autonomous driving data. This effectively achieves the goal of driving the upgrade and iteration of autonomous driving systems based on autonomous driving data.

[0061] In one example, the method of this disclosure embodiment can be applied to traffic flow data mining operations to serve as a guide for the specialized testing of the decision planning module and the construction of the scenario library.

[0062] In one embodiment, the autonomous driving scenario mining method of this disclosure includes steps S101 to S104, and further includes:

[0063] Step S105: Determine the motion parameter information corresponding to the target data segment based on the time period information of the autonomous driving data and the target data segment.

[0064] Step S106: Transmit the time period information and motion parameter information of the target data segment back.

[0065] According to the embodiments of this disclosure, it should be noted that:

[0066] The time period information of the target data segment can be understood as the time interval corresponding to the target data segment. For example, the target data segment of the main vehicle continuously changing right lanes is the autonomous driving data collected when the main vehicle is driving between 10:01 and 10:02. Then the time period information of the target data segment is 10:01 to 10:02.

[0067] The motion parameter information corresponding to the target data segment can be understood as the information on the main vehicle's driving behavior contained within the target data segment. For example, the main vehicle's speed information, the main vehicle's acceleration information, the relative position information between the main vehicle and the obstacle, and the main vehicle's collision time information.

[0068] According to the technology disclosed herein, by transmitting back the time period information and motion parameter information of the target data segment, it is convenient to use the information of the target data segment to analyze the autonomous driving scenario.

[0069] In one example, the method of this disclosure embodiment can be executed by the nodes (servers) of a distributed system. The distributed system includes at least a master node and multiple worker nodes. The specific implementation process is as follows:

[0070] The master node sends multiple mining tasks to multiple worker nodes one by one.

[0071] Multiple worker nodes execute the mining task concurrently. Each worker node executes steps S101 to S106.

[0072] The recycling node receives the time period information and motion parameter information of the target data segment transmitted back by each worker node. The recycling node can be the master node or any of the worker nodes.

[0073] In one example, such as Figure 3 As shown, the master node is responsible for distributing tasks, each worker node is responsible for processing tasks, and the reclaim node is responsible for reclaiming tasks.

[0074] The master node determines multiple worker nodes through cluster scheduling;

[0075] The master node distributes the mining task to multiple worker nodes;

[0076] Each worker node's processing tasks include:

[0077] Prepare the raw data. This part of the autonomous driving data can come from road data collected by autonomous vehicles and undergo necessary preprocessing such as noise reduction and smoothing.

[0078] In the tag extraction stage, the raw data is parsed to obtain multiple data fragments, and tags are extracted from each data fragment according to certain dimensions. The tag dimensions include road topology dimension, vehicle (main vehicle) behavior dimension, traffic participant (obstacle) behavior dimension, and interaction behavior dimension, which are then broken down and mined to generate tags.

[0079] Mining Logic Customization: This customization work occurs before the mining task starts. Based on the target autonomous driving scenario to be mined, the mining logic is written according to a specific syntax (preset mining architecture + label variables + logic variables) to obtain the mining logic.

[0080] Syntax parser parsing: The syntax parser uses the labels of multiple data fragments as raw materials to interpret and execute the mining logic, and mines the target data fragments corresponding to the target autonomous driving scenario from multiple data fragments.

[0081] Target scene output: After being parsed and executed by the syntax parser, the target data fragment will be obtained. In addition to the time period information of the fragment, the target data fragment also includes motion parameters such as velocity, acceleration, relative position, collision time and other parameter information.

[0082] The recycling node receives target scene fragment information and recycles the data for subsequent business applications and other scenarios.

[0083] In one embodiment, the autonomous driving scenario mining method of this disclosure includes steps S101 to S104, wherein step S101: determining the autonomous driving data to be mined and the target autonomous driving scenario based on the received mining task, and prior to this, further including:

[0084] Step S201: Divide the autonomous driving data to be mined into multiple data segments.

[0085] Step S202: Determine the label of each data segment among multiple data segments according to the preset label dimensions.

[0086] According to the embodiments of this disclosure, it should be noted that:

[0087] The preset label dimensions can be selected and adjusted as needed, and no specific limitations are imposed here. For example, preset label dimensions may include at least one of the following: road network topology dimension, vehicle behavior dimension, obstacle behavior dimension, and interaction behavior dimension.

[0088] Dividing the autonomous driving data to be mined into multiple data segments can be understood as dividing the data of each behavioral object involved in the autonomous driving data separately. For example, if the behavioral objects in the autonomous driving data include the driver vehicle, obstacle vehicles (obstacles), and pedestrians, then the data can be divided based on the driver vehicle, obstacle vehicles, and pedestrians, so that the driver vehicle includes multiple data segments, the obstacle vehicle includes multiple data segments, and the pedestrian includes multiple data segments.

[0089] Multiple data segments can have different lengths. Each data segment can have one or more labels, depending on the label dimension. For example, during the driver's straight-ahead movement from 0 seconds to 5 seconds, the data segment corresponding to that time period can have a "driver's straight-ahead behavior" label. Simultaneously, if an obstacle interacts with the driver during this time period, the corresponding data segment can also have a "vehicle deceleration" interaction label. Furthermore, the data segment corresponding to the obstacle during this time period can have an "obstacle behavior" label indicating that the obstacle passed from the driver's left front.

[0090] According to the technology disclosed herein, by pre-labeling each data segment, it is easier to use mining logic to perform scene mining on autonomous driving data.

[0091] In one example, such as Figure 4 As shown, the specific labels that can be included in the road network topology dimension, the main vehicle behavior dimension, the obstacle behavior dimension, and the interaction behavior dimension are illustrated.

[0092] For example, scene labels in the road network topology dimension can include labels such as: straight road, curved road, main road, auxiliary road, multi-lane road, and few-lane road.

[0093] The tags for the main vehicle behavior dimension can include: left lane change, right lane change, left turn, right turn, straight ahead, parking, etc.

[0094] The labels for the obstacle behavior dimension can include: location labels for the obstacle being directly in front of, to the right front of, or directly behind the vehicle, and can also include category labels for the obstacle as a pedestrian, bicycle, or motor vehicle.

[0095] The tags for interactive behavior can include: obstacles cutting into the main vehicle, obstacles driving in the opposite direction to the main vehicle, etc.

[0096] In one embodiment, the autonomous driving scenario mining method of this disclosure includes steps S101 to S104, and steps S201 and S202, wherein step S201: dividing the autonomous driving data to be mined into multiple data segments, including:

[0097] From the autonomous driving data to be mined, determine the main vehicle data and obstacle data.

[0098] Based on the driving behavior data in the main vehicle data, the main vehicle data is divided into multiple first data segments.

[0099] Based on the obstacle behavior data in the obstacle data, the obstacle data is divided into multiple second data segments.

[0100] According to the embodiments of this disclosure, it should be noted that:

[0101] Master vehicle data can be understood as data associated with the master vehicle, such as the master vehicle's own behavior data, or the master vehicle's interaction data with obstacles.

[0102] Obstacle data can be understood as data related to obstacles, such as the obstacle's own behavior data, or the behavior data of the interaction between the vehicle and the obstacle.

[0103] Multiple first data segments may have different lengths. Each first data segment may have one or more labels, depending on the label dimension.

[0104] Multiple second data segments may have different lengths. Each second data segment may have one or more labels, depending on the label dimension.

[0105] According to the technology disclosed herein, it is possible to label different behavioral objects in autonomous driving data. This enables more accurate extraction of target data segments corresponding to the target autonomous driving scenario when mining autonomous driving data based on mining logic.

[0106] In one embodiment, the autonomous driving scenario mining method of this disclosure includes steps S101 to S104, and steps S201 and S202, wherein step S202: determining the label of each data segment in a plurality of data segments according to a preset label dimension, including:

[0107] Based on driving behavior data, determine the main vehicle behavior labels for multiple first data segments.

[0108] Based on obstacle behavior data, obstacle behavior labels are determined for multiple second data segments.

[0109] Based on road data in autonomous driving data, determine the road network topology labels for multiple first data segments and multiple second data segments.

[0110] Based on the interaction data in the autonomous driving data, determine the interaction behavior labels of multiple first data segments and multiple second data segments.

[0111] According to the technology disclosed herein, the first and second data segments can be equipped with label information of different dimensions, enabling more comprehensive labeling of data segments. This facilitates the more accurate discovery of target data segments corresponding to the target autonomous driving scenario based on the labels when mining autonomous driving data using mining logic.

[0112] In one embodiment, the autonomous driving scenario mining method of this disclosure includes steps S101 to S104, wherein step S102: based on the target autonomous driving scenario, at least determine label variables and logical variables in a preset mining architecture, including:

[0113] Based on the scene objects and their behaviors in the target autonomous driving scenario, determine the label variables in the preset mining architecture.

[0114] Based on the behavioral relationships in the target autonomous driving scenario, determine the logical variables in the preset mining architecture.

[0115] According to the embodiments of this disclosure, it should be noted that:

[0116] A scene object can be understood as the behavioral entity in the target autonomous driving scenario. For example, if the target autonomous driving scenario involves the main vehicle continuously changing right lanes, then the main vehicle is the scene object.

[0117] Scene object behavior can be understood as the behavior performed by scene objects in the target autonomous driving scenario. For example, if the target autonomous driving scenario involves the main vehicle continuously changing right lanes, then changing right lanes is a scene object behavior.

[0118] The pre-defined tag variables in the mining architecture are generated based on the scene object and its behavior. For example, if the target autonomous driving scenario is a main vehicle continuously changing right lanes, the main vehicle is the scene object, and changing right lanes is the scene object's behavior, then the tag variable is the main vehicle's right lane change tag.

[0119] Behavioral relationships can be understood as the relationships between the behaviors of multiple scene objects in a target autonomous driving scenario. For example, if the target autonomous driving scenario involves the main vehicle continuously changing right lanes, then continuous right lane changes are a behavioral relationship. The logical variable is whether the time between the data segments corresponding to the two main vehicle right lane change labels is continuous. Specifically, if the time between the data segments corresponding to the two main vehicle right lane change labels is continuous, it indicates that the main vehicle is continuously changing right lanes.

[0120] According to the technology disclosed herein, by utilizing scene objects, scene object behaviors, and behavior relationships in a target autonomous driving scenario, the label variables and logical variables required to construct the mining logic of the target autonomous driving scenario can be accurately determined.

[0121] In one embodiment, the autonomous driving scenario mining method of this disclosure includes steps S101 to S104, wherein step S102: based on the target autonomous driving scenario, at least determine label variables and logical variables in a preset mining architecture, including:

[0122] Based on the scene objects and their behaviors in the target autonomous driving scenario, determine the label variables in the preset mining architecture.

[0123] Based on the behavioral relationships in the target autonomous driving scenario, determine the logical variables in the preset mining architecture.

[0124] Based on the road and behavioral constraints in the target autonomous driving scenario, determine the constraint variables in the preset mining architecture.

[0125] According to the embodiments of this disclosure, it should be noted that:

[0126] Road constraints can be understood as the road environment that limits the occurrence of autonomous driving scenarios. For example, if an autonomous driving scenario involves the main vehicle continuously changing right lanes in a non-intersection section, then the intersection section is a road constraint.

[0127] Behavioral constraints can be understood as conditions that limit the behavior of objects in an autonomous driving scenario. For example, if an autonomous driving scenario involves the main vehicle continuously changing right lanes at a speed below 40 mph, then the speed below 40 mph is the behavioral constraint.

[0128] Constraint variables can be understood as the constraints that need to be satisfied when mining target data from multiple data fragments based on label variables and logical variables.

[0129] According to the technology disclosed herein, by utilizing scene objects, scene object behaviors, behavioral relationships, road constraints, and behavioral constraints in a target autonomous driving scenario, the label variables, logical variables, and constraint variables required to construct the mining logic of the target autonomous driving scenario can be accurately determined.

[0130] In one embodiment, the autonomous driving scenario mining method of this disclosure includes steps S101 to S104, wherein step S104: according to mining logic, mining a target data segment corresponding to the target autonomous driving scenario from multiple data segments, including:

[0131] Step S1041: Based on the label variables of the mining logic, obtain candidate data segments corresponding to the label variables from multiple data segments.

[0132] Step S1042: Based on the logical variables of the mining logic and the time period information of the candidate data segments, determine the target data segments that meet the logical variables.

[0133] According to the technology disclosed herein, by mining logical label variables and logical variables, the target data segment corresponding to the target autonomous driving scenario can be accurately extracted from multiple data segments.

[0134] In one example, the target autonomous driving scenario is continuous right lane changes by the main vehicle, and the label variable is "main vehicle right lane change". Therefore, the candidate data segments corresponding to the label variable are all the main vehicle right lane change data segments in the autonomous driving data. If the logical variable requires the two right lane changes to be consecutive, then two consecutive main vehicle right lane change data segments are found from all the main vehicle right lane change data segments as the target segments.

[0135] In one embodiment, the autonomous driving scenario mining method of this disclosure includes steps S101 to S104, step S1041, and step S1042, wherein step S1042: based on the logical variables of the mining logic and the time period information of the candidate data segments, the target data segments that meet the logical variables are determined, including:

[0136] Based on the time period logical variable in the logical variable, and using the time period information of the candidate data segment, the first data segment that meets the time period logical variable is determined.

[0137] Based on the comparison operation logic variables in the logic variables, and the time period information of the first data segment, the target data segment that matches the comparison operation logic variables is determined.

[0138] According to the embodiments of this disclosure, it should be noted that:

[0139] Time-period logical variables can be understood as variables used to limit the time-period information of a data segment. For example, the time when a pedestrian crosses the road can occur 3 seconds after the main vehicle passes through the intersection (time-period logical variable). That is to say, the time of the pedestrian crossing does not directly overlap with the time of the main vehicle passing through the intersection, but the candidate data segment of the pedestrian that meets the 3-second condition can also be retained as the first data segment.

[0140] The comparison operation logic variable can be understood as the operational logic between multiple first data segments. For example, in a target autonomous driving scenario where the main vehicle continuously changes right lanes, the comparison operation logic variable compares whether the start and end times of two first data segments are consecutive. If they are consecutive, it means that the two first data segments are the target data segments that meet the comparison operation logic variable.

[0141] According to the technology disclosed herein, by utilizing time-period logical variables and comparison operation logical variables, the target data segment corresponding to the target autonomous driving scenario can be accurately extracted from multiple data segments.

[0142] In one embodiment, the autonomous driving scenario mining method of this disclosure includes steps S101 to S104, wherein step S104: according to mining logic, mining a target data segment corresponding to the target autonomous driving scenario from multiple data segments, including:

[0143] Step S1043: Based on the label variables of the mining logic, obtain the first candidate data segment corresponding to the label variables from multiple data segments.

[0144] Step S1044: Based on the constraint variables of the mining logic, determine the second candidate data segment that meets the constraint variables based on the first candidate data segment.

[0145] Step S1045: Based on the logical variables of the mining logic, determine the target data segment that matches the logical variables based on the second candidate data segment.

[0146] According to the technology disclosed herein, by mining the label variables, logical variables, and constraint variables of the logic, the target data segment corresponding to the target autonomous driving scenario can be accurately extracted from multiple data segments.

[0147] In one embodiment, the autonomous driving scenario mining method of this disclosure includes steps S101 to S104, and steps S1043 to S1045, wherein step S1045: based on the logical variables of the mining logic, a target data segment that conforms to the logical variables is determined based on the second candidate data segment, including:

[0148] Based on the time period logical variable in the logical variable, and using the time period information of the second candidate data segment, the first data segment that meets the time period logical variable is determined.

[0149] Based on the comparison operation logic variables in the logic variables, and the time period information of the first data segment, the target data segment that matches the comparison operation logic variables is determined.

[0150] According to the embodiments of this disclosure, it should be noted that:

[0151] Time-period logical variables can be understood as variables used to limit the time-period information of a data segment. For example, the time when a pedestrian crosses the road can occur 3 seconds after the main vehicle passes through the intersection (time-period logical variable). That is to say, the time of the pedestrian crossing does not directly overlap with the time of the main vehicle passing through the intersection, but the candidate data segment of the pedestrian that meets the 3-second condition can also be retained as the first data segment.

[0152] The comparison operation logic variable can be understood as the operational logic between multiple first data segments. For example, in a target autonomous driving scenario where the main vehicle continuously changes right lanes, the comparison operation logic variable compares whether the start and end times of two first data segments are consecutive. If they are consecutive, it means that the two first data segments are the target data segments that meet the comparison operation logic variable.

[0153] According to the technology disclosed herein, by utilizing time-period logical variables and comparison operation logical variables, the target data segment corresponding to the target autonomous driving scenario can be accurately extracted from multiple data segments.

[0154] In one example, the mining logic can include multiple sub-mining logics. Each sub-mining logic can be generated based on label variables, or based on both label variables and logical variables. After the sub-mining logics are executed in a certain logical order, autonomous driving scenario mining can be achieved from the autonomous driving data. The execution results of each sub-mining logic can be cached.

[0155] After finding the target data segment based on the mining logic, if the mining results are not ideal, you can view the execution results of each sub-mining logic and adjust the label variables and / or logic variables of a certain sub-mining logic accordingly. This eliminates the need to redesign the overall mining logic, saving the development cycle and design difficulty of the mining logic.

[0156] like Figure 5 As shown, this disclosure provides an apparatus for identifying autonomous driving scenarios, including:

[0157] The first determining module 510 is used to determine the autonomous driving data to be mined and the target autonomous driving scenario based on the received mining task. The autonomous driving data includes multiple data segments with labels.

[0158] The second determining module 520 is used to determine at least the label variables and logical variables in the preset mining architecture based on the target autonomous driving scenario.

[0159] The generation module 530 is used to generate mining logic based on label variables and logical variables, and according to a preset mining architecture.

[0160] The mining module 540 is used to extract target data segments corresponding to the target autonomous driving scenario from multiple data segments according to the mining logic.

[0161] In one implementation, the second determining module 520 includes:

[0162] The first determination submodule is used to determine the label variables in the preset mining architecture based on the scene objects and scene object behaviors in the target autonomous driving scenario.

[0163] The second determination submodule is used to determine the logical variables in the preset mining architecture based on the behavioral relationships in the target autonomous driving scenario.

[0164] In one embodiment, the second determining module 520 further includes:

[0165] The third determination submodule is used to determine the constraint variables in the preset mining architecture based on the road constraints and behavioral constraints in the target autonomous driving scenario.

[0166] In one embodiment, the mining module 540 includes:

[0167] The first acquisition submodule is used to acquire candidate data segments corresponding to the label variables from multiple data segments based on the label variables of the mining logic.

[0168] The fourth determination submodule is used to determine the target data segment that matches the logical variables based on the time period information of the candidate data segments, according to the logical variables of the mining logic.

[0169] In one implementation, the fourth determining submodule is further configured to:

[0170] Based on the time period logical variable in the logical variable, and using the time period information of the candidate data segment, the first data segment that meets the time period logical variable is determined.

[0171] Based on the comparison operation logic variables in the logic variables, and the time period information of the first data segment, the target data segment that matches the comparison operation logic variables is determined.

[0172] In one embodiment, the mining module 540 includes:

[0173] The second acquisition submodule is used to acquire the first candidate data segment corresponding to the label variable from multiple data segments based on the label variable of the mining logic.

[0174] The fifth determination submodule is used to determine the second candidate data segment that meets the constraints of the mining logic based on the first candidate data segment.

[0175] The sixth determination submodule is used to determine the target data segment that matches the logical variables based on the second candidate data segment, according to the logical variables of the mining logic.

[0176] In one implementation, the sixth determining submodule is used to:

[0177] Based on the time period logical variable in the logical variable, and using the time period information of the second candidate data segment, the first data segment that meets the time period logical variable is determined.

[0178] Based on the comparison operation logic variables in the logic variables, and the time period information of the first data segment, the target data segment that matches the comparison operation logic variables is determined.

[0179] In one implementation, the device for identifying autonomous driving scenarios further includes:

[0180] The third determining module is used to determine the motion parameter information corresponding to the target data segment based on the time period information of the autonomous driving data and the target data segment.

[0181] The feedback module is used to feedback the time period information and motion parameter information of the target data segment.

[0182] In one implementation, the device for identifying autonomous driving scenarios further includes:

[0183] The segmentation module is used to divide the autonomous driving data to be mined into multiple data segments.

[0184] The fourth determination module is used to determine the label of each data segment among multiple data segments based on the preset label dimensions.

[0185] In one implementation, the preset label dimensions include at least one of the following: road network topology dimension, vehicle behavior dimension, obstacle behavior dimension, and interaction behavior dimension.

[0186] In one implementation, the partitioning module is used for:

[0187] From the autonomous driving data to be mined, determine the main vehicle data and obstacle data.

[0188] Based on the driving behavior data in the main vehicle data, the main vehicle data is divided into multiple first data segments.

[0189] Based on the obstacle behavior data in the obstacle data, the obstacle data is divided into multiple second data segments.

[0190] In one implementation, the fourth determining module is used to:

[0191] Based on driving behavior data, determine the main vehicle behavior labels for multiple first data segments.

[0192] Based on obstacle behavior data, obstacle behavior labels are determined for multiple second data segments.

[0193] Based on road data in autonomous driving data, determine the road network topology labels for multiple first data segments and multiple second data segments.

[0194] Based on the interaction data in the autonomous driving data, determine the interaction behavior labels of multiple first data segments and multiple second data segments.

[0195] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0196] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0197] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0198] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0199] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0200] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0201] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the method for mining autonomous driving scenarios. For example, in some embodiments, the method for mining autonomous driving scenarios can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the method for mining autonomous driving scenarios described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the method for mining autonomous driving scenarios by any other suitable means (e.g., by means of firmware).

[0202] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0203] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0204] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0205] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0206] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0207] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0208] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0209] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for mining autonomous driving scenarios, comprising: Based on the received mining task, the autonomous driving data to be mined and the target autonomous driving scenario are determined; wherein, the autonomous driving data includes multiple data fragments with labels; Based on the target autonomous driving scenario, at least the label variables, logical variables, and constraint variables in the preset mining architecture are determined; wherein, the constraint variables are used to limit the constraint conditions that need to be met based on the mining of the multiple data fragments; Based on the label variables, the logical variables, and the constraint variables, and using the preset mining architecture, mining logic is generated; and Based on the mining logic, target data segments corresponding to the target autonomous driving scenario are mined from the plurality of data segments; The step of mining the target data segment corresponding to the target autonomous driving scenario from the plurality of data segments according to the mining logic includes: Based on the label variable of the mining logic, candidate data segments corresponding to the label variable are obtained from the plurality of data segments; Based on the time period logical variable in the logical variable, and based on the time period information of the candidate data segment, a first data segment that conforms to the time period logical variable is determined; Based on the comparison operation logic variables in the logic variables, and based on the time period information of the first data segment, a target data segment that conforms to the comparison operation logic variables is determined; or, The step of mining the target data segment corresponding to the target autonomous driving scenario from the plurality of data segments according to the mining logic includes: Based on the label variable of the mining logic, a first candidate data segment corresponding to the label variable is obtained from the plurality of data segments; Based on the constraint variables of the mining logic, a second candidate data segment that meets the constraint variables is determined based on the first candidate data segment. Based on the logical variables of the mining logic, and based on the second candidate data fragment, a target data fragment that conforms to the logical variables is determined.

2. The method according to claim 1, determining the label variables and logical variables in the preset mining architecture, including: Based on the scene objects and scene object behaviors in the target autonomous driving scenario, determine the label variables in the preset mining architecture; Based on the behavioral relationships in the target autonomous driving scenario, determine the logical variables in the preset mining architecture.

3. The method according to claim 1, wherein determining the constraint variables in the preset mining architecture includes: Based on the road and behavioral constraints in the target autonomous driving scenario, the constraint variables in the preset mining architecture are determined.

4. The method according to claim 1, wherein, The step of determining the target data segment that matches the logical variables based on the second candidate data segment, according to the logical variables of the mining logic, includes: Based on the time period logical variable in the logical variable, and based on the time period information of the second candidate data segment, a first data segment that conforms to the time period logical variable is determined; Based on the comparison operation logic variables in the logic variables, and based on the time period information of the first data segment, a target data segment that conforms to the comparison operation logic variables is determined.

5. The method according to any one of claims 1 to 4, further comprising: Based on the autonomous driving data and the time period information of the target data segment, determine the motion parameter information corresponding to the target data segment; The time period information and motion parameter information of the target data segment are transmitted back.

6. The method according to any one of claims 1 to 4, wherein, Before determining the autonomous driving data to be mined and the target autonomous driving scenario based on the received mining task, the process also includes: The autonomous driving data to be mined is divided into multiple data segments; The label of each data segment among the plurality of data segments is determined according to the preset label dimension.

7. The method according to claim 6, wherein, The preset label dimensions include at least one of the following: road network topology dimension, vehicle behavior dimension, obstacle behavior dimension, and interactive behavior dimension.

8. The method according to claim 6, wherein, The process of dividing the autonomous driving data to be mined into multiple data segments includes: From the autonomous driving data to be mined, determine the main vehicle data and obstacle data; Based on the driving behavior data in the main vehicle data, the main vehicle data is divided into multiple first data segments; Based on the obstacle behavior data in the obstacle data, the obstacle data is divided into multiple second data segments.

9. The method according to claim 8, wherein, The step of determining the label of each data segment among the plurality of data segments according to a preset label dimension includes: Based on the driving behavior data, determine the main vehicle behavior labels for the plurality of first data segments; Based on the obstacle behavior data, determine the obstacle behavior labels for the plurality of second data segments; Based on the road data in the autonomous driving data, determine the road network topology labels for the plurality of first data segments and the plurality of second data segments; Based on the interaction data in the autonomous driving data, determine the interaction behavior labels of the plurality of first data segments and the plurality of second data segments.

10. A device for excavating autonomous driving scenarios, comprising: The first determining module is used to determine the autonomous driving data to be mined and the target autonomous driving scenario based on the received mining task; wherein, the autonomous driving data includes multiple data fragments with tags; The second determining module is used to determine at least the label variables, logical variables, and constraint variables in the preset mining architecture based on the target autonomous driving scenario; wherein the constraint variables are used to limit the constraint conditions that need to be met based on the mining of the multiple data fragments. The generation module is used to generate mining logic based on the preset mining architecture, according to the label variables, the logical variables, and the constraint variables; and The mining module is used to mine target data segments corresponding to the target autonomous driving scenario from the plurality of data segments according to the mining logic; The mining module includes: The first acquisition submodule is used to acquire candidate data segments corresponding to the label variables from the plurality of data segments based on the label variables of the mining logic; The fourth determining submodule is used to determine a first data segment that conforms to the time period logical variable based on the time period information of the candidate data segment, according to the time period logical variable in the logical variable; and to determine a target data segment that conforms to the comparison operation logical variable based on the time period information of the first data segment, according to the comparison operation logical variable in the logical variable. or, The mining module includes: The second acquisition submodule is used to acquire a first candidate data segment corresponding to the label variable from the plurality of data segments based on the label variable of the mining logic; The fifth determining submodule is used to determine a second candidate data segment that conforms to the constraint variables based on the first candidate data segment, according to the constraint variables of the mining logic. The sixth determining submodule is used to determine the target data segment that conforms to the logical variables based on the second candidate data segment, according to the logical variables of the mining logic.

11. The apparatus according to claim 10, wherein, The second determining module includes: The first determining submodule is used to determine the label variables in the preset mining architecture based on the scene objects and scene object behaviors in the target autonomous driving scenario; The second determining submodule is used to determine the logical variables in the preset mining architecture based on the behavioral relationships in the target autonomous driving scenario.

12. The apparatus according to claim 11, wherein, The second determining module further includes: The third determining submodule is used to determine the constraint variables in the preset mining architecture based on the road constraint conditions and behavior constraint conditions in the target autonomous driving scenario.

13. The apparatus according to claim 10, wherein, The sixth determining submodule is used for: Based on the time period logical variable in the logical variable, and based on the time period information of the second candidate data segment, a first data segment that conforms to the time period logical variable is determined; Based on the comparison operation logic variables in the logic variables, and based on the time period information of the first data segment, a target data segment that conforms to the comparison operation logic variables is determined.

14. The apparatus according to any one of claims 10 to 13, further comprising: The third determining module is used to determine the motion parameter information corresponding to the target data segment based on the time period information of the autonomous driving data and the target data segment; The feedback module is used to feedback the time period information and motion parameter information of the target data segment.

15. The apparatus according to any one of claims 10 to 13, further comprising: The segmentation module is used to divide the autonomous driving data to be mined into multiple data segments; The fourth determining module is used to determine the label of each data segment among the multiple data segments according to the preset label dimension.

16. The apparatus according to claim 15, wherein, The preset label dimensions include at least one of the following: road network topology dimension, vehicle behavior dimension, obstacle behavior dimension, and interactive behavior dimension.

17. The apparatus according to claim 15, wherein, The partitioning module is used for: From the autonomous driving data to be mined, determine the main vehicle data and obstacle data; Based on the driving behavior data in the main vehicle data, the main vehicle data is divided into multiple first data segments; Based on the obstacle behavior data in the obstacle data, the obstacle data is divided into multiple second data segments.

18. The apparatus according to claim 17, wherein, The fourth determining module is used for: Based on the driving behavior data, determine the main vehicle behavior labels for the plurality of first data segments; Based on the obstacle behavior data, determine the obstacle behavior labels for the plurality of second data segments; Based on the road data in the autonomous driving data, determine the road network topology labels for the plurality of first data segments and the plurality of second data segments; Based on the interaction data in the autonomous driving data, determine the interaction behavior labels of the plurality of first data segments and the plurality of second data segments.

19. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 9.

21. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images