Automatic driving data processing method and system, electronic device and computer storage medium

Abstract

Embodiments of the present application provide an automatic driving data processing method and system, electronic equipment and computer storage medium, wherein the automatic driving data processing method comprises: generating corresponding atomic behavior data and behavior primitives based on a motion topology graph corresponding to a behavior object in a driving scene sample; collecting atomic behaviors indicated by the atomic behavior data for a target object matching the behavior object in a to-be-processed automatic driving scene; and determining whether the motion behavior of the target object conforms to the motion described by the motion topology graph based on the collected atomic behaviors and the behavior primitives. Through the embodiments of the present application, the driving safety of the device with the automatic driving function in different driving scenes can be effectively guaranteed, or the effectiveness and safety of the test case can be effectively judged.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to an autonomous driving data processing method, system, electronic device, and computer storage medium. Background Technology

[0002] With the development of autonomous driving technology, more and more industries and fields are using equipment with autonomous driving capabilities (such as vehicles and aircraft) to perform corresponding tasks, in order to improve work efficiency and reduce the burden of manual labor. In order to ensure the driving safety of these devices in actual operation, autonomous driving testing has become an essential step.

[0003] Current autonomous driving tests primarily utilize test cases. Test cases that demonstrate good performance will provide guidance for the actual driving control of devices with autonomous driving capabilities. However, due to the diversity and complexity of driving scenarios, the implementation of autonomous driving solutions is also flexible and varied. Therefore, determining whether devices with autonomous driving capabilities can be effectively applied to different driving scenarios under the control of their autonomous driving solutions, while ensuring driving safety in these scenarios, has become a pressing issue. Summary of the Invention

[0004] In view of this, embodiments of this application provide an autonomous driving data processing solution to at least partially solve the above-mentioned problems.

[0005] According to a first aspect of the embodiments of this application, an autonomous driving data processing method is provided, comprising: generating corresponding atomic behavior data and behavior primitives based on a motion topology graph corresponding to a behavior object in a driving scenario example; collecting atomic behaviors indicated by the atomic behavior data for a target object in the autonomous driving scenario to be processed that matches the behavior object; and determining whether the motion behavior of the target object conforms to the motion described by the motion topology graph based on the collected atomic behaviors and the behavior primitives.

[0006] According to a second aspect of the embodiments of this application, an autonomous driving data processing system is provided, comprising: a behavior editor and a behavior engine; wherein: the behavior editor is configured to provide a human-machine interface to receive user behavior editing operations on behavior objects in a driving scenario example in the human-machine interface, and generate a corresponding motion topology map; the behavior engine is configured to generate atomic behavior data and behavior primitives corresponding to the behavior objects based on the motion topology map; collect atomic behaviors indicated by the atomic behavior data for a target object in the autonomous driving scenario to be processed that matches the behavior objects; and determine whether the motion behavior of the target object conforms to the motion described by the motion topology map based on the collected atomic behaviors and the behavior primitives.

[0007] According to a third aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the method described in the first aspect.

[0008] According to a fourth aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0009] According to the autonomous driving data processing scheme provided in this application, regardless of whether it is in autonomous driving testing or actual autonomous driving scenarios, and regardless of the specific autonomous driving control algorithm used by the device with autonomous driving function, it has a corresponding behavior sequence in the corresponding driving scenario. This behavior sequence contains both behavioral information and information about the relationships between behaviors. Based on this, a corresponding driving scenario example can be determined first. Based on the motion topology graph corresponding to the behavior sequence of the behavior object in the driving scenario example, corresponding atomic behavior data and behavior primitives can be generated. Thus, when determining whether the corresponding object (i.e., the target object matching the behavior object) in a certain autonomous driving scenario (i.e., the autonomous driving scenario to be processed) has performed a behavior operation that conforms to the scenario, atomic behavior can be collected from the target object in the scenario. Then, based on the collected atomic behaviors and the relationships between these atomic behaviors, the motion behavior of the target object can be determined according to the behavior primitives to determine whether it conforms to the motion described by the motion topology graph, that is, whether it has performed the behavior sequence that can be performed in the autonomous driving scenario. Thus, the device with autonomous driving function can be effectively applied to different driving scenarios under the control of its autonomous driving scheme and can ensure driving safety in these driving scenarios. In the testing scenario, the validity and safety of the test cases can be effectively judged.

[0010] Furthermore, while the above example uses a device with autonomous driving capabilities, in practical applications, the solution of this application embodiment can also be applied to any intelligent agent behavior scenario, such as a device without autonomous driving capabilities, or even to determine whether a pedestrian's behavior is compliant. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0012] Figure 1 A schematic diagram of an exemplary system to which the embodiments of this application are applicable;

[0013] Figure 2A This is a flowchart of the steps of an autonomous driving data processing method according to Embodiment 1 of this application;

[0014] Figure 2B for Figure 2A An example of a motion topology diagram in the illustrated embodiment;

[0015] Figure 2C for Figure 2A A schematic diagram of a scenario example in the illustrated embodiment;

[0016] Figure 3A This is a structural block diagram of an autonomous driving data processing system according to Embodiment 2 of this application;

[0017] Figure 3B for Figure 3A The diagram shows an example implementation of an autonomous driving data processing system in the embodiment shown.

[0018] Figure 4 This is a schematic diagram of the structure of an electronic device according to Embodiment 3 of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0020] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0021] Figure 1 An exemplary system applicable to embodiments of this application is shown. For example... Figure 1 As shown, the system 100 may include a cloud server 102, a communication network 104, and / or one or more user devices 106. Figure 1 The example in the text shows multiple user devices.

[0022] The cloud server 102 can be any suitable device for storing information, data, programs, and / or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud server clusters, etc. In some embodiments, the cloud server 102 can perform any suitable function. For example, in some embodiments, the cloud server 102 can be used to generate corresponding motion topology maps and description files based on multiple driving scenario examples. As an optional example, in some embodiments, the motion topology map generated by the cloud server 102 is displayed in the form of driving scenario icons. As another example, in some embodiments, the motion topology map of the cloud server 102 is generated based on the behavioral data of at least one behavioral object in the driving scenario example and the behavioral relationships between the behavioral data.

[0023] In some embodiments, the communication network 104 may be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 104 may include any one or more of the following: the Internet, intranet, wide area network (WAN), local area network (LAN), wireless network, digital subscriber line (DSL) network, frame relay network, asynchronous transfer mode (ATM) network, virtual private network (VPN), and / or any other suitable communication network. User equipment 106 may be connected to the communication network 104 via one or more communication links (e.g., communication link 112), and the communication network 104 may be linked to the cloud server 102 via one or more communication links (e.g., communication link 114). The communication link can be any communication link suitable for transmitting data between user equipment 106 and cloud server 102, such as a network link, dial-up link, wireless link, hardwired link, any other suitable communication link, or any suitable combination of such links.

[0024] User device 106 may include any one or more user devices suitable for user interaction. For example, in some embodiments, user device 106 may obtain motion topology maps and / or description files from cloud server 102 to generate corresponding atomic behavior data and behavior primitives, and process the user-input autonomous driving scenario to determine whether the target object in the autonomous driving scenario exhibits the motion described by the motion topology map and / or description file. As an optional example, in some embodiments, user device 106 may perform atomic behavior acquisition on target objects in the autonomous driving scenario that match the behavior objects corresponding to the motion topology map, to determine whether the target object in the autonomous driving scenario exhibits the motion described by the motion topology map and / or description file based on the acquired atomic behaviors and behavior primitives. It should be noted that in some scenarios, if user device 106 has high hardware and software performance, as an optional example, in some embodiments, the processing of the cloud server 102 may also be implemented by user device 106. That is, the construction of the motion topology map and subsequent processing of the autonomous driving scenario to be processed are implemented on a single machine. For example, after constructing the corresponding motion topology map and its description file based on multiple driving scenario examples, the user equipment can store them. Subsequently, when needed, the stored data can be retrieved to generate corresponding atomic behavior data and behavior primitives, and the autonomous driving scenario to be processed can be performed based on this. For instance, atomic behavior acquisition is performed on a target object in the autonomous driving scenario that matches the behavior object; then, based on the acquired atomic behavior and the aforementioned behavior primitives, it is determined whether the motion behavior of the target object conforms to the motion described in the motion topology map or description file. In some embodiments, the user equipment 106 may include any suitable type of device. For example, in some embodiments, the user equipment 106 may include a mobile device, tablet computer, laptop computer, desktop computer, and / or any other suitable type of user equipment.

[0025] Based on the above system, the autonomous driving data processing scheme of this application will be described below through several embodiments.

[0026] Example 1

[0027] Reference Figure 2A The diagram shows a flowchart of the steps of an autonomous driving data processing method according to Embodiment 1 of this application.

[0028] The autonomous driving data processing method in this embodiment includes the following steps:

[0029] Step S202: Based on the motion topology graph corresponding to the behavior objects in the driving scenario example, generate the corresponding atomic behavior data and behavior primitives.

[0030] In this embodiment, the driving scenario example can be a real historical driving scenario or a test case specifically used for simulation testing. This embodiment does not limit the specific implementation form of the driving scenario example.

[0031] Driving scenario examples can describe various driving scenarios involved in the autonomous driving process of devices such as vehicles with autonomous driving capabilities. Examples include passing through traffic lights, autonomous driving on highways, autonomous driving on complex roads, overtaking, lane changing, and reaching a destination. Typically, a driving scenario includes roads, objects moving on the road (including but not limited to motor vehicles with or without autonomous driving capabilities, pedestrians, and non-motorized vehicles), and road signs or obstacles. In specific applications, driving scenario examples can be implemented in any suitable form, including but not limited to video data streams or data frames, point cloud data streams or data frames, etc. Any data format that can describe a driving scenario is applicable to the solutions in this application's embodiments.

[0032] Furthermore, based on the above description of the behavioral objects in driving scenarios, it is clear that the solutions in this application are not only applicable to devices with autonomous driving functions, but also to other objects capable of autonomous behavior. However, for ease of description, multiple embodiments of this application use vehicles with autonomous driving functions (hereinafter referred to as autonomous vehicles) as examples for illustration. Those skilled in the art should understand that when the behavioral object is another type of behavioral object, the corresponding solutions and functions can also be implemented with reference to the descriptions in the embodiments of this application.

[0033] A motion topology graph is used to represent the motion of a specific object (such as an autonomous vehicle, a manually driven vehicle, or a pedestrian) in a specific driving scenario. It can include both specific motion behaviors and the relationships between multiple behaviors. Taking an autonomous vehicle as an example, it can include the autonomous vehicle's driving behaviors in the scenario (such as braking, overtaking, lane changing, and arrival), as well as the relationships between multiple driving behaviors (such as changing lanes before arriving). In one feasible approach, nodes in the motion topology graph are represented by driving scenario icons, which include motion behavior icons and behavior relationship icons. These driving scenario icons can be obtained from a pre-set driving scenario icon library, which stores icons corresponding to motion behaviors and icons corresponding to behavior relationships. Therefore, based on the motion behaviors and behavior relationships included in the driving behavior topology graph, the corresponding icon can be matched from the driving scenario icon library. The specific settings of the driving scenario icons can be flexibly configured by those skilled in the art according to actual conditions, as long as they effectively reflect the corresponding driving behaviors or behavior relationships.

[0034] Therefore, in implementing this step, in the first feasible approach, the motion topology graph corresponding to the behavior objects in the pre-constructed driving scenario example can be used to generate the corresponding atomic behavior data and behavior primitives. In the second feasible approach, the motion topology graph corresponding to the behavior objects in the driving scenario example can be constructed manually, and then the corresponding atomic behavior data and behavior primitives can be generated.

[0035] When using the first feasible approach, as mentioned earlier, nodes in the motion topology graph can represent either motion behaviors or relationships between motion behaviors. Although the motion topology graph is presented to the user in graph form on the interface, it is stored in the corresponding device as a graph data structure. Based on this, by parsing the graph data structure, information about each node and its attributes, as well as the relationships between nodes, can be obtained. Furthermore, based on this information, corresponding atomic behavior data and behavior primitives can be generated.

[0036] In autonomous driving, atomic behavior data refers to a complete and indivisible action, such as reversing, emergency braking, following, arriving, passing through traffic lights, lane keeping, lane changing, and meeting other vehicles or obstacles in autonomous driving. Behavioral primitives, on the other hand, are primitives used to represent the execution of actions. A primitive generally refers to a program segment consisting of several instructions used to implement a specific function, and its execution cannot be interrupted. For example, arriving at the destination after changing lanes is an autonomous driving behavior primitive, indicating that when controlling an autonomous vehicle before it reaches its destination, it must first change to the predetermined lane before arriving at the destination; this process must be executed sequentially and cannot be interrupted.

[0037] Based on this, in one feasible approach, corresponding atomic behavior data can be generated based on the information of motion behavior (also atomic behavior) obtained from the motion topology graph; and corresponding behavior primitives can be generated based on the information of behavior relationships obtained from the analysis.

[0038] However, to facilitate and expedite the generation of atomic behavior data and behavior primitives, one feasible approach is to obtain a description file corresponding to the motion topology graph, and then generate the corresponding atomic behavior data and behavior primitives based on this description file. The description file is a machine-readable file that is easy to program, and can be considered an intermediary bridge between the low-level programming language and the upper-level, understandable graphical representation. For example, it can be a text file. This description file can be generated and stored based on the parsing results of the motion topology graph. Optionally, it can also be encapsulated as a dynamically loadable file, such as a DLL (Dynamic Link Library) file or other forms of files. However, it is not limited to this; other forms of description files that effectively describe the motion topology graph, such as data files corresponding to tree structures or graph structures, are also applicable to the solutions in this application.

[0039] Furthermore, to improve the efficiency of generating atomic behavior data and behavior primitives, in addition to obtaining the description file, one feasible approach is to dynamically load the description file corresponding to a driving scenario example that matches the autonomous driving scenario to be processed; and generate the corresponding atomic behavior data and behavior primitives based on the description file. Here, the description file is a description file of the motion topology graph corresponding to the behavior objects in the driving scenario example.

[0040] Taking an autonomous vehicle as an example, the motion topology graph corresponding to this vehicle object represents the driving behavior topology graph of multiple driving behaviors with behavioral relationships in the driving scenario shown in the example driving scenario. Based on the description file, generating the corresponding atomic behavior data and behavior primitives can be achieved as follows: Parse the information of multiple driving behaviors and the behavioral relationship information between multiple driving behaviors from the description file; generate multiple atomic behavior data corresponding to the multiple driving behaviors based on the information of the multiple driving behaviors; and generate the corresponding behavior primitives for the driving behaviors based on the behavior relationship information. Thus, atomic behavior data and behavior primitives applicable to autonomous vehicles can be generated efficiently and quickly.

[0041] Among them, the behavioral relationship includes at least one of the following: a sequential relationship used to indicate the order between two consecutive driving behaviors, a prohibition relationship used to indicate that a driving behavior is prohibited from occurring, and an AND relationship used to indicate that driving behaviors must occur simultaneously.

[0042] In this context, the sequential relationship indicates that two driving actions must be performed sequentially; the prohibition relationship indicates that a certain driving action is prohibited (or cannot) be performed; and the AND relationship indicates that at least two driving actions must be performed simultaneously. For example, for the following driving actions: passing through an intersection, not stopping without cause, not colliding with other road users, not braking suddenly, and reasonably avoiding other road users, behavioral relationship analysis can determine the behavioral relationships as follows: while the driving action of "passing through an intersection" occurs, one cannot "stop without cause," cannot "collide with other road users," cannot "brake suddenly," and must "avoid other road users." The "cannot" and "AND" relationships can represent the behavioral relationships between subsequent driving actions and the "passing through an intersection" driving action. "Cannot" indicates a prohibition relationship, and "AND" indicates an AND relationship, indicating that driving actions must occur simultaneously. When using the second method of self-constructing the motion topology graph and then generating the corresponding atomic behavior data and behavior primitives, the construction of the motion topology graph can be achieved through interaction with the user via a human-computer interface.

[0043] In one feasible approach, the motion of behavioral objects in driving scenario examples can be analyzed manually. Then, based on a human-computer interaction interface, the driving scenario icons corresponding to the motion behaviors and the driving scenario icons corresponding to the behavioral relationships can be visualized and dragged to generate a corresponding motion topology map. In this approach, the device containing the human-computer interaction interface can obtain the user's operations on multiple driving scenario icons (including selection, dragging, and placement), and generate a motion driving topology map based on the multiple driving scenario icons operated on (corresponding to motion behaviors) and the relationships between the multiple driving scenario icons (corresponding to behavioral relationships).

[0044] In another feasible approach, a corresponding driving topology map can be automatically generated based on driving scenario examples. For example, the scene data of driving scenario examples can be automatically analyzed to obtain traffic data corresponding to the traffic behavior of the behavioral object in the driving scenario corresponding to the example; based on the traffic data, the motion behavior of the behavioral object and the behavioral relationships between the motion behaviors are obtained, and a basic motion topology map with behavioral relationships is generated for the behavioral object; driving scenario icons that match the driving behaviors or behavioral relationships represented by the nodes in the basic motion topology map are determined from a preset driving scenario icon library; and a motion topology map is generated based on the matching driving scenario icons and the basic motion topology map. This enables the generation of motion topology maps more efficiently without human intervention.

[0045] Analyzing the scenario data of driving scenario examples can yield traffic data corresponding to the traffic behaviors of objects within the driving scenario. Taking an autonomous vehicle as an example, traffic behaviors can be traffic actions occurring in the driving scenario, such as following other vehicles, overtaking, yielding to pedestrians, and pedestrians crossing traffic lights. The traffic data corresponding to these behaviors can be data on various objects (including but not limited to the objects themselves) and their surrounding environment in the driving scenario at the time of the traffic behavior. In one example, the aforementioned traffic data includes at least one of the following: traffic environment data in the driving scenario, motion behavior data of the objects themselves, and relationship data between the objects themselves and other traffic participants. Specifically, the traffic environment data in the driving scenario is used to indicate data related to the road and other traffic environments, such as whether lane lines are solid lines (if solid, lane changing is not allowed); the speed limits indicated by road signs, requiring the vehicle's speed to match the speed limits indicated by the signs; the specific lane number of the vehicle, requiring it to travel in the left or right lane if a left or right turn is needed, and so on. Motion behavior data of a driving scenario describes the movement of the driving object within the driving environment. Taking an autonomous vehicle as an example, this data could include: driving behavior data such as maintaining a constant speed, pedestrian avoidance, emergency braking, and overtaking. Relationship data between the driving object and other traffic participants describes the relationship between the driving object and other traffic participants, such as vehicles, pedestrians, or non-motorized vehicles. Taking an autonomous vehicle as an example, this data could include location data, whether the driving object needs to avoid other traffic participants, whether the driving object follows other traffic participants, and whether the driving object stays ahead of other traffic participants during its journey.

[0046] Based on this, traffic data can be used to analyze the movement behavior and relationships of the subject, in order to determine the corresponding multiple movement behaviors and relationships. Taking an autonomous vehicle as an example, the analysis shows that the autonomous vehicle's driving behavior and relationships include "passing the intersection" followed by "following the vehicle in front," where "following" indicates the sequential relationship between the two driving behaviors.

[0047] Based on the aforementioned analysis of several driving behaviors and their relationships, a basic motion topology graph (not in icon form) can be obtained. Then, by combining this with a driving scenario icon library, icons corresponding to each driving behavior and its relationships are retrieved. These icons are then used to replace the nodes in the basic motion topology graph to generate the final motion topology graph.

[0048] An exemplary generated motion topology graph is as follows: Figure 2BAs shown in the diagram, the behavioral object in this example is still an autonomous vehicle. The driving scenario in this example is that the autonomous vehicle needs to reach destination A, and destination B is parallel to destination A. The autonomous vehicle must first encounter an obstacle before reaching destination A, and cannot reach destination B, which is adjacent to destination A. Therefore, after analyzing the autonomous vehicle in this driving scenario, the resulting motion topology diagram, i.e., the driving behavior topology diagram, is as follows: Figure 2B As shown, there is a prohibition relationship between destination A and destination B. At the same time (and relationship), the autonomous vehicle must also meet the obstacle (sequential relationship) before reaching destination A.

[0049] As can be seen, this motion topology diagram allows users to easily and cost-effectively understand the motion of objects in a driving scenario. Furthermore, for those annotating driving scenarios, breaking down complex motion behaviors into atomic motion lines and their relationships allows annotators to manually generate motion behavior topologies through drag-and-drop visualization, or automatically generate them by recognizing the driving scenario. This allows for the simple and efficient combination of annotations for arbitrarily complex motion behaviors, solving the problem of annotators struggling to annotate complex scenarios and significantly improving their productivity.

[0050] Step S204: Collect the atomic behaviors indicated by the atomic behavior data for the target object that matches the behavior object in the autonomous driving scenario to be processed.

[0051] The target object that matches the behavior object refers to an object of the same type as the behavior object in the autonomous driving scenario to be processed. For example, if the behavior object is an autonomous vehicle, then the target object must also be a driving vehicle, not a pedestrian or a non-autonomous vehicle.

[0052] For example, if the motion topology graph is for an autonomous vehicle, driving behavior (since the atomic behaviors determined based on the motion topology graph and / or its description file) of a specific autonomous vehicle in the autonomous driving scenario to be processed can be collected. Similarly, if the motion topology graph is for a non-autonomous vehicle, driving behavior (since the atomic behaviors determined based on the motion topology graph and / or its description file) of a specific non-autonomous vehicle in the autonomous driving scenario to be processed can be collected. If the motion topology graph is for a pedestrian, walking behavior (since the atomic behaviors determined based on the motion topology graph and / or its description file) of a specific pedestrian in the autonomous driving scenario to be processed can be collected, such as walking behavior at an intersection.

[0053] It should be noted that atomic behaviors also correspond to specific time information, such as point-in-time information or time period information. Therefore, atomic behavior acquisition also includes the acquisition of the time information corresponding to the atomic behavior.

[0054] For example, the acquisition of atomic behaviors can be achieved using the Runtime algorithm, which can be associated with a target object to obtain information on the real-time motion behavior of the target object in an autonomous driving scenario. However, this is not the only approach; other methods for acquiring atomic behaviors of target objects are also applicable to the solutions in this application embodiment, such as using a three-dimensional target detection model.

[0055] Step S206: Based on the collected atomic behaviors and the behavior primitives, determine whether the motion behavior of the target object conforms to the motion described by the motion topology graph.

[0056] After acquiring the atomic behaviors of the target object and their corresponding time information, the atomic behaviors performed by the target object at a certain point in time or within a certain time period, as well as the behavioral relationships between these atomic behaviors, can be determined based on this data. Then, combined with the behavioral primitives generated from the motion topology diagram or description file, it is determined whether these atomic behaviors and their behavioral relationships satisfy the behavioral relationships indicated by the behavioral primitives. If they do, the motion behavior of the target object is considered to conform to the motion described in the motion topology diagram or the corresponding description file; otherwise, it is considered not to conform.

[0057] Therefore, whether in testing scenarios or in actual driving scenarios, the strategies for controlling the driving behavior of autonomous vehicles can be modified to ensure driving safety.

[0058] The following example uses a specific scenario to illustrate the above process. Figure 2C As shown.

[0059] Figure 2C Taking autonomous vehicles as an example, the process assumes that after manual analysis of driving scenario samples by annotators on a cloud server, a corresponding driving behavior topology map is generated by dragging and dropping driving scenario icons corresponding to the driving behaviors (atomic behaviors) and the behavioral relationships between driving behaviors through a human-computer interaction interface. Further, a corresponding description file is generated based on this driving behavior topology map, and this description file is encapsulated as a dynamically loadable file and distributed to user devices, or provided through download links.

[0060] Based on this, the user device obtains the dynamically loadable description file through distribution or download. Then, based on this description file, atomic behavior data corresponding to driving behavior nodes and behavior primitives corresponding to behavior relationship nodes are generated in the driving behavior topology graph.

[0061] Next, the autonomous driving scenario to be processed is obtained. In this example, it is assumed that the autonomous driving vehicle in the scenario is vehicle X, and vehicle X is the target object that matches the aforementioned behavior object. The driving behavior of vehicle X in the current autonomous driving scenario is collected based on the atomic behavior data, and the corresponding atomic behaviors and their corresponding time information are obtained.

[0062] Based on the atomic behavior and its corresponding time information, the behavioral relationships between multiple atomic behaviors at multiple time points or time periods can be obtained. Based on the obtained atomic behaviors and behavioral relationships, a query is performed to determine whether they match the motion of the behavioral object described in the description file. The determination result is then output.

[0063] As can be seen from this embodiment, regardless of whether it is in autonomous driving testing or actual autonomous driving scenarios, and regardless of the specific autonomous driving control algorithm used by the device with autonomous driving function, it has a corresponding behavior sequence in the corresponding driving scenario. This behavior sequence contains both behavioral information and information about the relationships between behaviors. Based on this, a corresponding driving scenario example can be determined first. Based on the motion topology graph corresponding to the behavior sequence of the behavior object in the driving scenario example, corresponding atomic behavior data and behavior primitives can be generated. Thus, when determining whether the corresponding object (i.e., the target object matching the behavior object) in a certain autonomous driving scenario (i.e., the autonomous driving scenario to be processed) has performed a behavior operation that conforms to the scenario, atomic behaviors of the target object in the scenario can be collected. Then, based on the collected atomic behaviors and the relationships between these atomic behaviors, the behavior primitives are used to determine whether the motion behavior of the target object conforms to the motion described by the motion topology graph, that is, whether it has performed the behavior sequence that can be performed in the autonomous driving scenario. This allows the device with autonomous driving function to be effectively applied to different driving scenarios under the control of its autonomous driving scheme and to ensure driving safety in these driving scenarios. In the testing scenario, the effectiveness and safety of the test cases can be effectively judged.

[0064] Example 2

[0065] This embodiment describes the solution provided in this application from the perspective of a complete system for implementing the above-mentioned autonomous driving data processing.

[0066] Reference Figure 3AThe diagram shows a structural block diagram of an autonomous driving data processing system according to Embodiment 2 of this application.

[0067] like Figure 3A As shown in the figure, the autonomous driving data processing system of this embodiment includes: behavior editor 302 and behavior engine 304.

[0068] in:

[0069] The behavior editor 302 is used to provide a human-computer interaction interface to receive the user's behavior editing operations on the behavior objects in the driving scenario example in the human-computer interaction interface and generate the corresponding motion topology map.

[0070] The behavior engine 304 is used to generate atomic behavior data and behavior primitives corresponding to the behavior object according to the motion topology graph; to collect the atomic behavior indicated by the atomic behavior data for the target object that matches the behavior object in the autonomous driving scenario to be processed; and to determine whether the motion behavior of the target object conforms to the motion described by the motion topology graph based on the collected atomic behavior and the behavior primitives.

[0071] In one feasible approach, the behavior editing operations received by the behavior editor 302 for the behavior objects in the driving scenario example may include editing operations for multiple motion behaviors of the behavior objects and editing operations for the behavior relationships between the multiple motion behaviors.

[0072] In one feasible approach, the behavior editor 302 may include a behavior editing module 3022 and a behavior description module 3024. The behavior editing module 3022 provides a human-computer interaction interface to receive user editing operations on behavior objects in a driving scenario example within the interface, generating a corresponding motion topology diagram. The behavior description module 3024 compiles the motion topology diagram into a machine-readable description file.

[0073] In one feasible approach, the behavior engine 304 includes a behavior construction module 3042, a behavior acquisition module 3044, and a behavior query module 3046.

[0074] in:

[0075] The behavior construction module 3042 is used to generate atomic behavior data and behavior primitives corresponding to behavior objects based on the description file corresponding to the motion topology graph.

[0076] The behavior acquisition module 3044 is used to acquire atomic behaviors indicated by atomic behavior data for target objects that match the behavior objects in the autonomous driving scenario to be processed.

[0077] The behavior query module 3046 is used to determine whether the motion behavior of the target object conforms to the motion described in the description file based on the collected atomic behaviors and the behavior primitives.

[0078] Optionally, the behavior construction module 3042 is used to dynamically load the description file corresponding to the driving scenario example that matches the autonomous driving scenario to be processed; and generate atomic behavior data and behavior primitives corresponding to the behavior object according to the description file; wherein, the description file is the description file of the motion topology graph corresponding to the behavior object in the driving scenario example.

[0079] The following, combined with Figure 3B The autonomous driving data processing system of this embodiment will be described with a specific example.

[0080] Depend on Figure 3B As can be seen, annotators use the behavior editing module 3022 in the behavior editor 302 to perform visual drag-and-drop operations based on the driving scene icons corresponding to motion behaviors and driving scene relationships provided by the human-computer interaction interface, as well as the analysis results of driving scene examples, to generate a motion behavior topology map. The behavior editing module 3022 enables behavior annotation for arbitrarily complex driving scenes.

[0081] The generated motion behavior topology map is translated into a general, machine-readable behavior description file, such as a text file, by the behavior description module 3024. This description file can be packaged into a dynamically loadable file for subsequent dynamic loading and use by the behavior engine 304.

[0082] The behavior construction module 3042 in the behavior engine 304 dynamically loads the description file generated by the behavior editor 302, and builds a series of corresponding behavior primitives and atomic behavior data based on this.

[0083] The behavior acquisition module 3044 in the behavior engine 304, based on the constructed atomic behavior data, uses the Runtime algorithm to acquire atomic behaviors of the target objects corresponding to the behavior objects in the autonomous driving scene to be processed, including atomic behaviors and the time information corresponding to the atomic behaviors.

[0084] The behavior query module 3046 in the behavior engine 304 uses the query engine QueryEngine to perform behavior queries based on the collected atomic rows and their time information, using the behavior primitives constructed by the behavior construction module 3042, and outputs the behavior query results. Because the query engine QueryEngine can perform queries based on behavior primitives and the collected atomic behavior data, it has great reusability and can be reused in various production scenarios such as tagging, slicing, and evaluation.

[0085] The autonomous driving data processing system of this embodiment is used to implement the aforementioned autonomous driving data processing method and has the beneficial effects corresponding to the method embodiments, which will not be elaborated here. Furthermore, the functional implementation of each module in this embodiment is described relatively simply; relevant parts can be referred to the aforementioned method embodiments.

[0086] Example 3

[0087] Reference Figure 4 The diagram shows a structural schematic of an electronic device according to Embodiment 3 of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0088] like Figure 4 As shown, the electronic device may include: a processor 402, a communication interface 404, a memory 406, and a communication bus 408.

[0089] in:

[0090] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0091] Communication interface 404 is used to communicate with other electronic devices or servers.

[0092] The processor 402 is used to execute program 410, specifically to perform the relevant steps in the above-described autonomous driving data processing embodiment.

[0093] Specifically, program 410 may include program code that includes computer operation instructions.

[0094] Processor 402 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0095] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0096] Program 410 may include multiple computer instructions. Specifically, program 410 can cause processor 402 to execute the operations corresponding to the autonomous driving data processing method described in the foregoing method embodiments through multiple computer instructions.

[0097] The specific implementation of each step in procedure 410 can be found in the corresponding descriptions of the steps and units in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0098] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing method embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk.

[0099] This application also provides a computer program product, including computer instructions that instruct a computing device to perform the operation corresponding to the autonomous driving data processing method in the above method embodiments.

[0100] Furthermore, it should be noted that the user-related information (including but not limited to user's device information with autonomous driving function, user's personal information, user's driving behavior information, etc.) and data (including but not limited to sample data used for training the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0101] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0102] The methods described above according to the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and to be stored in a local recording medium, downloaded via a network. Thus, the methods described herein can be stored as software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., random access memory (RAM), read-only memory (ROM), flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses the code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0103] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for specific applications, but such implementations should not be considered beyond the scope of the embodiments of this application.

[0104] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. An autonomous driving data processing method, comprising: Based on the motion topology graph corresponding to the behavior objects in the driving scenario examples, corresponding atomic behavior data and behavior primitives are generated. The atomic behavior data is used to indicate that a certain behavior is a complete and indivisible behavior. The behavior primitives are primitives used to represent the execution of behavior. The primitives are program segments that cannot be interrupted during the execution process. The motion topology graph is used to characterize the motion of the behavior objects in the driving scenario, including: the behavior of the movement behavior or the behavior relationship between multiple movement behaviors. For the target object in the autonomous driving scenario to be processed that matches the behavior object, the atomic behavior data is used to collect the atomic behavior. Based on the collected atomic behaviors and behavioral primitives, it is determined whether the motion behavior of the target object conforms to the motion described by the motion topology graph.

2. The method according to claim 1, wherein, The process of generating corresponding atomic behavior data and behavior primitives based on the motion topology graph corresponding to the behavior objects in the driving scenario includes: Based on the description file of the motion topology graph corresponding to the behavioral objects in the driving scenario, the corresponding atomic behavior data and behavior primitives are generated.

3. The method according to claim 2, wherein, The description file based on the motion topology graph corresponding to the behavioral objects in the driving scenario generates corresponding atomic behavior data and behavior primitives, including: Based on the autonomous driving scenario to be processed, a description file corresponding to a driving scenario example that matches the autonomous driving scenario is dynamically loaded; wherein, the description file is a description file of the motion topology graph corresponding to the behavior object in the driving scenario example; Based on the description file, generate the corresponding atomic behavior data and behavior primitives.

4. The method according to claim 3, wherein, The behavior object is a vehicle object, and the motion topology graph corresponding to the vehicle object is a driving behavior topology graph that represents the vehicle object in the driving scenario shown in the driving scenario example and has a behavioral relationship among multiple driving behaviors. The step of generating corresponding atomic behavior data and behavior primitives based on the description file includes: Information on multiple driving behaviors and the behavioral relationships between these multiple driving behaviors are parsed from the description file. Based on the information of the multiple driving behaviors, generate multiple atomic behavior data corresponding to the multiple driving behaviors; based on the behavior relationship information, generate the corresponding driving behavior primitives.

5. The method according to claim 3, wherein, The behavioral relationships include at least one of the following: a sequential relationship indicating the order between two consecutive driving behaviors, a prohibition relationship indicating that a driving behavior is prohibited from occurring, and an AND relationship indicating that driving behaviors must occur simultaneously.

6. The method according to any one of claims 1-5, wherein, The nodes in the motion topology graph are represented by driving scene icons, which include motion behavior icons and behavior relationship icons.

7. An autonomous driving data processing system, comprising: Behavior editor and behavior engine; in: The behavior editor is used to provide a human-computer interaction interface to receive user behavior editing operations on behavior objects in the driving scenario example in the human-computer interaction interface and generate corresponding motion topology maps. The behavior engine is used to generate atomic behavior data and behavior primitives corresponding to the behavior object based on the motion topology graph. The atomic behavior data indicates that a behavior is a complete and indivisible behavior, and the behavior primitives are primitives used to represent behavior execution; each primitive is a program segment that cannot be interrupted during execution. The motion topology graph characterizes the movement of the behavior object in a driving scenario, including: movement behaviors or the behavioral relationships between multiple movement behaviors. For a target object in the autonomous driving scenario to be processed that matches the behavior object, atomic behaviors indicated by the atomic behavior data are collected. Based on the collected atomic behaviors and behavior primitives, it is determined whether the movement behavior of the target object conforms to the movement described by the motion topology graph.

8. The system according to claim 7, wherein, The behavior engine includes a behavior construction module, a behavior acquisition module, and a behavior query module; in: The behavior construction module is used to generate atomic behavior data and behavior primitives corresponding to the behavior object based on the description file corresponding to the motion topology graph. The behavior acquisition module is used to acquire the atomic behaviors indicated by the atomic behavior data for the target object that matches the behavior object in the autonomous driving scenario to be processed. The behavior query module is used to determine whether the motion behavior of the target object conforms to the motion described in the description file corresponding to the motion topology graph, based on the collected atomic behaviors and the behavior primitives.

9. The system according to claim 8, wherein, The behavior construction module is used to dynamically load a description file corresponding to a driving scenario example that matches the autonomous driving scenario to be processed; and generate atomic behavior data and behavior primitives corresponding to the behavior object based on the description file; wherein, the description file is a description file of the motion topology graph corresponding to the behavior object in the driving scenario example.

10. The system according to claim 8 or 9, wherein, The behavior editor includes a behavior editing module and a behavior description module; in, The behavior editing module is used to provide a human-computer interaction interface to receive the user's behavior editing operations on the behavior objects in the driving scenario example in the human-computer interaction interface and generate the corresponding motion topology map. The behavior description module is used to compile the motion topology graph into a machine-readable description file.

11. The system according to claim 10, wherein, The behavior editing operations include editing operations on multiple motion behaviors of a behavior object and editing operations on the behavioral relationships between the multiple motion behaviors.

12. An electronic device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to the method as described in any one of claims 1-6.

13. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-6.

14. A computer program product comprising computer instructions that instruct a computing device to perform an operation corresponding to any one of the methods described in claims 1-6.

Images