A task processing method, device, electronic device and storage medium

Abstract

The embodiment of the present invention discloses a task processing method, device, electronic device and storage medium. The task processing method comprises: obtaining sensor data streams output by at least two sensors and adding frame numbers to the sensor data; obtaining a directed acyclic graph, the directed acyclic graph comprising multiple task nodes and execution paths of the task nodes, the task nodes comprising synchronization nodes; inputting sensor data with the same frame number to execute subtasks corresponding to the task nodes according to the execution paths and sensor data in the directed acyclic graph, the subtasks of the synchronization nodes being used to synchronize the output data of the subtasks corresponding to the parent nodes of the synchronization nodes, so that the output data of each task node in the directed acyclic graph can be synchronized, and the correct execution order of each task node is ensured through the directed acyclic graph, so that the perception module can output the perception results to the decision module at a stable frequency, thereby ensuring the accuracy and security of the decision made by the decision module.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of autonomous driving technology, and in particular to a task processing method, device, electronic device and storage medium. Background Art

[0002] In autonomous driving vehicles, sensors are needed to perceive the environment around the vehicle to obtain perception information to control the vehicle, such as controlling vehicle driving, obstacle avoidance, parking, etc.

[0003] At present, autonomous vehicles are usually equipped with a variety of different sensors to use the data from a variety of different sensors to analyze and identify road conditions and provide stable obstacle information to the vehicle decision-making module. However, the traditional perception operation mode is that each sensor has a corresponding process processing logic. When an autonomous vehicle is equipped with multiple sensors, the data perceived by the multiple sensors are not synchronized, resulting in the output of each sensor algorithm in the perception module being out of sync, and the entire perception module cannot stably output the perception results to the decision-making module, affecting the accuracy and safety of the decision-making module. Summary of the invention

[0004] The embodiments of the present invention provide a task processing method, device, electronic device and storage medium to solve the problem that the data sensed by multiple sensors on existing unmanned vehicles are not synchronized, resulting in the output of each algorithm in the perception module being not synchronized, and the entire perception module cannot stably output the perception results to the decision module, affecting the accuracy and safety of the decision-making module.

[0005] In a first aspect, an embodiment of the present invention provides a task processing method, including:

[0006] Acquire a sensor data stream output by at least two sensors, wherein the sensor data stream includes multiple frames of sensor data;

[0007] Adding frame numbers to the sensor data;

[0008] Obtaining a directed acyclic graph, the directed acyclic graph comprising a plurality of task nodes and execution paths of the task nodes, the task nodes comprising synchronization nodes;

[0009] Input sensor data with the same frame number to execute the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph until the subtasks of all task nodes in the directed acyclic graph are executed, wherein the subtask of the synchronization node is used to add the frame number to the output data of the subtask corresponding to the parent node of the synchronization node for data synchronization.

[0010] In a second aspect, an embodiment of the present invention provides a task processing device, including:

[0011] A sensor data acquisition module, used to acquire a sensor data stream output by at least two sensors, wherein the sensor data stream includes multiple frames of sensor data;

[0012] A frame number adding module, used for adding frame numbers to the sensing data;

[0013] A directed acyclic graph acquisition module, used to acquire a directed acyclic graph, wherein the directed acyclic graph includes a plurality of task nodes and execution paths of the task nodes, wherein the task nodes include synchronization nodes;

[0014] A task execution module is used to input sensor data with the same frame number to execute the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph until the subtasks of all task nodes in the directed acyclic graph are executed, wherein the subtask of the synchronization node is used to add the frame number to the output data of the subtask corresponding to the parent node of the synchronization node for data synchronization.

[0015] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0016] one or more processors;

[0017] a memory for storing one or more computer programs;

[0018] When the one or more computer programs are executed by the one or more processors, the one or more processors implement the task processing method as described in the first aspect.

[0019] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the task processing method as described in the first aspect is implemented.

[0020] The task processing method of the embodiment of the present invention obtains a directed acyclic graph, which includes a plurality of task nodes and an execution path of the task nodes, wherein the task node includes a synchronization node, and after obtaining the sensor data streams of at least two sensors, adds a frame sequence number to the sensor data in the sensor data stream, inputs the sensor data with the same frame sequence number, and executes the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph, until the subtasks of all task nodes in the directed acyclic graph are executed, wherein the subtask of the synchronization node is used to add a frame sequence number to the output data of the subtask corresponding to the parent node of the synchronization node for data synchronization. Since the synchronization node is provided in the directed acyclic graph, the output data of the subtask corresponding to the parent node of the synchronization node is added with the frame sequence number of the sensor data, so that the output data of each task node in the directed acyclic graph can be synchronized, and the correct execution order of each task node is guaranteed by the directed acyclic graph, so that the perception module can output the perception result to the decision module at a stable frequency, and the accuracy and security of the decision of the decision module are guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flowchart of a task processing method provided in Embodiment 1 of the present invention;

[0022] Figure 2A A flowchart of a task processing method provided in Embodiment 2 of the present invention;

[0023] Figure 2B is a schematic diagram of a directed acyclic graph in an embodiment of the present invention;

[0024] Figure 2C It is a schematic diagram of performing task processing based on a main thread and a thread pool thread in an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of the structure of a task processing device provided in Embodiment 3 of the present invention;

[0026] Figure 4 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0028] Embodiment 1

[0029] Figure 1This is a flowchart of a task processing method provided in Embodiment 1 of the present invention. The embodiment of the present invention can be applied to the case where a sensing module is controlled to process a sensing task. The method can be executed by a task processing device. The task processing device can be implemented by software and / or hardware. The task processing device can be configured in an electronic device. The method specifically includes the following steps:

[0030] S101. Acquire a sensor data stream output by at least two sensors, where the sensor data stream includes multiple frames of sensor data.

[0031] Specifically, multiple types of sensors can be installed on an unmanned vehicle, and the number of each type of sensor can be one or more. By way of example, the sensor can include a lidar and multiple cameras. The sensor collects sensor data at a preset frequency to output a sensor data stream, that is, the sensor data stream includes multiple frames of sensor data. Taking the lidar as an example, the lidar can output a point cloud data stream after scanning at a preset frequency. For multiple cameras, an image stream can be formed after each camera exposes and collects images in a certain order.

[0032] Of course, the sensors are not limited to lidar and cameras, but may also include infrared sensors, millimeter wave radars, etc. The embodiments of the present invention do not limit the type and quantity of sensors.

[0033] S102: Add frame numbers to the sensing data.

[0034] In practical applications, each frame of sensor data in the sensor data stream may carry a timestamp, which may be the timestamp when the sensor data is collected. For example, the timestamp of the point cloud data output by the lidar may be the scan end time when the lidar completes each scan, or any time within the scan cycle. For image data output by the camera, the timestamp of the image data may be the exposure time of the camera.

[0035] For each sensor data output by the sensor, a frame number can be added to the sensor data through the timestamp of the sensor data. For example, the timestamp of the sensor data increases as the sensor continues to output, and the frame number can be added according to the sequence of the timestamps in the sensor data, and the frame number increases.

[0036] S103: Obtain a directed acyclic graph, wherein the directed acyclic graph includes a plurality of task nodes and execution paths of the task nodes, and the task nodes include synchronization nodes.

[0037] A directed acyclic graph can be a directed acyclic graph in which a perception model performs perception tasks. A directed acyclic graph refers to a directed graph without loops. If there is a non-directed acyclic graph, and node A starts from node B and can return to node A via node C to form a loop, changing the direction of the edge from node C to node A to from node A to node C will turn it into a directed acyclic graph.

[0038] Among them, the perception module can be a general term for the execution code used to perform the perception task, and can also be an integration of hardware or software that can perform the perception task. When the perception module performs the perception task, the perception task has multiple subtasks, and each subtask has an execution order, set input and output. The execution code and running logic of the perception module can be analyzed to obtain a directed acyclic graph of the perception module, which includes multiple task nodes and execution paths between task nodes. Among them, the task node includes a synchronization node, which is used to synchronize the output data of the direct upstream node (parent node) of the synchronization node. In an example, synchronizing the output data can be adding a frame number to the output data, and the frame number is the frame number of the sensor input to the perception module, so that the data of the direct upstream node of the synchronization node is synchronized.

[0039] S104. Input the sensor data with the same frame number to execute the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph until the subtasks of all task nodes in the directed acyclic graph are executed, wherein the subtask of the synchronization node is used to add the frame number to the output data of the subtask corresponding to the parent node of the synchronization node for data synchronization.

[0040] Since the directed acyclic graph includes multiple task nodes and execution paths between task nodes, when the sensor data is input into the perception module, the perception module is triggered to perform the perception task. First, the sensor data is input into the upstream task node, and then each task node is executed from top to bottom according to the execution path between the task nodes. After all task nodes are executed, the perception information is output at the downstream task node to the decision module for decision-making.

[0041] A synchronization node may be provided in a directed acyclic graph, and the synchronization node has at least two parent nodes. When the subtasks corresponding to at least two parent nodes are executed, data is outputted respectively. However, the time lengths for the subtasks corresponding to the parent nodes to process data may be different, resulting in inconsistency in the output data of the parent nodes. In order to synchronize, the output data of the parent node needs to be synchronized through the subtasks corresponding to the synchronization node. In an example, a frame number may be added to the output data of the parent node, and the frame number is the frame number of the input sensor data, so that the output data of the parent node is synchronized.

[0042] The task processing method of the embodiment of the present invention obtains a directed acyclic graph, which includes a plurality of task nodes and an execution path of the task nodes, wherein the task node includes a synchronization node, and after obtaining the sensor data streams of at least two sensors, adds a frame sequence number to the sensor data in the sensor data stream, inputs the sensor data with the same frame sequence number, and executes the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph, until the subtasks of all task nodes in the directed acyclic graph are executed, wherein the subtask of the synchronization node is used to add a frame sequence number to the output data of the subtask corresponding to the parent node of the synchronization node for data synchronization. Since the synchronization node is provided in the directed acyclic graph, the output data of the subtask corresponding to the parent node of the synchronization node is added with the frame sequence number of the sensor data, so that the output data of each task node in the directed acyclic graph can be synchronized, and the correct execution order of each task node is guaranteed by the directed acyclic graph, so that the perception module can output the perception result to the decision module at a stable frequency, and the accuracy and security of the decision of the decision module are guaranteed.

[0043] Embodiment 2

[0044] Figure 2A This is a flowchart of a task processing method provided in Embodiment 2 of the present invention. The embodiment of the present invention is optimized based on the above embodiment. The method specifically includes the following steps:

[0045] S201. Acquire a sensor data stream output by at least two sensors, where the sensor data stream includes multiple frames of sensor data.

[0046] Various types of sensors may be installed on an unmanned vehicle, and the number of each type of sensor may be one or more. The embodiment of the present invention is described using sensors including a laser radar and a camera as an example.

[0047] The laser radar collects point cloud data at a preset frequency to output a point cloud data stream, which includes multiple frames of point cloud data. The camera exposes and outputs an image data stream at a preset period, which includes multiple frames of image data.

[0048] S202: Add frame numbers to the sensing data.

[0049] In an optional embodiment of the present invention, when the sensor data streams from at least two sensors are received, the timestamp of each frame of sensor data in the sensor data stream is obtained, and the sensor data with the same timestamp are added with the same frame sequence number.

[0050] Taking laser radar and camera as an example, when the laser radar rotates to the optical center axis of the camera, the camera exposes and collects image data. The time when the laser radar scans the optical center axis of the camera can be used as the timestamp of the laser point cloud data and image data, and the laser point cloud data and image data with the same timestamp can be added with the same frame number.

[0051] For example, the laser radar outputs 3 frames of point cloud data at 0.2 milliseconds, 0.4 milliseconds, and 0.6 milliseconds, respectively. At the same time, the camera exposes 3 frames of image data at 0.2 milliseconds, 0.4 milliseconds, and 0.6 milliseconds. Then, frame number 1 can be added to the point cloud data and image data output at 0.2 milliseconds, frame number 2 can be added to the point cloud data and image data output at 0.4 milliseconds, frame number 3 can be added to the point cloud data and image data output at 0.6 milliseconds, and so on. As the sensor data increases, the frame number also increases.

[0052] It should be noted that, in another optional embodiment, the timestamps of the sensor data output by different sensors may be different. For each sensor, the frame number of the sensor data can be determined according to the order of the timestamps of the sensor data output by the sensor. For example, the laser radar outputs 3 frames of point cloud data at 0.2 milliseconds, 0.4 milliseconds, and 0.6 milliseconds respectively, and frame numbers 1, 2, and 3 are added to the point cloud data output at 0.2 milliseconds, 0.4 milliseconds, and 0.6 milliseconds respectively. At the same time, the camera outputs 3 frames of image data at 0.3 milliseconds, 0.5 milliseconds, and 0.7 milliseconds respectively, and frame numbers 1, 2, and 3 are added to the image data output at 0.3 milliseconds, 0.5 milliseconds, and 0.7 milliseconds respectively. The embodiment of the present invention does not limit the method of adding frame numbers to the sensor data, so that the sensor data output by different sensors can correspond one to one.

[0053] S203. Obtain a directed acyclic graph.

[0054] In an optional embodiment of the present invention, a perception task of a perception module can be obtained, which perception task includes multiple sub-tasks, and the execution order between the sub-tasks is determined. A directed acyclic graph is generated based on the sub-tasks and the execution order. Each task node in the directed acyclic graph corresponds to a sub-task, and the connection relationship between the task nodes represents the execution path between the task nodes.

[0055] Specifically, the operating logic and execution code of the perception module can be analyzed to obtain multiple subtasks of the perception module and the operating logic relationship, such as Figure 2B An example of a directed acyclic graph of a perception module is shown, in which the task nodes include a lidar receiving module, a camera receiving module, an obstacle recognition module, a headlight recognition module, a synchronization module, an obstacle trajectory tracking module, and a perception output module.

[0056] Figure 2B In the diagram, each task node is connected by a directed connection line. If a task node has a direct upstream node, the input data of the task node is the output data of the direct upstream node. A task node can have one or more direct upstream nodes. The directed acyclic graph of the perception module can be generated in advance, for example, manually edited and input into a computer device. Of course, it can also be automatically generated after software analyzes the running code of the perception module.

[0057] Among them, the task node of the synchronization module can be set according to actual conditions. For example, if a task node has more than one direct upstream node, a task node of the synchronization module can be added between the task node and the upstream node. For example, if the task node has more than one direct upstream node and is time-sensitive, a synchronization node of the synchronization module can be added between the task node and the direct upstream node.

[0058] S204 , inputting sensor data with the same sequence number as the current frame, so as to sequentially obtain input data and execution codes of the task nodes on the execution path according to the execution path.

[0059] In an optional embodiment of the present invention, a directed acyclic graph represents the execution order and logical relationship of each subtask in the perception task. Each time a task node is executed, a subtask of the task node can be generated. After the sensor data with the same sequence number as the current frame is input into the perception module, the perception module can be triggered to execute the perception task. For each task node in the directed acyclic graph, the parent node of the task node is determined, the output data of the parent node is obtained as the input data of the task node, and the execution code of the task node is obtained.

[0060] In one example, when the task node is a sensor input node, the sensor is used as a parent node, and the main thread is called to obtain the sensor data output by the sensor as the input data of the sensor input node, such as Figure 2B As shown, the task node where the laser radar receiving module and the camera receiving module are located is the sensor input node, the parent node of the laser radar receiving module is the laser radar, and the parent node of the camera receiving module is the camera. The point cloud data output by the laser radar can be obtained as the input data of the laser radar receiving module, and the image data output by the camera can be obtained as the input data of the camera receiving module.

[0061] The execution code of a task node may be the code required to execute the subtask corresponding to the task node. For example, if the subtask of a laser radar receiving module is to receive the point cloud data output by the laser radar and process the received point cloud data, then the execution code of the laser radar receiving module is the code for processing the point cloud data.

[0062] In another example, when the task node is a synchronization node, the parent node of the synchronization node is determined, and the output data of the subtask corresponding to the parent node after being executed is obtained as the input data of the synchronization node. Specifically, Figure 2B As shown, the synchronization module has parent nodes, namely, an obstacle recognition module and a headlight recognition module. The output data of the obstacle recognition module and the output data of the headlight recognition module can be obtained as input data of the synchronization module, and a code for synchronizing the output data of the obstacle recognition module and the output data of the headlight recognition module can be obtained as an execution code.

[0063] In practical applications, each task node in the directed acyclic graph is provided with a message name of the input data and a message name of the output data of the task node, and each task node can extract corresponding input data from the memory by the message name of the input message.

[0064] Taking the sensor data output by the sensor as an example, the sensor data carries a timestamp and a message name, where the message name indicates the type of sensor and sensor number from which the sensor data comes, so that the most upstream sensor input node in the directed acyclic graph knows what type of sensor output data should be input.

[0065] And Figure 2B Taking the obstacle recognition module in as an example, the input data of the obstacle recognition module can be set as the point cloud data output by the lidar receiving module and the image data output by the camera receiving module, and the output data of the obstacle recognition module can be set as obstacle information.

[0066] S205: Encapsulate the input data and the execution code as subtasks of the task node.

[0067] In the embodiment of the present invention, the input data and execution code of each task node may be encapsulated in a standard task container to obtain the subtask corresponding to the task node.

[0068] In one example, for the most upstream sensor input node in the directed acyclic graph, the sensor input node input data and execution code may be encapsulated as subtasks through the main thread.

[0069] In another example, for a synchronization node, a thread pool thread is called to encapsulate the input data and execution code of the synchronization node into a subtask of the synchronization node, so that after the subtask of the synchronization node is executed, the frame number of the output data of the parent node of the synchronization node is the current frame number.

[0070] S206: Add the subtask to the task queue.

[0071] The task queue can be a task queue of a thread pool. The tasks in the task queue are tasks to be executed. In the task queue, the tasks that enter the task queue first will be executed first. After the subtask of the task node in the directed acyclic graph is generated, the subtask can be added to the task queue to wait for execution by the thread of the thread pool.

[0072] S207 , calling a thread to execute a subtask in the task queue to obtain output data after the subtask is executed, until the subtasks of all task nodes in the directed acyclic graph are executed.

[0073] In an optional embodiment of the present invention, the subtask that is first added to the task queue can be determined from the task queue as the target subtask. When a preset event is detected, the idle thread in the thread pool is called to execute the target subtask to obtain the output data of the target subtask, and it is determined whether the task node corresponding to the target subtask has a downstream task node. If the task node has a downstream node, it is determined whether the timer has timed out. The timer is started when the sensor data input perception module has the same sequence number as the current frame. If the timer has not timed out, the execution code of the downstream node is obtained, and the output data and the execution code are encapsulated as a subtask and added to the task queue. If the timer times out, the data generated by the sensor data input perception module with the same sequence number as the current frame is destroyed. If the task node has no downstream node, after the subtasks of all task nodes in the directed acyclic graph are executed, the data generated by the sensor data input perception module with the same sequence number as the current frame is destroyed.

[0074] like Figure 2C The following is the execution principle of the task processing of the perception module: Figure 2C and Figure 2B As shown, after the main thread of the perception module receives the sensor data, Figure 2B The laser radar receiving module and the camera receiving module receive the i-th frame of point cloud data and the i-th frame of image data respectively, then the i-th frame of point cloud data and the execution code of the laser radar receiving module are encapsulated as a subtask and pushed to the task queue of the thread pool, and the i-th frame of image data and the execution code of the camera receiving module are encapsulated as a subtask and pushed to the task queue of the thread pool, and then the main thread continues to receive the i+1-th frame of point cloud data and the i+1-th frame of image data, and repeats the above process.

[0075] like Figure 2CAs shown, in the thread pool, a preset event is set. The preset event can be one of the following: a new task enters the task queue, the system time is the preset time, the preset cycle ends, and the number of tasks in the task queue is greater than the preset value. If a preset event is detected, a thread call signal is sent, that is, when there is an idle thread in the thread pool and a call signal is received, the first task that enters is obtained from the task queue, and the thread pool thread is called to execute the task. After the task is completed, it is determined whether the task node corresponding to the task has a downstream node in the directed acyclic graph. If there is a downstream node, the output data of the task node and the execution code of the downstream node are encapsulated as a new task and added to the task queue of the thread pool. If there is no downstream node, wait for notification.

[0076] After each task node's task is executed, the logic in the following table is executed:

[0077]

[0078] by Figure 2B The directed acyclic graph shown in the figure is used as an example. When the i-th frame point cloud data and the i-th frame image data are input into the perception module, a timer is started. After the thread pool thread executes the synchronization module to output data, it can be determined whether the synchronization module has a downstream task node. Figure 2B It can be known that the synchronization module has a downstream task node obstacle trajectory tracking module, and then further determines whether the timer has timed out. If the timer has not timed out, the execution code of the downstream task node obstacle trajectory tracking module is obtained, and the output data of the synchronization module and the execution code of the obstacle trajectory tracking module are encapsulated as subtasks and added to the task queue. If the timer times out, the i-th frame point cloud data and the i-th frame image data input to the data generated by the perception module are destroyed.

[0079] If a task has no downstream nodes after execution, such as Figure 2B The perception output module shown has no downstream nodes, which indicates that the node is an output node. All subtasks of the perception module have been executed and the perception results are output to the decision module. The data generated by the sensor data input perception module with the same current frame number can be destroyed.

[0080] The embodiment of the present invention obtains a directed acyclic graph provided with a synchronization node, and after obtaining the sensor data streams of at least two sensors, adds a frame sequence number to the sensor data in the sensor data stream, inputs the sensor data with the same frame sequence number, and executes the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph, until the subtasks of all task nodes in the directed acyclic graph are executed, wherein the subtask of the synchronization node is used to add a frame sequence number to the output data of the subtask corresponding to the parent node of the synchronization node for data synchronization. The output data of the subtask corresponding to the parent node of the synchronization node is added with the frame sequence number of the sensor data through the synchronization node, so that the output data of each task node in the directed acyclic graph can be synchronized, and the correct execution order of each task node is guaranteed by the directed acyclic graph, so that the perception module can output the perception result to the decision module at a stable frequency, thereby ensuring the accuracy and security of the decision of the decision module.

[0081] Furthermore, the subtasks of the sensor input nodes in the directed acyclic graph are encapsulated through the main thread of the perception module and pushed to the task queue of the thread pool. The tasks in the task queue are executed by the thread pool threads. There is no need to allocate dedicated threads for the subtasks of each task node in the directed acyclic graph. The number of threads in the thread pool is usually less than the number of physical cores of the CPU. The use of the thread pool can effectively reduce the thread switching time, and the thread utilization rate is high, which improves the CPU utilization rate, reduces the delay in the output of the perception results to the decision module, and improves the security and accuracy of the decision-making module.

[0082] Furthermore, a timer is set to determine whether the current frame of sensor data has timed out after being input into the perception module. If the timer has timed out, the data generated by the current frame of sensor data input into the perception module is destroyed, and the next frame of sensor data is run in time, thereby avoiding the problem that the current frame of sensor data input into the perception module takes too long to output the perception result, resulting in unstable output of the perception module and reduced decision delay of the decision module, thereby improving the real-time nature of the decision-making module.

[0083] Embodiment 3

[0084] Figure 3 This is a structural diagram of a task processing device provided in Embodiment 3 of the present invention. The task processing device may specifically include:

[0085] A sensor data acquisition module 301 is used to acquire a sensor data stream output by at least two sensors, wherein the sensor data stream includes multiple frames of sensor data;

[0086] A frame number adding module 302, used to add frame numbers to the sensing data;

[0087] A directed acyclic graph acquisition module 303, used to acquire a directed acyclic graph, wherein the directed acyclic graph includes a plurality of task nodes and execution paths of the task nodes, wherein the task nodes include synchronization nodes;

[0088] The task execution module 304 is used to input the sensor data with the same frame number to execute the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph until the subtasks of all task nodes in the directed acyclic graph are executed, wherein the subtask of the synchronization node is used to add the frame number to the output data of the subtask corresponding to the parent node of the synchronization node for data synchronization.

[0089] Optionally, the directed acyclic graph acquisition module 303 includes:

[0090] A perception task acquisition submodule, used to acquire a perception task, wherein the perception task includes subtasks;

[0091] A subtask execution order determination submodule, used to determine the execution order between the subtasks;

[0092] A directed acyclic graph generation submodule is used to generate a directed acyclic graph based on the subtasks and the execution order, each task node in the directed acyclic graph corresponds to a subtask, and the connection relationship between the task nodes represents the execution path between the task nodes.

[0093] Optionally, the frame number adding module 302 includes:

[0094] A timestamp acquisition submodule, used for acquiring the timestamp of each frame of sensor data in the sensor data stream when receiving the sensor data streams of at least two sensors;

[0095] The frame number adding submodule is used to add the same frame number to the sensing data with the same timestamp.

[0096] Optionally, the task execution module 304 includes:

[0097] An input data and execution code acquisition submodule, used to input the sensor data with the same sequence number as the current frame, so as to sequentially acquire the input data and execution code of the task nodes on the execution path according to the execution path;

[0098] A subtask encapsulation submodule, used for encapsulating the input data and the execution code into a subtask of the task node;

[0099] A task queue adding submodule, used for adding the subtask to the task queue;

[0100] The thread calling submodule is used to call a thread to execute a subtask located in the task queue to obtain output data after the subtask is executed, until the subtasks of all task nodes in the directed acyclic graph are executed.

[0101] Optionally, the input data and execution code acquisition submodule includes:

[0102] A sensor data input unit, used to input sensor data with the same sequence number as the current frame;

[0103] A parent node determination unit, used for determining the parent node of each task node in the directed acyclic graph;

[0104] An input data acquisition unit, used to acquire the output data of the parent node as the input data of the task node;

[0105] The execution code acquisition unit is used to acquire the execution code of the task node.

[0106] Optionally, the task node includes at least two sensor input nodes at the most upstream, and the parent node determination unit includes:

[0107] A first parent node determination subunit, configured to use the sensor as a parent node when the task node is a sensor input node;

[0108] The input data acquisition unit comprises:

[0109] The main thread calling subunit is used to call the main thread to obtain the sensor data output by the sensor as the input data of the sensor input node;

[0110] The subtask encapsulation submodule includes:

[0111] The first subtask encapsulation unit is used to encapsulate the sensor input node input data and the execution code into subtasks through the main thread.

[0112] Optionally, the parent node determination unit includes:

[0113] A second parent node determination subunit, used for determining the parent node of the synchronization node when the task node is a synchronization node;

[0114] The input data acquisition unit comprises:

[0115] An input data acquisition subunit, used to acquire output data of the subtask corresponding to the parent node after execution as input data of the synchronization node;

[0116] The subtask encapsulation submodule includes:

[0117] The subtask encapsulation unit is used to call the thread pool thread to encapsulate the input data of the synchronization node and the execution code of the synchronization node into the subtask of the synchronization node. After the subtask of the synchronization node is executed, the frame number of the output data of the parent node of the synchronization node is the current frame number.

[0118] Optionally, the thread calling submodule includes:

[0119] a target subtask determining unit, configured to determine, from the task queue, a subtask first added to the task queue as a target subtask;

[0120] A thread pool thread calling unit is used to call an idle thread in the thread pool to execute the target subtask and obtain output data of the target subtask when a preset event is detected;

[0121] A first judging unit, used to judge whether the task node corresponding to the target subtask has a downstream task node;

[0122] A timeout determination unit, used to determine whether a timer has timed out, the timer being started when sensing data having the same sequence number as the current frame is input;

[0123] A downstream node execution code acquisition unit, configured to acquire the execution code of the downstream node if the timer has not timed out;

[0124] A task encapsulation unit, used for encapsulating the output data and the execution code into subtasks and adding them to the task queue;

[0125] a first data destroying unit, configured to destroy data generated by sensor data having the same sequence number as the current frame if the timer times out;

[0126] The second data destruction unit is used to destroy the data generated by the sensor data with the same sequence number as the current frame after all the subtasks of all the task nodes in the directed acyclic graph are executed.

[0127] The task processing device provided in the embodiment of the present invention can execute the task processing method provided in the first embodiment or the second embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0128] Embodiment 4

[0129] Figure 4 This is a schematic diagram of the structure of an electronic device provided by Embodiment 4 of the present invention. Figure 4 As shown, the electronic device includes a processor 400, a memory 401, a communication module 402, an input device 403 and an output device 404; the number of the processor 400 in the electronic device can be one or more. Figure 4A processor 400 is taken as an example; the processor 400, the memory 401, the communication module 402, the input device 403 and the output device 404 in the electronic device can be connected through a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0130] The memory 401 is a computer-readable storage medium that can be used to store software programs, computer executable programs, and modules, such as the modules corresponding to the task processing method of the embodiment of the present invention (for example, Figure 3 The sensor data acquisition module 301, the frame number adding module 302, the directed acyclic graph acquisition module 303 and the task execution module 304 in the task processing device shown in the figure). The processor 400 executes various functional applications and task processing of the electronic device by running the software programs, instructions and modules stored in the memory 401, that is, realizes the above-mentioned task processing method.

[0131] The memory 401 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 401 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 401 may further include a memory remotely arranged relative to the processor 400, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0132] The communication module 402 is used to establish a connection with the display screen and implement data interaction with the display screen.

[0133] The input device 403 can be used to receive input digital or character information and generate key signal input related to user settings and function control of the electronic device. It can also be a laser radar for acquiring point cloud data and a camera for acquiring image data.

[0134] The output device 404 may include an audio device such as a speaker.

[0135] It should be noted that the specific composition of the input device 403 and the output device 404 can be set according to actual conditions.

[0136] The processor 400 executes various functional applications and task processing of the device by running the software programs, instructions and modules stored in the memory 401, that is, implements the above-mentioned task processing method.

[0137] The electronic device provided in this embodiment can execute the task processing method provided in the embodiment of the present invention, and has corresponding functions and beneficial effects.

[0138] Embodiment 5

[0139] Embodiment 5 of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, a task processing method is implemented. The method includes:

[0140] Acquire a sensor data stream output by at least two sensors, wherein the sensor data stream includes multiple frames of sensor data;

[0141] Adding frame numbers to the sensor data;

[0142] Obtaining a directed acyclic graph, the directed acyclic graph comprising a plurality of task nodes and execution paths of the task nodes, the task nodes comprising synchronization nodes;

[0143] Input sensor data with the same frame number to execute the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph until the subtasks of all task nodes in the directed acyclic graph are executed, wherein the subtask of the synchronization node is used to add the frame number to the output data of the subtask corresponding to the parent node of the synchronization node for data synchronization.

[0144] Of course, the computer-readable storage medium provided in the embodiment of the present invention and its computer program are not limited to the method operations described above, but can also execute related operations in the task processing method provided in any embodiment of the present invention.

[0145] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform the task processing method described in each embodiment of the present invention.

[0146] It is worth noting that in the embodiment of the above-mentioned task processing device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0147] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A task processing method, characterized in that: include: Acquire a sensor data stream output by at least two sensors, wherein the sensor data stream includes multiple frames of sensor data; Adding frame numbers to the sensor data; Obtaining a directed acyclic graph, the directed acyclic graph comprising a plurality of task nodes and execution paths of the task nodes, the task nodes comprising synchronization nodes; Input sensor data with the same frame number to execute the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph until the subtasks of all task nodes in the directed acyclic graph are executed, wherein the subtask of the synchronization node is used to add the frame number to the output data of the subtask corresponding to the parent node of the synchronization node for data synchronization.

2. The task processing method according to claim 1, characterized in that: The directed acyclic graph comprises: Acquire a perception task, wherein the perception task includes subtasks; Determining the execution order of the subtasks; A directed acyclic graph is generated based on the subtasks and the execution order, wherein each task node in the directed acyclic graph corresponds to a subtask, and the connection relationship between the task nodes represents the execution path between the task nodes.

3. The task processing method according to claim 1, characterized in that: The adding frame sequence numbers to the sensing data comprises: When receiving sensor data streams from at least two sensors, obtaining a timestamp of each frame of sensor data in the sensor data streams; The sensing data with the same timestamp are added with the same frame number.

4. The task processing method according to claim 1, characterized in that: The sensor data with the same input frame sequence number is used to execute the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph until the subtasks of all task nodes in the directed acyclic graph are executed, including: Inputting sensor data with the same sequence number as the current frame to sequentially obtain input data and execution code of the task nodes on the execution path according to the execution path; Encapsulating the input data and the execution code as subtasks of the task node; Adding the subtask to the task queue; The calling thread executes the subtask located in the task queue to obtain output data after the subtask is executed, until the subtasks of all task nodes in the directed acyclic graph are executed.

5. The task processing method according to claim 4, characterized in that: The sensor data with the same input frame sequence number is sequentially acquired according to the execution path. The input data and execution code of the task node on the execution path include: Input the sensor data with the same sequence number as the current frame; For each task node in the directed acyclic graph, determining a parent node of the task node; Obtaining output data of the parent node as input data of the task node; Get the execution code of the task node.

6. The task processing method according to claim 5, characterized in that: The task node includes at least two sensor input nodes at the most upstream, and for each task node in the directed acyclic graph, determining a parent node of the task node includes: When the task node is a sensor input node, taking the sensor as a parent node; The obtaining the output data of the parent node as the input data of the task node includes: Calling the main thread to obtain the sensor data output by the sensor as input data of the sensor input node; The step of encapsulating the input data and the execution code as a subtask of the task node includes: The sensor input node input data and the execution code are encapsulated as subtasks through the main thread.

7. The task processing method according to claim 5, characterized in that: The step of determining, for each task node in the directed acyclic graph, a parent node of the task node, comprises: When the task node is a synchronization node, determining a parent node of the synchronization node; The obtaining the output data of the parent node as the input data of the task node includes: Obtaining output data of the subtask corresponding to the parent node after execution as input data of the synchronization node; The step of encapsulating the input data and the execution code as a subtask of the task node includes: The thread pool thread is called to encapsulate the input data of the synchronization node and the execution code of the synchronization node into a subtask of the synchronization node. After the subtask of the synchronization node is executed, the frame sequence number of the output data of the parent node of the synchronization node is the current frame sequence number.

8. The task processing method according to claim 4, characterized in that: The calling thread executes the subtask located in the task queue to obtain output data after the subtask is executed, until the subtasks of all task nodes in the directed acyclic graph are executed, including: Determining from the task queue the subtask that is first added to the task queue as the target subtask; When a preset event is detected, an idle thread in the thread pool is called to execute the target subtask to obtain output data of the target subtask; Determine whether the task node corresponding to the target subtask has a downstream task node; If so, determining whether a timer has timed out, the timer being started when sensor data having the same sequence number as the current frame is input; If the timer has not timed out, obtaining the execution code of the downstream task node; Encapsulating the output data and the execution code into subtasks and adding them to the task queue; If the timer times out, the data generated by the sensor data having the same sequence number as the current frame is destroyed; If not, after all subtasks of all task nodes in the directed acyclic graph are executed, the data generated by the sensor data having the same sequence number as the current frame is destroyed.

9. A task processing device, characterized in that: include: A sensor data acquisition module, used to acquire a sensor data stream output by at least two sensors, wherein the sensor data stream includes multiple frames of sensor data; A frame number adding module, used for adding frame numbers to the sensing data; A directed acyclic graph acquisition module, used to acquire a directed acyclic graph, wherein the directed acyclic graph includes a plurality of task nodes and execution paths of the task nodes, wherein the task nodes include synchronization nodes; A task execution module is used to input sensor data with the same frame number to execute the subtask corresponding to the task node according to the execution path and the sensor data in the directed acyclic graph until the subtasks of all task nodes in the directed acyclic graph are executed, wherein the subtask of the synchronization node is used to add the frame number to the output data of the subtask corresponding to the parent node of the synchronization node for data synchronization.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; a memory for storing one or more computer programs; When the one or more computer programs are executed by the one or more processors, the one or more processors implement the task processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the task processing method according to any one of claims 1 to 8 is implemented.

Images