Driving decision-making method and apparatus, and storage medium

Abstract

The present application provides a driving decision-making method and apparatus, and a storage medium. The method comprises: constructing a plurality of computational graphs, wherein any computational graph among the plurality of computational graphs comprises a plurality of nodes; constructing a hybrid computational graph on the basis of the plurality of computational graphs, wherein the hybrid computational graph comprises topological relationships, and the topological relationships are used for representing the topological relationships between the plurality of computational graphs and between the nodes in each computational graph among the plurality of computational graphs; on the basis of the topological relationships, deploying a plurality of sub-hybrid computational graphs in a plurality of computational units to obtain a deployment result, wherein the plurality of sub-hybrid calculation graphs are the plurality of computational graphs or the plurality of nodes in the plurality of computational graphs, the plurality of computational units are used for executing the plurality of sub-hybrid computational graphs on the basis of the deployment result so as to obtain an execution result, and the execution result is used for driving decision-making of a vehicle. The method provided by the present application helps to establish the topological relationships between the computational graphs, and provides a basis for subsequent deployment of the computational graphs, so as to achieve automatic deployment of the computational graphs in the computational units.

Description

Driving decision method, device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on February 23, 2024, with application number 202410204265.7 and application name “Driving Decision Method, Device and Storage Medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of autonomous driving, and in particular to a driving decision-making method, device, and storage medium. Background Art

[0003] With the continuous advancement of technology, many vehicles now have autonomous driving capabilities. The core of autonomous driving technology lies in environmental perception and decision-making. To achieve autonomous driving, vehicles need to be equipped with various perception units such as cameras, sensors, and radar to obtain information about the surrounding environment. These acquired images and sensor data are then processed through multimodal fusion, enabling the vehicle to understand traffic participants and the road environment, thus achieving autonomous driving.

[0004] At present, the computational processing of autonomous driving of vehicles is usually implemented by a computing unit. For example, the computational graph can be deployed in the computing unit, and the computing unit can integrate dedicated accelerators such as image processing and deep learning operations, and can execute corresponding perception algorithms to achieve autonomous driving.

[0005] It is understandable that the vehicle's autonomous driving decisions are usually determined by the execution results of the computational graph. In other words, the deployment and execution of the computational graph will affect the performance of the vehicle's autonomous driving.

[0006] In the existing technology, computation graph deployment methods mainly include single-computing unit-based deployment and multi-computing unit-based deployment. Single-computing unit-based deployment refers to deploying multiple computation graphs in a single computation unit, while multi-computing unit-based deployment refers to deploying multiple computation graphs in multiple computation units. Compared to single-computing unit-based deployment, multi-computing unit-based deployment can increase parallelism and reduce overall latency through concurrent execution of multiple computation units, thereby improving overall vehicle driving performance.

[0007] However, current deployment based on multiple computing units is usually done manually, which cannot be automated and is inefficient. In addition, existing manual deployment methods are highly customized and have high overhead.

[0008] Summary of the Invention

[0009] The present application provides a driving decision-making method, device and storage medium, which can establish a topological relationship between calculation graphs, provide a basis for the subsequent deployment of calculation graphs, and realize the automatic deployment of calculation graphs within the computing unit.

[0010] In a first aspect, the present application provides a driving decision-making method, which is applied to a vehicle, wherein the vehicle includes multiple computing units, and the method includes: constructing multiple computing graphs, wherein any one of the multiple computing graphs includes multiple nodes; constructing a hybrid computing graph based on the multiple computing graphs, wherein the hybrid computing graph includes a topological relationship, and the topological relationship is used to characterize the topological relationship between the multiple computing graphs and between the nodes in each computing graph in the multiple computing graphs; deploying multiple sub-hybrid computing graphs in multiple computing units based on the topological relationship to obtain deployment results, wherein the multiple sub-hybrid computing graphs are the multiple computing graphs or multiple nodes in the multiple computing graphs, and the multiple computing units are used to execute the multiple sub-hybrid computing graphs based on the deployment results to obtain execution results, and the execution results are used for driving decision-making of the vehicle.

[0011] In the prior art, multiple computation graphs can only be deployed in multiple computing units manually, and automated deployment of multiple computation graphs in multiple computing units is not possible. This application constructs a hybrid computation graph from multiple computation graphs, thereby establishing a topological relationship between the computation graphs, providing a basis for subsequent deployment of computation graphs, and thus enabling computation graphs to be deployed in multiple computing units to achieve automated deployment of computation graphs.

[0012] In one possible implementation, the deployment result includes the deployment position and execution order of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs, and the deploying of the multiple sub-hybrid computation graphs in multiple computing units based on the topological relationship includes: simultaneously determining the deployment position and execution order of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs; deploying the multiple sub-hybrid computation graphs in the multiple computing units based on the deployment position and execution order of each sub-hybrid computation graph, and the execution order of the sub-hybrid computation graphs is obtained based on the topological relationship, wherein the deployment position is the position of each sub-hybrid computation graph in the multiple computing units.

[0013] In the present application, by simultaneously determining the deployment location and execution order of the sub-hybrid computation graph, the sub-hybrid computation graph can be quickly deployed in the computing unit, thereby improving the deployment efficiency.

[0014] In one possible implementation, the deployment result includes the deployment location and execution order of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs, and the deployment of the multiple sub-hybrid computation graphs in multiple computing units based on the topological relationship includes: determining the deployment location of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs, and deploying the multiple sub-hybrid computation graphs to corresponding computing units based on the deployment location, wherein the deployment location is the location of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs in the multiple computing units; in any target computing unit, determining the execution order of each sub-hybrid computation graph in the target computing unit based on the topological relationship, and deploying the execution order of the multiple sub-hybrid computation graphs in the target computing unit based on the execution order, wherein the target computing unit is a computing unit that contains sub-hybrid computation graphs with the same deployment location.

[0015] In this application, by first determining the deployment location of the sub-hybrid computation graph and then determining the execution order of the sub-hybrid computation graph in the computing unit, there is no need to solve the deployment location, only the execution order needs to be solved, which can greatly reduce the solution space.

[0016] In one possible implementation, the deployment result includes the actual deployment location and execution order of each sub-hybrid computing graph in the multiple sub-hybrid computing graphs, the types of the sub-hybrid computing graphs include a first type and a second type, and the deployment of the multiple sub-hybrid computing graphs in multiple computing units based on the topological relationship includes: splitting the multiple sub-hybrid computing graphs into multiple groups, determining the virtual deployment locations of the multiple groups, and virtually deploying the multiple groups to corresponding computing units based on the virtual deployment locations, wherein each of the multiple groups includes one or more sub-hybrid computing graphs in the multiple sub-hybrid computing graphs; and simultaneously determining the actual deployment location and execution order of the sub-hybrid computing graphs of the first type in the multiple groups, the execution order of the sub-hybrid computing graphs of the first type being obtained based on the topological relationship; and The actual deployment position and execution order of a sub-hybrid computing graph of a type are used to deploy the sub-hybrid computing graph of the first type in the multiple computing units; and, the virtual deployment position of the sub-hybrid computing graph of the second type in the multiple groups is determined to be the actual deployment position of the sub-hybrid computing graph of the second type, and the sub-hybrid computing graph of the second type is deployed to the corresponding computing unit based on the actual deployment position of the sub-hybrid computing graph of the second type; in any target computing unit, the execution order of each sub-hybrid computing graph of the second type in the target computing unit is determined based on the topological relationship, and the execution order of each sub-hybrid computing graph of the second type in the target computing unit is deployed based on the execution order, wherein the target computing unit is a computing unit containing the sub-hybrid computing graph of the second type with the same actual deployment position.

[0017] In this application, deployment is performed according to the type of sub-hybrid computation graph, which can take into account both the performance of important sub-hybrid computation graphs and the computing power requirements.

[0018] In one possible implementation, the execution order of the sub-hybrid computation graph is determined by the slack of the sub-hybrid computation graph, and the slack is used to characterize the importance of the sub-hybrid computation graph in the hybrid computation graph based on the topological relationship.

[0019] In one possible implementation, after deploying the multiple sub-hybrid computation graphs in the multiple computing units based on the topological relationship, the method further includes: associating the deployment results with a target scene and storing them in the vehicle, wherein the target scene is the driving scene or road scene in which the vehicle is currently located.

[0020] In this application, by associating the deployment results with the current scene of the vehicle and then storing them, a mapping relationship between the deployment results and the vehicle scene can be effectively established, providing a basis for the vehicle to make decisions on the deployment results during subsequent driving.

[0021] In one possible implementation, before constructing the multiple computation graphs, the method further includes: querying whether a deployment result corresponding to the target scenario is stored in the vehicle; and constructing the multiple computation graphs includes: if no deployment result corresponding to the target scenario is found stored in the vehicle, constructing the multiple computation graphs.

[0022] In this application, before building the calculation graph, the deployment results stored in the vehicle are queried. After the stored deployment results corresponding to the current scenario of the vehicle are queried, the deployment results can be used directly without obtaining the deployment results by building a hybrid calculation graph, thereby improving the efficiency of deployment.

[0023] In one possible implementation, constructing multiple computation graphs includes: constructing multiple computation graphs based on the value of a first parameter and the value of a second parameter, wherein the first parameter is used to characterize the computation delay of each node in any one of the multiple computation graphs, and the second parameter is used to characterize the amount of data transmission between two nodes in any one of the multiple computation graphs.

[0024] In one possible implementation, constructing a hybrid computation graph based on the multiple computation graphs includes: determining the values ​​of a third parameter and a fourth parameter, wherein the third parameter is used to characterize the execution cycle corresponding to any one of the multiple computation graphs, and the fourth parameter is used to characterize the number of paths corresponding to any one of the multiple computation graphs, and the number of paths is used to characterize the number of concurrent executions of the same computation graph; and fusing the multiple computation graphs into a hybrid computation graph based on the values ​​of the third parameter and the fourth parameter.

[0025] In one possible implementation, the values ​​of the third parameter and the fourth parameter are determined by the scene in which the vehicle is currently located.

[0026] In a second aspect, the present application provides a driving decision-making device, comprising one or more functional modules, wherein the one or more functional modules are used to implement the driving decision-making method as described in the first aspect.

[0027] In a third aspect, the present application provides a driving decision-making device, comprising: a processor and a memory, wherein the memory is used to store a program; and the processor is used to run the program to implement the driving decision-making method as described in the first aspect.

[0028] In a fourth aspect, the present application provides a readable storage medium, which stores a program. When the program is run on a driving decision-making device, the driving decision-making device implements the driving decision-making method described in the first aspect.

[0029] In a fifth aspect, the present application provides a program, which, when executed on a processor of a driving decision-making device, enables the driving decision-making device to execute the driving decision-making method as described in the first aspect.

[0030] In one possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged with the processor, or may be stored in whole or in part on a memory not packaged with the processor.

[0031] Among them, the driving decision-making device in the second aspect and the third aspect can be a vehicle or a chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a schematic structural diagram of an embodiment of a driving decision-making device provided by the present application;

[0033] FIG2 is a flow chart of an embodiment of a driving decision-making method provided by the present application;

[0034] FIG3 is a schematic diagram of an embodiment of a calculation graph provided by the present application;

[0035] 4A and 4B are schematic diagrams of a method for obtaining a hybrid computation graph according to an embodiment of the present application;

[0036] FIG5A is a schematic diagram of an embodiment of a hybrid computation graph provided by the present application;

[0037] FIG5B is a schematic diagram of another embodiment of a hybrid computing graph provided by the present application;

[0038] Figures 6 and 7 are schematic diagrams of topological sorting provided in embodiments of the present application;

[0039] FIG8A is a schematic diagram of an embodiment of a deployment result provided by the present application;

[0040] FIG8B is a schematic diagram of another embodiment of a deployment result provided by the present application;

[0041] FIG9A is a schematic diagram of another embodiment of a deployment result provided by the present application;

[0042] FIG9B is a schematic diagram of another embodiment of a deployment result provided by the present application;

[0043] FIG10A is a schematic diagram of an embodiment of a deployment process provided by the present application;

[0044] FIG10B is a schematic diagram of another embodiment of the deployment process provided by the present application;

[0045] FIG11 is a schematic structural diagram of another embodiment of the driving decision-making device provided in this application. DETAILED DESCRIPTION

[0046] In the embodiments of this application, unless otherwise specified, the character " / " indicates that the associated objects are in an "or" relationship. For example, A / B can represent A or B. "And / or" describes the relationship between the associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exists simultaneously, or B exists alone.

[0047] It should be pointed out that the words "first", "second", etc. involved in the embodiments of this application are only used for distinguishing description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated, nor can they be understood as indicating or implying order.

[0048] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. In addition, "at least one of the following" or similar expressions refers to any combination of these items, which may include any combination of single items or plural items. For example, at least one of A, B, or C can represent: A, B, C, A and B, A and C, B and C, or A, B and C. Among them, each of A, B, and C can be an element itself, or a set containing one or more elements.

[0049] In the embodiments of this application, the terms "exemplary," "in some embodiments," and "in another embodiment" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" in this application should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner.

[0050] In the embodiments of this application, the terms "of," "corresponding," and "relevant" may sometimes be used interchangeably. It should be noted that, when the distinction between them is not emphasized, the meanings they convey are the same. In the embodiments of this application, the terms "communication" and "transmission" may sometimes be used interchangeably. It should be noted that, when the distinction between them is not emphasized, the meanings they convey are the same. For example, "transmission" may include "sending" and / or "receiving," and may be either a noun or a verb.

[0051] In the embodiments of this application, "equal to" can be used in conjunction with "greater than" and is applicable to the technical solution adopted when "greater than" is used, and can also be used in conjunction with "less than" and is applicable to the technical solution adopted when "less than" is used. It should be noted that when "equal to" is used in conjunction with "greater than", it cannot be used in conjunction with "less than"; and when "equal to" is used in conjunction with "less than", it cannot be used in conjunction with "greater than".

[0052] The following explains some of the terms involved in the embodiments of the present application to facilitate understanding by those skilled in the art.

[0053] 1. Perception unit.

[0054] The sensing unit in the embodiment of the present application refers to an element or device for sensing the surrounding environment information. For example, the sensing unit may include but is not limited to elements or devices such as a camera, a radar, and a sensor.

[0055] 2. Computing unit.

[0056] The computing unit in the embodiments of the present application refers to a chip for performing computing processing. For example, the computing unit may be a system on chip (SOC) chip, or the computing unit may be other types of chips, which are not specifically limited in the embodiments of the present application.

[0057] 3. Computational graph.

[0058] The computational graph in the embodiments of the present application refers to a neural network computational graph generated based on a perception unit. A computational graph can be generated based on a perception unit. It is understood that a perception unit can generate multiple computational graphs. For example, a perception unit can generate different computational graphs for different scenarios in which a vehicle is located. A computational graph can contain multiple nodes and the dependencies between nodes.

[0059] The above is a brief introduction to some of the concepts involved in this application.

[0060] With the continuous advancement of technology, many vehicles now have autonomous driving capabilities. The core of autonomous driving technology lies in environmental perception and decision-making. To achieve autonomous driving, vehicles need to be equipped with various perception units such as cameras, sensors, and radar to obtain information about the surrounding environment. These acquired images and sensor data are then processed through multimodal fusion, enabling the vehicle to understand traffic participants and the road environment, thus achieving autonomous driving.

[0061] At present, the computational processing of autonomous driving of vehicles is usually implemented by a computing unit. For example, the computational graph can be deployed in the computing unit, and the computing unit can integrate dedicated accelerators such as image processing and deep learning operations, and can execute corresponding perception algorithms to achieve autonomous driving.

[0062] It is understandable that the vehicle's autonomous driving decisions are usually determined by the execution results of the computational graph. In other words, the deployment and execution of the computational graph will affect the performance of the vehicle's autonomous driving.

[0063] In the existing technology, computation graph deployment methods mainly include single-computing unit-based deployment and multi-computing unit-based deployment. Single-computing unit-based deployment refers to deploying multiple computation graphs in a single computation unit, while multi-computing unit-based deployment refers to deploying multiple computation graphs in multiple computation units. Compared to single-computing unit-based deployment, multi-computing unit-based deployment can increase parallelism and reduce overall latency through concurrent execution of multiple computation units, thereby improving overall vehicle driving performance.

[0064] However, current deployment based on multiple computing units is usually done manually, which cannot be automated and is inefficient. In addition, existing manual deployment methods are highly customized and have high overhead.

[0065] To address the above issues, embodiments of the present application propose a driving decision-making method for a vehicle. The vehicle may include multiple perception units and multiple computing units. The perception units may include, but are not limited to, cameras, radars, sensors, and other components or devices, and may be used to generate corresponding computational graphs. The computing units may include, but are not limited to, SOC chips, and may be used to deploy and execute computational graphs.

[0066] Through the driving decision-making method provided in the embodiment of the present application, a topological relationship between calculation graphs can be established, providing a basis for the subsequent automatic deployment of calculation graphs, so as to realize the automatic deployment of calculation graphs within the computing unit.

[0067] In order to achieve better performance of autonomous driving, the driving decision method can also be applied to a simulation platform, so that the simulation platform can verify the performance of the driving decision algorithm. Among them, the simulation platform can construct a simulated vehicle, and the simulated vehicle can be a real vehicle, or the simulated vehicle can also be a virtual vehicle built by a computer and a network system. The embodiments of the present application do not specifically limit this. The simulated vehicle can include multiple simulated perception units and multiple simulated computing units, wherein the simulated perception unit can be a real perception element or perception device, or the simulated perception unit can be a virtual perception element or virtual device; the simulated computing unit can be a real chip, or the simulated perception unit can be a virtual chip. The embodiments of the present application do not specifically limit this.

[0068] Through the above simulation platform, the driving decision-making method can be simulated, thereby obtaining the performance of the driving decision in advance, and the driving decision-making algorithm can be evaluated based on the simulation performance.

[0069] In some optional embodiments, the above-mentioned simulation platform can also be used to verify and improve the results obtained by the driving decision-making method, thereby improving the performance of the driving decision-making algorithm.

[0070] 1 exemplarily shows a schematic structural diagram of a driving decision-making device 100. The aforementioned vehicle, chip, and simulation platform may be the driving decision-making device 100.

[0071] The driving decision-making device 100 may include: at least one processor; and at least one memory in communication with the processor, wherein the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the method provided in the embodiment shown in this document.

[0072] FIG1 shows a block diagram of an exemplary driving decision-making device 100 suitable for implementing the embodiments of this document. The driving decision-making device 100 shown in FIG1 is merely an example and should not limit the functionality and scope of use of the embodiments of this document.

[0073] As shown in FIG1 , components of the driving decision-making device 100 may include but are not limited to: one or more processors 110 , a memory 120 , a communication bus 140 connecting different system components (including the memory 120 and the processor 110 ), and a communication interface 130 .

[0074] Communication bus 140 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of such architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnection (PCI) bus.

[0075] The driving decision-making device 100 typically includes a variety of computer system readable media, which can be any available media that can be accessed by the device, including volatile and non-volatile media, removable and non-removable media.

[0076] The memory 120 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Although not shown in FIG1 , a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a Compact Disc Read Only Memory (CD-ROM), a Digital Video Disc Read Only Memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to the communication bus 140 via one or more data medium interfaces. The memory 120 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments herein.

[0077] A program / utility having a set (at least one) of program modules may be stored in memory 120. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. The program modules generally perform the functions and / or methods of the embodiments described herein.

[0078] The driving decision device 100 can also communicate with one or more external devices (e.g., a keyboard, pointing device, display, etc.), one or more devices that enable user interaction with the device, and / or any device that enables the device to communicate with one or more other devices (e.g., a network card, modem, etc.). This communication can occur via a communication interface 130. Furthermore, the driving decision device 100 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter (not shown in FIG. 1 ). This network adapter can communicate with other modules of the device via a communication bus 140. It should be understood that, although not shown in FIG. 1 , other hardware and / or software modules can be used in conjunction with the driving decision device 100, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0079] The processor 110 executes various functional applications and data processing by running the programs stored in the memory 120, such as implementing the methods provided in the embodiments of this document.

[0080] It should be understood that the interface connection relationships between the modules illustrated in the embodiments herein are for illustrative purposes only and do not constitute a structural limitation on the driving decision device 100. In other embodiments herein, the driving decision device 100 may also employ different interface connection methods from those in the above embodiments, or a combination of multiple interface connection methods.

[0081] Next, the driving decision-making method provided in the embodiment of the present application is described with reference to FIG. 2 to FIG. 10 .

[0082] FIG2 is a flow chart of an embodiment of the driving decision-making method provided by the present application, which specifically includes the following steps:

[0083] Step 201: construct multiple computation graphs, where any one of the multiple computation graphs includes multiple nodes.

[0084] Specifically, during the driving process of the vehicle, the current scene can be identified in real time, and the scene may include a driving scene or a road scene.

[0085] For example, driving scenes may include scenes such as highways and parking, and road scenes may include scenes such as cities and mountainous areas. In some embodiments, other types of driving scenes or road scenes may also be included, and the embodiments of the present application do not specifically limit this.

[0086] The scene recognition may be obtained by identifying an image captured by a camera on the vehicle, or the scene recognition may be obtained by other means, which is not specifically limited in the embodiments of the present application.

[0087] The vehicle can include multiple perception units. After identifying the scene the vehicle is currently in, it can determine the target perception unit required for the vehicle's current scene and build a corresponding computational graph for the target perception unit.

[0088] Among them, one perception unit can generate a corresponding computational graph.

[0089] It is understandable that in different scenarios, a perception unit can generate different computational graphs.

[0090] For example, assume that a vehicle includes multiple perception units, namely: perception unit 1, perception unit 2, ..., perception unit N, where N is a positive integer greater than 1. Next, assume that the scene currently in which the vehicle is located is identified as scene 1. Then, the required target perception unit can be determined from the multiple perception units mentioned above based on scene 1. Assuming that the target perception unit includes target perception unit 1 and target perception unit 2, target perception unit 1 can generate a corresponding computation graph A based on scene 1, and target perception unit 2 can generate a corresponding computation graph B based on scene 1.

[0091] For another example, assume that a vehicle includes multiple perception units, namely: perception unit 1, perception unit 2, ..., perception unit N. Next, assume that the scene currently in which the vehicle is located is identified as scene 2. Then, the required target perception unit can be determined from the above multiple perception units based on scene 2. Assuming that the target perception units include target perception unit 1, target perception unit 2, and target perception unit 3, target perception unit 1 can generate a corresponding computation graph C based on scene 2, target perception unit 2 can generate a corresponding computation graph D based on scene 2, and target perception unit 3 can generate a corresponding computation graph E based on scene 2.

[0092] Now, taking a computation graph as an example, and combining it with FIG3 , the construction of the computation graph is exemplarily explained.

[0093] Referring to Figure 3, the computation graph includes multiple nodes. For example, the multiple nodes are Node 1, Node 2, Node 3, Node 4, and Node 5, respectively, indicated by numbers 1 to 5. It is understood that a node may also be referred to as an operator or a node operator. In some embodiments, a node may also be referred to by other names. A node is the smallest unit used to implement computational processing in the embodiments of the present application. Any object that implements the same function as a node is within the scope of protection of this application.

[0094] Next, referring to Figure 3, any node in the computation graph can have a corresponding node weight, where the node weight can be used to characterize the computational latency of the node, which can be obtained through profiling. It is understood that computational latency is an important indicator for measuring computing power.

[0095] For any two nodes in a computation graph, the edge formed by the two nodes may have a corresponding edge weight, wherein the edge weight may be used to represent the amount of data transmitted between the nodes. It is understood that the amount of data transmitted is an important indicator for measuring data transmission performance.

[0096] For example, the data transmission volume between two nodes can be represented by the following formula: link (i→j)=shape×ByteSize;

[0097] Among them, W link (i→j) is the amount of data transmitted from the i-th node to the j-th node, shape is the tensor being transmitted, ByteSize is the number of bytes corresponding to the data type of each element in the tensor, and the amount of data transmitted can be obtained by multiplying the number of bytes by the tensor.

[0098] In some optional embodiments, for any two nodes in the computation graph, if the two nodes are deployed in the same computation unit, the transmission delay between the two nodes may not be considered. Alternatively, if the two nodes are deployed in different computation units, the transmission delay between the two nodes may be considered.

[0099] The transmission delay can be represented by the following formula:

[0100] Where delay(i→j) is the transmission delay from the i-th node to the j-th node, and bandwidth(i→j) is the bandwidth from the i-th node to the j-th node.

[0101] Step 202: construct a hybrid computation graph based on the multiple computation graphs. The hybrid computation graph includes a topological relationship, which is used to represent the topological relationship between the multiple computation graphs and between nodes in each of the multiple computation graphs.

[0102] Specifically, after constructing multiple computation graphs, a hybrid computation graph may be constructed based on the multiple computation graphs. For example, the multiple computation graphs may be fused to construct a hybrid computation graph.

[0103] The hybrid computation graph may include dependencies between nodes within multiple computation graphs and dependencies between computation graphs, which may also be referred to as topological relationships.

[0104] It is understandable that by fusing multiple computation graphs, the topological relationships between computation graphs and between nodes can be obtained, thereby providing a basis for the automatic deployment of computation graphs or nodes in computing units.

[0105] In some optional embodiments, multiple computation graphs can be fused into a hybrid computation graph by frequency and number of paths. Frequency refers to the execution frequency of the computation graph. It is understood that in some embodiments, this frequency can also be converted into the execution cycle of the computation graph, which is not specifically limited in this embodiment of the present application.

[0106] For example, taking the calculation frequency of FIG1 as 10 Hz as an example, it is explained that the calculation of FIG1 can be executed once within 0.1 s.

[0107] The number of paths refers to how many paths can be executed in parallel for the same computation graph, that is, the number of paths can refer to the number of concurrent executions of the same computation graph.

[0108] Exemplarily, taking the case where the number of paths in the calculation graph 1 is 4 as an example, it is explained that the 4-path calculation graph 1 can be executed in parallel.

[0109] The frequency and number of paths can be determined based on the scene the vehicle is currently in. For example, after determining the target sensing unit required for the vehicle's current scene, the frequency and number of paths corresponding to the target sensing unit can be obtained. Since the computation graph corresponds one-to-one to the sensing unit, obtaining the frequency and number of paths corresponding to the target sensing unit means obtaining the frequency and number of paths corresponding to the computation graph.

[0110] Next, the fusion of computation graphs is exemplarily illustrated with reference to FIG4A and FIG4B .

[0111] Referring to FIG4A , it is assumed that calculation graph A and calculation graph B have been constructed, wherein the frequency of calculation graph A is 10 Hz and the number of paths of calculation graph A is 4; the frequency of calculation graph B is 20 Hz and the number of paths of calculation graph B is 1.

[0112] By calculating the frequency and number of paths of graph A, we can see that graph A executes four paths in parallel within 0.1s, that is, it is executed four times within 0.1s. By calculating the frequency and number of paths of graph B, we can see that graph B executes one path in parallel within 0.05s, that is, it is executed once within 0.05s.

[0113] Next, computation graph A and computation graph B are fused. By combining the periods of computation graph A and computation graph B, a unified period can be determined between the frequencies of computation graph A and computation graph B.

[0114] In some optional embodiments, the unified period can be a common multiple of the periods of all calculation graphs. For example, the period of calculation graph A is 0.1s, and the period of calculation graph B is 0.05s. Since 0.1, 0.2, 0.3... etc. are common multiples of 0.05 and 0.1, one period among 0.1s, 0.2s, 0.3s... can be selected as the unified period.

[0115] In some optional embodiments, the unified period can be the least common multiple of the periods of all calculation graphs. For example, the period of calculation graph A is 0.1s, and the period of calculation graph B is 0.05s. Since 0.1 is the least common multiple of 0.05 and 0.1, 0.1s can be selected as the unified period.

[0116] After determining a unified period, we can determine the execution of computation graphs A and B in the hybrid computation graph. For example, since computation graph A's period remains unchanged, computation graph A can still execute four parallel paths within 0.1s. However, since computation graph B's period changes from 0.05s to 0.1s, computation graph B can be executed in parallel, instead of one path within 0.05s, to two paths within 0.1s. This results in the hybrid computation graph shown in Figure 4B.

[0117] Referring to Figure 4B, assuming that the constraint condition is that calculation graph A is executed earlier than calculation graph B, therefore, in the hybrid calculation graph consisting of calculation graph A and perception calculation B, calculation graph A is executed first, and calculation graph B is executed after calculation graph A is executed.

[0118] It is understandable that since computation graph A can execute 4 paths in parallel, the output of any of the 4 paths in computation graph A can be used as the input of computation graph B. Since computation graph B can execute 2 paths in parallel, the output of computation graph A can be used as the input of any of the 2 paths in computation graph B.

[0119] In some embodiments, in order to control the output of computation graph A and the input of computation graph B, the hybrid computation graph may further include output nodes, input nodes, and control nodes. An output node is used to receive and store the output results of one computation graph A and can send the output results of that computation graph A to the control node. The control node is used to receive the output results of any computation graph A and can determine to which input node the output results of any computation graph A are sent. The input node is used to receive the output results of any computation graph A sent by the control node and can send the output results of any computation graph A to the corresponding computation graph B.

[0120] For example, taking two computation graphs (e.g., computation graph A and computation graph B) as an example, the number of newly added nodes and the number of newly added edges can be obtained by the following formulas: new =rd out +rd in +1;

[0121] E new =(N out +1)×r dout +(N in +1)×rd in ;

[0122] Among them, N new is the number of newly added nodes, rd out is the number of paths of the computational graph executed first, rd in is the number of paths in the computational graph to be executed later, E new is the number of newly added edges, N out is the number of nodes used for output in the first executed computation graph, N in The number of nodes used for input in the computation graph to be executed later.

[0123] It is understandable that the calculation of the number of newly added nodes and the number of newly added edges of multiple calculation graphs can refer to the calculation method of the above two calculation graphs (for example, calculation graph A and calculation graph B), which will not be repeated here.

[0124] Step 203: topologically sort the sub-hybrid computation graphs in the constructed hybrid computation graph.

[0125] It is understandable that, whether for each computation graph in a hybrid computation graph or for each node in a computation graph, the topological relationship in the hybrid computation graph only represents the dependency between computation graphs or the dependency between nodes, but this dependency does not include the execution order.

[0126] For example, the computation graphs in a hybrid computation graph only contain dependency relationships, but not the execution order between the computation graphs; or, the nodes in a computation graph only contain dependency relationships, but not the execution order between the nodes.

[0127] Therefore, in order to achieve automatic deployment of calculation graphs or nodes, the calculation graphs in the hybrid calculation graph constructed this time can be topologically sorted to determine the execution order of the calculation graphs in the hybrid calculation graph, or the nodes in the hybrid calculation graph constructed this time can be topologically sorted to determine the execution order of the nodes in the hybrid calculation graph.

[0128] In some optional embodiments, the execution order of a computation graph or node may be determined by slack, wherein the value of the slack may be represented by the difference between a first value and a second value, the first value may be calculated based on the principle of as late as possible execution (ALAP), and the second value may be calculated based on the principle of as early as possible execution (ASAP).

[0129] For example, the value of the slack can be represented by the following formula:

[0130] slack(node)=ALAP(node)-ASAP(node);

[0131] It is understood that the node in the above formula can be referred to as a sub-hybrid computation graph. This sub-hybrid computation graph can be a computation graph within a hybrid computation graph, or it can be a node within a computation graph. This embodiment of the present application does not specifically limit this. For ease of explanation, the sub-hybrid computation graph is hereinafter referred to as a node.

[0132] Here, slack(node) is the slack of a node. Slack can be used to represent the importance of a node in a hybrid computation graph based on topological relationships. For example, a smaller slack indicates a more important node, while a larger slack indicates a less important node. ALAP(node) indicates the number of layers a node corresponds to in the ALAP graph, which is a hybrid computation graph sorted according to the ALAP principle. ASAP(node) indicates the number of layers a node corresponds to in the ASAP graph, which is a hybrid computation graph sorted according to the ASAP principle.

[0133] Next, the sub-mixing calculation graph is exemplarily described with reference to FIG5A and FIG5B .

[0134] Taking a sub-hybrid computation graph as an example, Figure 5A illustrates an exemplary deployment method for a sub-hybrid computation graph. Referring to Figure 5A , the topology of a hybrid computation graph 500 includes a two-way computation graph X and a one-way computation graph Y. In the hybrid computation graph shown in Figure 5A , since the sub-hybrid computation graph is a computation graph, i.e., the computation graph is deployed within a computing unit using the computation graph as the dimension, any computation graph 510 can be deployed as a sub-hybrid computation graph.

[0135] Taking the sub-hybrid computation graph as a node in the computation graph as an example, FIG5B exemplifies the deployment method of the sub-hybrid computation graph. Referring to FIG5B , the topological relationship of the hybrid computation graph 500 includes a 2-way computation graph X and a 1-way computation graph Y, wherein the computation graph X and the computation graph Y each contain multiple nodes. In the hybrid computation graph shown in FIG5B , since the sub-hybrid computation graph is a node in the computation graph, that is, the nodes in the computation graph are deployed in the computing unit with the nodes in the computation graph as the dimension, any node 520 in the computation graph X and the computation graph Y can be deployed as a sub-hybrid computation graph.

[0136] Next, the method of obtaining the slack is exemplarily described with reference to FIG6 .

[0137] Referring to Figure 6, the topology of the hybrid computing graph includes node1-node9. When these nodes1-node9 are sorted based on the ALAP principle, an ALAP layered graph can be obtained. In the ALAP layered graph, node1 is on layer 1, node2 is on layer 2, node3 and node4 are on layer 3, node6 and node7 are on layer 4, and node5, node8, and node9 are on layer 5.

[0138] Alternatively, when node1-node9 are sorted based on the ASAP principle, an ASAP hierarchical diagram can be obtained. In the ASAP hierarchical diagram, node1 is in the first layer numbered 1, node2 and node3 are in the second layer numbered 2, node4, node5 and node6 are in the third layer numbered 3, node7 and node8 are in the fourth layer numbered 4, and node9 is in the fifth layer numbered 5.

[0139] It can be understood that the correspondence between the number and the number of layers is only an example and does not constitute a limitation on the embodiments of the present application. In some embodiments, the correspondence between the number and the number of layers can also be represented in other forms, for example, number 0 corresponds to the first layer, number 1 corresponds to the second layer, and so on.

[0140] Then, the slack of each node can be calculated using the slack calculation formula.

[0141] For example, taking node 3 as an example, since node 3 corresponds to the third layer in the ALAP layered graph, the ALAP value ALAP(3) of node 3 is 3, and since node 3 corresponds to the second layer in the ASAP layered graph, the ASAP value ASAP(3) of node 3 is 2. Therefore, the slack of node 3, slack(3) = ALAP(3) - ASAP(3) = 1, that is, the slack of node 3 is 1.

[0142] For another example, taking node 5 as an example, since node 5 corresponds to the 5th layer in the ALAP layered graph, the ALAP value ALAP(5) of node 5 is 5. And since node 5 corresponds to the 3rd layer in the ASAP layered graph, the ASAP value ASAP(5) of node 5 is 3. Therefore, the slack of node 5 is slack(5) = ALAP(5) - ASAP(5) = 2, that is, the slack of node 5 is 2.

[0143] For another example, taking node 7 as an example, since node 7 corresponds to the 4th layer in the ALAP layered graph, the ALAP value ALAP(7) of node 7 is 4. And since node 7 corresponds to the 4th layer in the ASAP layered graph, the ASAP value ASAP(7) of node 7 is 4. Therefore, the slack of node 7 slack(7) = ALAP(7) - ASAP(7) = 0, that is, the slack of node 7 is 0.

[0144] Then, the nodes in the hybrid computation graph may be sorted based on the ALAP layered graph and in combination with the slack value; or, the nodes in the hybrid computation graph may be sorted based on the ASAP layered graph and in combination with the slack value.

[0145] For example, by performing sorting based on the ALAP layered graph, the node sorting graph shown in FIG7 can be obtained. The sorting principle can be: in the ALAP layered graph, nodes are arranged from lower to higher numbered layers. Within the same layer, nodes with smaller slack can be placed first, and nodes with larger slack can be placed last.

[0146] Referring to FIG. 7, in the first layer, there is only node1. Therefore, the order of node1 is the first, that is, node1 can be the first to execute. In the second layer, there is only node2. Therefore, the order of node2 is the second, that is, node2 can be executed after node1. In the third layer, there are node3 and node4. By calculating the slack, the slack of node3, slack(3), is 1, and the slack of node4, slack(4), is 0. Since slack(4) < slack(3), the order of node4 is the third, and the order of node3 is the fourth, that is, node4 can be executed after node2, and node3 can be executed after node4. In the fourth layer, there are node6 and node7. By calculating the slack, the slack of node6, slack(6), is 1, and the slack of node7, slack(7), is 0. Since slack(7) < slack(6), the order of node7 is the fifth, and the order of node6 is the sixth, that is, node7 can be executed after node3, and node6 can be executed after node7. In the fifth layer, there are node5, node8, and node9. By calculating the slack, the slack of node5, slack(5), is 2, the slack of node8, slack(8), is 1, and the slack of node9, slack(9), is 0. Since slack(9) < slack(8) < slack(5), the order of node9 is the seventh, the order of node8 is the eighth, and the order of node5 is the ninth, that is, node9 can be executed after node6, node8 can be executed after node9, and node5 can be executed after node8.

[0147] It can be understood that the method of sorting based on the ASAP layered graph can specifically refer to the method of sorting based on the ALAP layered graph in the above embodiments, which will not be elaborated here.

[0148] Step 204: Deploy the sub-mixed computation graph in the mixed computation graph.

[0149] Specifically, a vehicle includes multiple computing units. After determining the deployment location and execution order of the sub-mixed computation graph in the mixed computation graph, the sub-mixed computation graph can be deployed in multiple computing units based on the deployment location and execution order, realizing the automatic deployment of the sub-mixed computation graph in the computing units. As a result, the sub-mixed computation graph can be executed orderly in the computing units, enabling the vehicle to make driving decisions based on the deployment results to achieve autonomous driving.

[0150] Taking a sub-hybrid computation graph as an example, after building a hybrid computation graph, multiple computation graphs in the hybrid computation graph can be deployed, that is, multiple computation graphs can be deployed based on the dimensions of the computation graph.

[0151] The deployment of the computation graph may include determining a deployment location and an execution order of the computation graph, and deploying the computation graph in a computing unit based on the deployment location and the execution order of the computation graph.

[0152] For example, assuming that the hybrid computing graph includes computing graph A, computing graph B, and computing graph C, and the computing units may include computing unit 1 and computing unit 2, then computing graph A and computing graph B can be deployed on computing unit 1 and computing graph C can be deployed on computing unit 2 through automatic deployment; or, computing graph A and computing graph B can be deployed on computing unit 2 and computing graph C can be deployed on computing unit 1 through automatic deployment; or, computing graph A, computing graph B, and computing graph C can be deployed on computing unit 1 through automatic deployment; or, computing graph A, computing graph B, and computing graph C can be deployed on computing unit 2 through automatic deployment. It will be understood that the above combinations of deployment locations are merely exemplary and do not constitute a limitation on the embodiments of the present application. In some embodiments, other combinations of deployments are also possible.

[0153] The specific deployment method is described in detail below and will not be repeated here.

[0154] In some optional embodiments, as described above, each computation graph may include multiple nodes. Therefore, multiple nodes in each computation graph in the hybrid computation graph may be deployed, that is, multiple nodes in multiple computation graphs may be deployed in the dimension of nodes.

[0155] The deployment of the nodes may include determining the deployment positions and execution order of the nodes, and deploying the nodes in the computing units based on the deployment positions and execution order of the nodes.

[0156] For example, assuming that the hybrid computing graph includes computing graph A and computing graph B, the computing units may include computing unit 1 and computing unit 2, wherein computing graph A includes node 1 and node 2, and the computing graph includes node 3 and node 4, then through automatic deployment, node 1 and node 3 can be deployed in computing unit 1, and node 2 and node 4 can be deployed in computing unit 2; or, through automatic deployment, node 1 and node 3 can be deployed in computing unit 2, and node 2 and node 4 can be deployed in computing unit 1; or, through automatic deployment, node 1, node 2, and node 3 can be deployed in computing unit 1, and node 4 can be deployed in computing unit 2; or, through automatic deployment, node 1 can be deployed in computing unit 1, and node 2, node 3, and node 4 can be deployed in computing unit 2. It can be understood that the above combination of deployment locations is only an exemplary description and is not sufficient to limit the embodiments of the present application. In some embodiments, other combinations of deployments are also possible.

[0157] Next, the specific deployment method is exemplified below.

[0158] The deployment methods include the following three:

[0159] Method 1: Determine the deployment location and execution order of the sub-hybrid computation graphs based on the topological relationship, and deploy them based on the deployment location and execution order of the sub-hybrid computation graphs.

[0160] Simultaneously determining the deployment location and execution order of sub-hybrid computation graphs based on topological relationships means that the computational units required to be deployed for each sub-hybrid computation graph in the hybrid computation graph and the execution order of each sub-hybrid computation graph within the computational unit can be determined simultaneously based on the topological relationships. The method of simultaneously determining the deployment location and execution order of the sub-hybrid computation graphs does not require calculating the deployment location and execution order separately, thereby speeding up deployment, improving deployment efficiency, and thus improving the performance of autonomous driving.

[0161] Exemplarily, the method of simultaneously determining the deployment position and execution order of the sub-hybrid computation graph based on the topological relationship may be: determining the deployment position and execution order of each sub-hybrid computation graph based on the greedy algorithm in the topological relationship. It can be understood that the idea of ​​determining the deployment position and execution order of each sub-hybrid computation graph by the greedy algorithm may be: for any sub-hybrid computation graph in the hybrid computation graph, the optimal solution of the sub-hybrid computation graph may be solved, that is, on which computing unit the sub-hybrid computation graph is optimally deployed, and the optimal solution may include the optimal deployment position and the optimal execution order of the sub-hybrid computation graph. The optimal solution may be a local optimal solution, not necessarily a global optimal solution. After all sub-hybrid computation graphs in the hybrid computation graph are solved based on the greedy idea, the deployment position and execution order of all sub-hybrid computation graphs in the hybrid computation graph may be obtained.

[0162] It is understandable that the method of simultaneously determining the deployment location and execution order of the sub-hybrid computation graph is more suitable for large-bandwidth scenarios, and can significantly improve performance in large-bandwidth scenarios.

[0163] 8A and 8B are schematic diagrams showing the effects of the deployment results provided in an embodiment of the present application.

[0164] Taking the sub-hybrid computation graph as a node in the computation graph as an example, FIG8A exemplarily shows a schematic diagram of the effect of the deployment result when the sub-hybrid computation graph is used as a node in the computation graph. Referring to FIG8A , assuming that the hybrid computation graph contains 27 nodes, namely, node 1 to node 27, the deployment positions and execution order of these 27 nodes can be determined based on the greedy algorithm, and node 1, node 3, node 9, node 10, node 11, node 16, node 19, node 22, and node 25 can be deployed in computing unit 1 according to the deployment positions and execution order of these 27 nodes, respectively. Among them, node 1, node 3, node 9, node 10, node 11, node 16, node 19, node 22, and node 25 are arranged in the computing unit 1 in the order of execution, and node 2, node 7, node 8, node 12, node 13, node 18, node 25 are arranged in the order of execution. Point 17, node 24, and node 26 are deployed in computing unit 2, wherein node 2, node 7, node 8, node 12, node 13, node 18, node 17, node 24, and node 26 are arranged in sequence in the order of execution in computing unit 2, and node 4, node 6, node 5, node 14, node 15, node 20, node 21, node 23, and node 27 are deployed in computing unit 3, wherein node 4, node 6, node 5, node 14, node 15, node 20, node 21, node 23, and node 27 are arranged in sequence in the order of execution in computing unit 3, so that the effect diagram of the deployment result can be obtained as shown in Figure 8A.

[0165] Assume that node 1, node 12, node 16, node 20, node 21, and node 22 belong to the same computation graph, for example, the computation graph is computation graph A, and node 2, node 3, node 5, node 18, and node 27 belong to the same computation graph, for example, the computation graph is computation graph B, and node 4, node 8, node 9, node 13, node 14, node 15, node 19, node 25, and node 26 belong to the same computation graph, for example, the computation graph is computation graph C, and node 6, node 7, and node 10 belong to the same computation graph, for example, the computation graph is computation graph D, and node 11, node 17, node 23, and node 24 belong to the same computation graph, for example, the computation graph is computation graph E.

[0166] In this case, by simultaneously determining the deployment location and execution order of the sub-hybrid computational graph, automatic deployment can be achieved quickly with a high degree of parallelism, thereby improving deployment efficiency and the performance of autonomous driving. However, it has high requirements on computing power and a large amount of calculation. Therefore, this automatic deployment method is suitable for application scenarios with large bandwidth.

[0167] Next, taking the sub-hybrid computation graph as an example, FIG8B exemplarily shows a schematic diagram of the effect of the deployment result when the sub-hybrid computation graph is used as a computation graph. Referring to FIG8B , assuming that the hybrid computation graph contains 27 computation graphs, namely, computation graph 1 to computation graph 27, the deployment positions and execution order of the 27 computation graphs can be determined based on the greedy algorithm, and computation graph 1, computation graph 3, computation graph 9, computation graph 10, computation graph 11, computation graph 16, computation graph 19, computation graph 22, and computation graph 25 can be deployed in computing unit 1 according to the deployment positions and execution order of the 27 computation graphs, respectively. Among them, computation graph 1, computation graph 3, computation graph 9, computation graph 10, computation graph 11, computation graph 16, computation graph 19, computation graph 22, and computation graph 25 are arranged in the order of execution in computing unit 1, and computation graph 2, computation graph 7, computation graph 8, computation graph 12, computation graph 13, computation graph 18, and computation graph 25 are arranged in the order of execution in computing unit 1. Figure 17, calculation graph 24, and calculation graph 26 are deployed in computing unit 2, wherein calculation graph 2, calculation graph 7, calculation graph 8, calculation graph 12, calculation graph 13, calculation graph 18, calculation graph 17, calculation graph 24, and calculation graph 26 are arranged in sequence in the order of execution in computing unit 2, and calculation graph 4, calculation graph 6, calculation graph 5, calculation graph 14, calculation graph 15, calculation graph 20, calculation graph 21, calculation graph 23, and calculation graph 27 are deployed in computing unit 3, wherein calculation graph 4, calculation graph 6, calculation graph 5, calculation graph 14, calculation graph 15, calculation graph 20, calculation graph 21, calculation graph 23, and calculation graph 27 are arranged in sequence in the order of execution in computing unit 3, thereby obtaining a schematic diagram of the deployment result as shown in Figure 8B.

[0168] It can be understood that, although not shown in Figure 8B, each of the 27 calculation graphs, calculation graph 1 to calculation graph 27, can contain one or more nodes, and the embodiment of the present application does not specifically limit the number of nodes contained in the calculation graph and the topological relationship between the nodes.

[0169] Method 2: First determine the deployment location, deploy each sub-hybrid computation graph in the computing unit based on the deployment location, and then determine the execution order of each sub-hybrid computation graph in each computing unit based on the topological relationship.

[0170] The method of first determining the deployment location and then determining the execution order may include: first determining the deployment location of each sub-hybrid computation graph in the hybrid computation graph. Then, deploying the sub-hybrid computation graph in the corresponding computing unit based on the deployment location. Then, in each computing unit, determining the execution order of each deployed sub-hybrid computation graph based on the topological relationship.

[0171] Among them, the deployment position of each sub-hybrid computing graph can be obtained by dividing the hybrid computing graph and then allocating it. The division of the hybrid computing graph can be based on the principle of balanced computing power. For example, all sub-hybrid computing graphs in the hybrid computing graph can be divided into multiple groups, each of the multiple groups contains one or more sub-hybrid computing graphs, and the computing power requirements of each group in the multiple groups are equivalent. The number of groups can be determined based on the number of idle computing units. Then, each group in the multiple groups can be allocated to a computing unit, so that the deployment position of each sub-hybrid computing graph in the hybrid computing graph can be determined, that is, one group can correspond to one computing unit. The way of allocating groups to computing units can be random allocation, or, they can be allocated according to the numbering order of the computing units, or, they can be allocated in other ways. The embodiments of the present application do not specifically limit this.

[0172] In some optional embodiments, when a sub-hybrid computational graph is used as a computational graph, multiple computational graphs corresponding to the same perception unit can be divided into the same group as much as possible, so that multiple computational graphs corresponding to the same perception unit can be deployed in the same computational unit.

[0173] In some optional embodiments, when a sub-hybrid computation graph is used as a node in a computation graph, multiple nodes belonging to the same computation graph can be divided into the same group as much as possible, so that multiple nodes belonging to the same computation graph can be deployed in the same computing unit.

[0174] It is understandable that the method of first determining the deployment location and then determining the execution order is more suitable for small bandwidth scenarios. This method does not need to solve the deployment location, but only needs to solve the execution order, which can greatly reduce the solution space. And for the scenario where the sub-hybrid calculation graph is used as a node in the calculation graph, since multiple nodes belonging to the same calculation graph are deployed in the same computing unit, the deployment of nodes in a calculation graph can be avoided from being too dispersed, which can reduce the switching cost during execution; and for the scenario where the sub-hybrid calculation graph is used as a calculation graph, since multiple calculation graphs corresponding to the same perception unit are deployed in the same computing unit, the deployment of multiple calculation graphs corresponding to a perception unit can be avoided from being too dispersed, which can reduce the switching cost during execution.

[0175] Taking the sub-hybrid computation graph as an example of a node in the computation graph, by first determining the deployment location and then determining the execution order, a schematic diagram of the deployment result as shown in FIG9A can be obtained.

[0176] Referring to Figure 9A, the hybrid computing graph may include 27 nodes from node 1 to node 27, wherein node 1, node 12, node 16, node 20, node 21, and node 22 belong to the same computing graph, for example, the computing graph may be computing graph A, and node 2, node 3, node 5, node 18, and node 27 belong to the same computing graph, for example, the computing graph is computing graph B, and node 4, node 8, node 9, node 13, node 14, node 15, node 19, node 25, and node 26 belong to the same computing graph, for example, the computing graph is computing graph C, and node 6, node 7, and node 10 belong to the same computing graph, for example, the computing graph is computing graph D, and node 11, node 17, node 23, and node 24 belong to the same computing graph, for example, the computing graph is computing graph E.

[0177] Next, based on the principle of balanced computing power, the 27 nodes 1 through 27 can be divided into three groups. For example, these three groups can include Group 1, Group 2, and Group 3. Group 1 includes Node 1, Node 12, Node 16, Node 20, Node 21, Node 22, Node 6, Node 7, and Node 10; Group 2 includes Node 2, Node 3, Node 5, Node 18, Node 27, Node 11, Node 17, Node 23, and Node 24; and Group 3 includes Node 4, Node 8, Node 9, Node 13, Node 14, Node 15, Node 19, Node 25, and Node 26. Clearly, by partitioning the hybrid computation graph, the computing power requirements of these three groups are comparable.

[0178] It can be understood that, since nodes 1, 12, 16, 20, 21, and 22 belong to the same computational graph A, nodes 1, 12, 16, 20, 21, and 22 can be grouped in the same group (e.g., group 1), which can avoid excessive dispersion of node deployment within the same computational graph, thereby reducing the switching cost during execution; similarly, since nodes 2, 3, 5, 18, and 27 belong to the same computational graph B, nodes 2, 3, 5, 18, and 27 can be grouped in the same group (e.g., group 2), since nodes 4, 8, 9, 13, 14, 15, and 1 9, node 25, and node 26 belong to the same computation graph C. Therefore, nodes 4, node 8, node 9, node 13, node 14, node 15, node 19, node 25, and node 26 are grouped in the same group (for example, group 3). Since nodes 6, node 7, and node 10 belong to the same computation graph D, nodes 6, node 7, and node 10 are grouped in the same group (for example, group 1). Since nodes 11, node 17, node 23, and node 24 belong to the same computation graph E, nodes 11, node 17, node 23, and node 24 are grouped in the same group (for example, group 2). This avoids excessive dispersion of node deployment within the same computation graph, thereby reducing the switching cost during execution.

[0179] Then, these three groups can be assigned to three computing units respectively, where the specific principles of allocation can be described in the relevant description in the above embodiments and will not be repeated here. For example, group 1 can be assigned to computing unit 1, that is, the deployment location of group 1 is computing unit 1; group 2 can be assigned to computing unit 2, that is, the deployment location of group 2 is computing unit 2; group 3 can be assigned to computing unit 3, that is, the deployment location of group 3 is computing unit 3, so that each group can be deployed in the corresponding computing unit according to the determined deployment location.

[0180] After each node from Node 1 to Node 27 is deployed in each computing unit based on the deployment position, the execution order between each node in each computing unit can be determined based on the topological relationship. For example, the execution order of the nodes in each computing unit can be determined based on the slack of the nodes. The specific method can refer to the relevant description in the above embodiment and will not be repeated here. The final deployment effect diagram can refer to Figure 9A. Among them, in computing unit 1, the execution order of each node is from left to right, starting from node 1 and ending at node 10; in computing unit 2, the execution order of each node is from left to right, starting from node 2 and ending at node 24; in computing unit 3, the execution order of each node is from left to right, starting from node 4 and ending at node 26.

[0181] It will be understood that the execution order between the above nodes is merely an illustrative description and does not constitute a limitation to the embodiments of the present application.

[0182] Next, the sub-hybrid computation graph is used as an example for illustrative explanation. By first determining the deployment location and then determining the execution order, a schematic diagram of the deployment result as shown in FIG9B can be obtained.

[0183] 9B , the hybrid computation graph may include 27 computation graphs, computation graph 1 to computation graph 27, wherein computation graph 1, computation graph 12, computation graph 16, computation graph 20, computation graph 21, and computation graph 22 correspond to the same perception unit, for example, the perception unit may be perception unit A, and computation graph 2, computation graph 3, computation graph 5, computation graph 18, and computation graph 27 correspond to the same perception unit, for example, the perception unit may be perception unit B, and computation graph 4, computation graph 8, computation graph 9, computation graph 13, computation graph 14, computation graph 15, computation graph 19, computation graph 25, and computation graph 26 correspond to the same perception unit, for example, the perception unit may be perception unit C, and computation graph 6, computation graph 7, and computation graph 10 correspond to the same perception unit, for example, the perception unit is perception unit D, and computation graph 11, computation graph 17, computation graph 23, and computation graph 24 correspond to the same perception unit, for example, the perception unit is perception unit E.

[0184] It can be understood that, although not shown in Figure 9B, each of the 27 calculation graphs, calculation graph 1 to calculation graph 27, can contain one or more nodes, and the embodiment of the present application does not specifically limit the number of nodes contained in the calculation graph and the topological relationship between the nodes.

[0185] Next, based on the principle of balanced computing power, the 27 computation graphs (Computation Graph 1-Computation Graph 27) can be divided into three groups. For example, these three groups can include Group x, Group y, and Group z. Group x includes Computation Graph 1, Computation Graph 12, Computation Graph 16, Computation Graph 20, Computation Graph 21, Computation Graph 22, Computation Graph 6, Computation Graph 7, and Computation Graph 10; Group y includes Computation Graph 2, Computation Graph 3, Computation Graph 5, Computation Graph 18, Computation Graph 27, Computation Graph 11, Computation Graph 17, Computation Graph 23, and Computation Graph 24; and Group z includes Computation Graph 4, Computation Graph 8, Computation Graph 9, Computation Graph 13, Computation Graph 14, Computation Graph 15, Computation Graph 19, Computation Graph 25, and Computation Graph 26. Clearly, by segmenting the hybrid computation graph, the computing power requirements of these three groups are comparable.

[0186] It can be understood that, since calculation graph 1, calculation graph 12, calculation graph 16, calculation graph 20, calculation graph 21, and calculation graph 22 correspond to perception unit A, calculation graph 1, calculation graph 12, calculation graph 16, calculation graph 20, calculation graph 21, and calculation graph 22 can be divided into the same group (for example, group x), which can avoid the deployment of multiple calculation graphs of the same perception unit being too dispersed, thereby reducing the switching cost during execution; similarly, since calculation graph 2, calculation graph 3, calculation graph 5, calculation graph 18, and calculation graph 27 correspond to perception unit B, calculation graph 2, calculation graph 3, calculation graph 5, calculation graph 18, and calculation graph 27 are divided into the same group (for example, group y), since calculation graph 4, calculation graph 8, calculation graph 9, calculation graph 13, calculation graph 14, calculation graph 15, calculation graph Figure 19, calculation graph 25, and calculation graph 26 correspond to perception unit C. Therefore, calculation graph 4, calculation graph 8, calculation graph 9, calculation graph 13, calculation graph 14, calculation graph 15, calculation graph 19, calculation graph 25, and calculation graph 26 are divided into the same group (for example, group z). Since calculation graph 6, calculation graph 7, and calculation graph 10 correspond to perception unit D, calculation graph 6, calculation graph 7, and calculation graph 10 are divided into the same group (for example, group x). Since calculation graph 11, calculation graph 17, calculation graph 23, and calculation graph 24 correspond to perception unit E, calculation graph 11, calculation graph 17, calculation graph 23, and calculation graph 24 are divided into the same group (for example, group y). This can avoid the deployment of multiple calculation graphs of the same perception unit being too dispersed, thereby reducing the switching cost during execution.

[0187] Then, these three groups can be assigned to three computing units respectively, where the specific principles of allocation can be described in the relevant description in the above embodiments and will not be repeated here. For example, group x can be assigned to computing unit 1, that is, the deployment location of group x is computing unit 1; group y can be assigned to computing unit 2, that is, the deployment location of group y is computing unit 2; group z can be assigned to computing unit 3, that is, the deployment location of group z is computing unit 3, so that each group can be deployed in the corresponding computing unit according to the determined deployment location.

[0188] After each of the computation graphs in computation graphs 1 to 27 is deployed in each computation unit based on the deployment location, the execution order of each computation graph within each computation unit can be determined based on the topological relationship. For example, the execution order of the computation graphs within each computation unit can be determined based on the slack of the computation graph. The specific method can refer to the relevant description in the above embodiment and will not be repeated here. The effect diagram of the final deployment result can refer to Figure 9B. Among them, in computation unit 1, the execution order of each computation graph is from left to right, starting from computation graph 1 to the end of computation graph 10; in computation unit 2, the execution order of each computation graph is from left to right, starting from computation graph 2 to the end of computation graph 24; in computation unit 3, the execution order of each computation graph is from left to right, starting from computation graph 4 to the end of computation graph 26.

[0189] It will be understood that the execution order between the above-mentioned computation graphs is merely an illustrative illustration and does not constitute a limitation on the embodiments of the present application.

[0190] Method 3: Deployment based on the type of sub-hybrid computation graph.

[0191] In approach 3, sub-hybrid computation graphs can be classified. For example, sub-hybrid computation graphs can be divided into two categories, one of which can be important and the other can be unimportant. For sub-hybrid computation graphs in the important category, approach 1 can be used for deployment; for sub-hybrid computation graphs in the unimportant category, approach 2 can be used for deployment.

[0192] For example, to ensure optimal single-network latency and guaranteed performance for important sub-hybrid computation graphs like BEV, deployment method 1 can be used. For other less important sub-hybrid computation graphs, deployment method 2 can be used, which reduces computing power requirements and switching costs. By using different deployment methods for different sub-hybrid computation graph types, we can balance the performance of important sub-hybrid computation graphs with computing power requirements.

[0193] For example, Method 3 may include: first, grouping each sub-hybrid computation graph in the hybrid computation graph, thereby obtaining multiple groups, each of which may include one or more sub-hybrid computation graphs. The grouping principle may be based on the principle of computing power balance. The specific grouping method can be referred to the relevant description in the above embodiment and will not be repeated here.

[0194] Next, each group can be assigned to a computing unit to determine a virtual deployment location for each group. Based on the virtual deployment location, the multiple groups can be virtually deployed to corresponding computing units. In other words, each group corresponds to a computing unit. The specific manner in which each group is assigned to a computing unit can be found in the description of the above embodiment and will not be repeated here.

[0195] It is understandable that although multiple groups are virtually deployed to different computing units, this virtual deployment does not mean that the sub-hybrid computing graph is actually deployed to the computing unit, but rather that the mapping relationship between the groups and the computing units is obtained.

[0196] Next, the corresponding deployment method can be selected according to the different types of sub-hybrid calculation graphs in each group. Among them, important types of sub-hybrid calculation graphs can be deployed using method 1, and non-important types of sub-hybrid calculation graphs can be deployed using method 2.

[0197] For example, suppose group 1 is virtually deployed to computing unit 1. Group 1 contains node 1 and node 2, node 1 is an important sub-hybrid computing graph, and node 2 is a non-important sub-hybrid computing graph. Then, node 2 in group 1 can be deployed using method 2, that is, the virtual deployment location of the group where node 2 is located can be used as the actual deployment location of node 2. In other words, node 2 can be actually deployed on computing unit 1 before the execution order of node 2 on computing unit 1 is determined. Node 1 in group 1 can be deployed using method 1, that is, node 1 in group 1 is virtually deployed. Although node 1 in group 1 is virtually deployed on computing unit 1, this virtual deployment only establishes a mapping relationship between node 1 and computing unit 1. That is, the actual deployment location of node 1 is not necessarily the virtual deployment location of the group where node 1 is located. During actual deployment, the actual deployment location and execution order of node 1 can be determined again based on, for example, a greedy algorithm. That is, node 1 may be actually deployed on computing unit 1 or on other computing units. Finally, node 1 is deployed based on its actual deployment location and execution order.

[0198] Method 3 is now exemplarily described with reference to FIG. 10A and FIG. 10B .

[0199] Taking a sub-hybrid computation graph as a node in a computation graph as an example, Figure 10A illustrates an exemplary schematic diagram of the deployment process when a sub-hybrid computation graph is used as a node in a computation graph. Referring to Figure 10A , assume that the hybrid computation graph contains 18 nodes, Node 1 through Node 18. Nodes 2, 5, 7, 8, 10, 11, 12, 16, and 17 belong to Computation Graph A, Nodes 1 and 3 belong to Computation Graph B, and Nodes 4, 6, 9, 13, 14, 15, and 18 belong to Computation Graph C.

[0200] By segmenting the 18 nodes in the hybrid computation graph, two groups can be obtained. For example, these two groups can be group a and group b, so that these two groups can be assigned to the two idle computation units, computation unit 1 and computation unit 2. Group a includes nodes 2, 5, 7, 8, 10, 11, 12, 16, and 17, and group b includes nodes 1, 3, 4, 6, 9, 13, 14, 15, and 18. Through the above grouping, the computing power of group a and group b is comparable, and the nodes of computation graph A are all divided into the same group, the nodes of computation graph B are all divided into the same group, and the nodes of computation graph C are all divided into the same group.

[0201] Next, the groups can be assigned to computing units to complete the virtual deployment of the groups on the computing units. Since the computing power of group a and group b is comparable, the groups can be arbitrarily assigned to deployable computing units (for example, idle computing units). For example, group a can be assigned to computing unit 1 and group b can be assigned to computing unit 2; or group a can be assigned to computing unit 2 and group b can be assigned to computing unit 1. This embodiment of the application does not specifically limit this.

[0202] Referring to FIG10A , taking group a assigned to computing unit 1 and group b assigned to computing unit 2 as an example, assuming that nodes 1 and 3 are important nodes, and nodes 2 and nodes 4-18 are non-important nodes, nodes 1 and 3 can be deployed using method 1, i.e., a greedy algorithm, for example, can be used to determine the actual deployment locations and execution order of nodes 1 and 3, and nodes 1 and 3 can be deployed based on the actual deployment locations and execution order of nodes 1 and 3. Nodes 2 and nodes 4-18 can be deployed using method 2, i.e., the virtual deployment locations of nodes 2 and nodes 4-18 can be used as the actual deployment locations of nodes 2 and nodes 4-18. For example, the actual deployment locations and virtual deployment locations of nodes 2, nodes 5, nodes 7, nodes 8, nodes 10, nodes 11, nodes 12, nodes 16, and nodes 17 are the same, all of which are computing unit 1. The actual deployment locations and virtual deployment locations of nodes 4, nodes 6, nodes 9, nodes 13, nodes 14, nodes 15, and nodes 18 are the same, all of which are computing unit 2. Next, after the actual deployment locations of nodes 2, 5, 7, 8, 10, 11, 12, 16, and 17 are determined, the execution order of nodes 2, 5, 7, 8, 10, 11, 12, 16, and 17 can be determined based on the topological relationship, thereby completing the complete deployment process of nodes 2, 5, 7, 8, 10, 11, 12, 16, and 17. Furthermore, after the actual deployment locations of nodes 4, 6, 9, 13, 14, 15, and 18 are determined, the execution order of nodes 4, 6, 9, 13, 14, 15, and 18 can be determined based on the topological relationship, thereby completing the complete deployment process of nodes 4, 6, 9, 13, 14, 15, and 18.

[0203] In the example shown in FIG10A , the deployment results obtained after deployment in Mode 1 and Mode 2 are respectively performed can be specifically referred to the relevant description in the above embodiments, which will not be repeated here.

[0204] Next, taking the sub-hybrid computation graph as an example, Figure 10B illustrates an exemplary schematic diagram of the deployment process when the sub-hybrid computation graph is used as a computation graph. Referring to Figure 10B, assume that the hybrid computation graph contains 18 computation graphs, namely, computation graphs 1 to 18. Among them, computation graphs 2, 5, 7, 8, 10, 11, 12, 16, and 17 correspond to perception unit A, computation graphs 1 and 3 correspond to perception unit B, and computation graphs 4, 6, 9, 13, 14, 15, and 18 correspond to perception unit C.

[0205] By segmenting the 18 computation graphs in the hybrid computation graph, two groups can be obtained. For example, these two groups can be group c and group d, so that these two groups can be assigned to the two idle computation units, computation unit 1 and computation unit 2. Group c includes computation graph 2, computation graph 5, computation graph 7, computation graph 8, computation graph 10, computation graph 11, computation graph 12, computation graph 16, and computation graph 17, and group d includes computation graph 1, computation graph 3, computation graph 4, computation graph 6, computation graph 9, computation graph 13, computation graph 14, computation graph 15, and computation graph 18. Through the above grouping, the computing power of group c and group d is equivalent, and the computation graphs corresponding to perception unit A are divided into the same group, the computation graphs corresponding to perception unit B are divided into the same group, and the computation graphs corresponding to perception unit C are divided into the same group.

[0206] Next, the groups can be assigned to computing units to complete the virtual deployment of the groups on the computing units. Since the computing power of groups c and d is comparable, the groups can be arbitrarily assigned to deployable computing units (e.g., idle computing units). For example, group c can be assigned to computing unit 1 and group d can be assigned to computing unit 2; or group c can be assigned to computing unit 2 and group d can be assigned to computing unit 1. This embodiment of the present application does not specifically limit this.

[0207] Referring to Figure 10B, taking group c assigned to computing unit 1 and group d assigned to computing unit 2 as an example, assuming that computing graph 1 and computing graph 3 are important class computing graphs, and computing graph 2, computing graph 4-computation graph 18 are non-important class computing graphs, then computing graph 1 and computing graph 3 can be deployed using method 1, that is, computing graph 1 and computing graph 3 can be determined using, for example, a greedy algorithm, and computing graph 1 and computing graph 3 can be deployed based on their actual deployment positions and execution order. Computational graph 2 and computational graphs 4-18 can be deployed using method 2, that is, the virtual deployment locations of computational graphs 2 and 4-18 can be used as the actual deployment locations of computational graphs 2 and 4-18. For example, the actual deployment locations and virtual deployment locations of computational graphs 2, 5, 7, 8, 10, 11, 12, 16, and 17 are the same, all of which are computational unit 1; the actual deployment locations and virtual deployment locations of computational graphs 4, 6, 9, 13, 14, 15, and 18 are the same, all of which are computational unit 2. Next, after determining the actual deployment locations of Graph 2, Graph 5, Graph 7, Graph 8, Graph 10, Graph 11, Graph 12, Graph 16, and Graph 17, the execution order of Graph 2, Graph 5, Graph 7, Graph 8, Graph 10, Graph 11, Graph 12, Graph 16, and Graph 17 can be determined based on the topological relationship, thereby completing the complete deployment process of Graph 2, Graph 5, Graph 7, Graph 8, Graph 10, Graph 11, Graph 12, Graph 16, and Graph 17. And after determining the actual deployment locations of calculation graphs 4, 6, 9, 13, 14, 15, and 18, the execution order of calculation graphs 4, 6, 9, 13, 14, 15, and 18 can be determined based on the topological relationship, thereby completing the complete deployment process of calculation graphs 4, 6, 9, 13, 14, 15, and 18.

[0208] In the example shown in FIG10B , the deployment results obtained after deployment using Mode 1 and Mode 2 respectively can be specifically referred to the relevant description in the above embodiments, which will not be repeated here.

[0209] It is understandable that any one of the above three deployment methods can be selected for use, and the embodiments of the present application do not impose any special limitations on this.

[0210] In some optional embodiments, for method 3, the deployment method for each sub-hybrid computation graph can be determined based on bandwidth pressure. For example, method 1 can be used for a sub-hybrid computation graph with low bandwidth pressure, or method 2 can be used for a sub-hybrid computation graph with high bandwidth pressure.

[0211] In some optional embodiments, after deploying the sub-hybrid computation graph in the computing unit, a deployment result can be obtained. The deployment result may include the deployment location and execution order of the sub-hybrid computation graph in the computing unit. The deployment result can then be associated with the current scenario, i.e., a mapping relationship between the deployment result and the current scenario is established, and this mapping relationship can be stored in a vehicle database. The current scenario may include, but is not limited to, the vehicle's current driving scenario or road scenario.

[0212] It can be understood that after the sub-hybrid computation graph is deployed in the computing unit, the vehicle's driving decision can be determined based on the execution result of the sub-hybrid computation graph in the computing unit, that is, the execution result of the sub-hybrid computation graph in the computing unit is used for the vehicle's decision.

[0213] In some optional embodiments, step 205 may be further included before step 201 .

[0214] Step 205: Query the vehicle database to see if there is a deployment result corresponding to the current scenario.

[0215] Specifically, if it is detected that there is a deployment result corresponding to the current scenario in the vehicle database, step 206 can be executed, that is, there is no need to execute steps 201 to 204, thereby improving the efficiency of autonomous driving. Or,

[0216] If it is detected that there is no deployment result corresponding to the current scenario in the vehicle database, step 201 may be executed.

[0217] Step 206: Deploy using the deployment result corresponding to the current scenario.

[0218] The driving decision-making method provided in the embodiments of the present application is exemplarily described above with reference to Figures 2 to 10. To further improve the performance of autonomous driving, the driving decision-making method may be simulated to improve the performance of the driving decision-making algorithm.

[0219] In some optional embodiments, step 207 may be further included after step 204 .

[0220] Step 207: Acquire the simulation value of the sub-hybrid calculation graph in the hybrid calculation graph.

[0221] It can be understood that the driving decision method in the above steps 201 to 206 can be applied to a simulation platform, wherein the simulation platform can be built based on a real scene or a virtual scene, and the embodiments of the present application do not specifically limit this.

[0222] Next, the above driving decision-making method is run in a simulation platform to obtain the simulation value of each sub-hybrid calculation graph in the hybrid calculation graph. The simulation value can be used to characterize the delay value of the sub-hybrid calculation graph in a real scenario.

[0223] Step 208: make corrections based on the comparison result between the simulation value and the theoretical value.

[0224] Specifically, the theoretical value may be a theoretical execution time of the sub-hybrid computation graph, wherein the theoretical value may be determined by the difference between the execution times of the two sub-hybrid computation graphs in FIG8-FIG10.

[0225] For example, taking the theoretical value of the sub-hybrid computation graph in FIG8 as an example, assuming that the execution time of node1 is t1 and the execution time of node3 is t2, the theoretical value of node1 is t2-t1.

[0226] The simulated values ​​can then be compared with the theoretical values.

[0227] If the simulation value is less than or equal to the theoretical value, it means that the error is within the allowable range, the driving decision algorithm meets the performance requirements, and the driving decision algorithm can be applied to actual driving. Or,

[0228] If the simulation value is greater than the theoretical value, it means that the error is not within the allowable range and the driving decision algorithm does not meet the performance requirements. The driving decision algorithm can be corrected.

[0229] Among them, the ways to correct the driving decision algorithm can include the following two methods:

[0230] Method 1: Modify the calculation graph.

[0231] It is understandable that, as mentioned above, the computational graph includes multiple nodes, each of which has a corresponding node weight. Therefore, the driving decision algorithm can be corrected by correcting the node weight.

[0232] Method 2: Correct the bandwidth.

[0233] It is understandable that one or more of the above two correction methods may be selected and used, and the embodiments of the present application do not impose any special limitations on this.

[0234] It is understandable that after correction is made once using the above correction method, simulation can be performed again, and the results of the theoretical value and the simulation value can be verified again. If the result still does not meet expectations, multiple corrections can be made until the result meets expectations.

[0235] In some optional embodiments, for a modified driving decision algorithm that meets expectations, a simulation result of the driving decision algorithm can be bound to a simulated driving scenario and stored in the vehicle. The simulation result can be a deployment result obtained through simulation.

[0236] FIG11 is a schematic diagram of the structure of an embodiment of the driving decision-making device of the present application. As shown in FIG11 , the driving decision-making device 1100 is applied to a vehicle. The vehicle includes multiple computing units. The driving decision-making device 1100 may include: a construction module 1110 and a deployment module 1120; wherein,

[0237] A construction module 1110 is configured to construct a plurality of computation graphs, each of which includes a plurality of nodes; construct a hybrid computation graph based on the plurality of computation graphs, the hybrid computation graph including a topological relationship, the topological relationship being used to characterize a topological relationship between the plurality of computation graphs and between nodes in each of the plurality of computation graphs;

[0238] A deployment module 1120 is configured to deploy multiple sub-hybrid computation graphs in multiple computing units based on the topological relationship to obtain deployment results, where the multiple sub-hybrid computation graphs are the multiple computation graphs or multiple nodes in the multiple computation graphs. The multiple computing units are configured to execute the multiple sub-hybrid computation graphs based on the deployment results to obtain execution results, which are used for driving decisions of the vehicle.

[0239] In one possible implementation, the deployment result includes the deployment position and execution order of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs, and the deployment module 1120 is specifically used to simultaneously determine the deployment position and execution order of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs; based on the deployment position and execution order of each sub-hybrid computation graph, the multiple sub-hybrid computation graphs are deployed in the multiple computing units, and the execution order of the sub-hybrid computation graphs is obtained based on the topological relationship, wherein the deployment position is the position of each sub-hybrid computation graph in the multiple computing units.

[0240] In one possible implementation, the deployment result includes a deployment location and an execution order of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs. The deployment module 1120 is specifically configured to determine a deployment location of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs, and deploy the multiple sub-hybrid computation graphs to corresponding computing units based on the deployment location, wherein the deployment location is a location of each sub-hybrid computation graph in the multiple computing units.

[0241] In any target computing unit, the execution order of each sub-hybrid computing graph in the target computing unit is determined based on the topological relationship, and the execution order of the multiple sub-hybrid computing graphs in the target computing unit is deployed based on the execution order, wherein the target computing unit is a computing unit that contains sub-hybrid computing graphs with the same deployment location.

[0242] In one possible implementation, the deployment result includes the deployment location and execution order of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs, the type of the sub-hybrid computation graph includes a first type and a second type, and the deployment module 1120 is specifically configured to split the multiple sub-hybrid computation graphs into multiple groups, determine virtual deployment locations of the multiple groups, and virtually deploy the multiple groups to corresponding computing units based on the virtual deployment locations, wherein each of the multiple groups includes one or more sub-hybrid computation graphs in the multiple sub-hybrid computation graphs;

[0243] Simultaneously determining actual deployment locations and execution order of the sub-hybrid computation graphs of the first type in the multiple groups, the execution order of the sub-hybrid computation graphs of the first type being obtained based on the topological relationship; deploying the sub-hybrid computation graphs of the first type in the multiple computing units based on the actual deployment locations and execution order of the sub-hybrid computation graphs of the first type; and,

[0244] Determine the virtual deployment location of the second-type sub-hybrid computing graph in the multiple groups as the actual deployment location of the second-type sub-hybrid computing graph, and deploy the second-type sub-hybrid computing graph to the corresponding computing unit based on the actual deployment location of the second-type sub-hybrid computing graph; in any target computing unit, determine the execution order of each second-type sub-hybrid computing graph in the target computing unit based on the topological relationship, and deploy the execution order of each second-type sub-hybrid computing graph in the target computing unit based on the execution order, wherein the target computing unit is a computing unit containing the second-type sub-hybrid computing graph with the same actual deployment location.

[0245] In one possible implementation, the execution order of the sub-hybrid computation graph is determined by the slack of the sub-hybrid computation graph, and the slack is used to characterize the importance of the sub-hybrid computation graph in the hybrid computation graph based on the topological relationship.

[0246] In one possible implementation, the driving decision-making device 1100 further includes:

[0247] A storage module is used to associate the deployment result with a target scene and store it in the vehicle, wherein the target scene is the driving scene or road scene in which the vehicle is currently located.

[0248] In one possible implementation, the driving decision-making device 1100 further includes:

[0249] A query module, configured to query whether a deployment result corresponding to the target scenario is stored in the vehicle;

[0250] The construction module 1110 is specifically used to construct multiple calculation graphs if no deployment result corresponding to the target scenario is found in the vehicle.

[0251] In one possible implementation, the construction module 1110 is specifically used to construct multiple calculation graphs based on the value of a first parameter and the value of a second parameter, wherein the first parameter is used to characterize the calculation delay of each node in any one of the multiple calculation graphs, and the second parameter is used to characterize the amount of data transmission between two nodes in any one of the multiple calculation graphs.

[0252] In one possible implementation, the construction module 1110 is specifically used to determine values ​​of a third parameter and a fourth parameter, wherein the third parameter is used to represent an execution cycle corresponding to any one of the multiple computation graphs, and the fourth parameter is used to represent the number of paths corresponding to any one of the multiple computation graphs, and the number of paths is used to represent the number of concurrent executions of the same computation graph.

[0253] The plurality of computation graphs are fused into a hybrid computation graph based on values ​​of the third parameter and the fourth parameter.

[0254] In one possible implementation, the values ​​of the third parameter and the fourth parameter are determined by the scene in which the vehicle is currently located.

[0255] In one possible implementation, the driving decision device 1100 is a vehicle or a chip.

[0256] The driving decision-making device 1100 provided in the embodiment shown in FIG11 can be used to execute the technical solution of the method embodiment shown in this application. Its implementation principle and technical effects can be further referred to the relevant description in the method embodiment.

[0257] It should be understood that the division of the various modules of the driving decision-making device 1100 shown in Figure 11 is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a single physical entity, or physically separated. Furthermore, these modules can be implemented entirely in the form of software called through a processing element; or entirely in the form of hardware; or some modules can be implemented in the form of software called through a processing element, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated into a chip of an electronic device. The implementation of other modules is similar. Furthermore, these modules can be fully or partially integrated together, or implemented independently. During implementation, each step of the above method or each of the above modules can be completed by hardware integrated logic circuits in the processor element or by software instructions.

[0258] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more microprocessors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, these modules may be integrated together to implement a system-on-a-chip (SOC).

[0259] In the above embodiments, the processor involved may include, for example, a CPU, a DSP, a microcontroller, or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units; hereinafter referred to as: NPU) and an image signal processor (hereinafter referred to as: ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASICs, or one or more integrated circuits for controlling the execution of the program of the technical solution of this application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.

[0260] An embodiment of the present application also provides a readable storage medium, which stores a program. When the program is run on an electronic device, the electronic device executes the method provided by the embodiment shown in the present application.

[0261] An embodiment of the present application also provides a program product, which includes a program. When the program product is run on an electronic device, the electronic device executes the method provided by the embodiment shown in the present application.

[0262] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b, c can be single or multiple.

[0263] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented using a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0264] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0265] In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), magnetic disk or optical disk, and other media that can store program codes.

[0266] The above description is merely a specific embodiment of the present application. Any person skilled in the art may easily conceive of variations or substitutions within the technical scope disclosed in this application, and such variations or substitutions shall be within the scope of protection of this application. The scope of protection of this application shall be subject to the scope of protection of the claims.

Claims

1. A driving decision-making method, characterized in that: Applied to a vehicle, the vehicle comprising a plurality of computing units, the method comprising: Constructing a plurality of computation graphs, wherein any one of the plurality of computation graphs includes a plurality of nodes; Constructing a hybrid computation graph based on the multiple computation graphs, wherein the hybrid computation graph includes a topological relationship, wherein the topological relationship is used to represent a topological relationship between the multiple computation graphs and between nodes in each of the multiple computation graphs; Based on the topological relationship between the multiple computation graphs, multiple sub-hybrid computation graphs are deployed in the multiple computation units to obtain deployment results, the multiple sub-hybrid computation graphs being the multiple computation graphs or multiple nodes in the multiple computation graphs, and the multiple computation units being used to execute the multiple sub-hybrid computation graphs based on the deployment results to obtain execution results, which are used for driving decisions of the vehicle.

2. The method according to claim 1, characterized in that The method further comprises: At the same time, the deployment position and execution order of each sub-hybrid computation graph among the multiple sub-hybrid computation graphs are determined; based on the deployment position and execution order of each sub-hybrid computation graph, the multiple sub-hybrid computation graphs are deployed in the multiple computing units, and the execution order of the sub-hybrid computation graphs is obtained based on the topological relationship, wherein the deployment position is the position of each sub-hybrid computation graph among the multiple sub-hybrid computation graphs in the multiple computing units.

3. The method according to claim 1, characterized in that The deployment result includes a deployment location and an execution order of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs, and automatically deploying the multiple sub-hybrid computation graphs in the multiple computing units based on the execution order includes: Determining a deployment location of each sub-hybrid computation graph in the plurality of sub-hybrid computation graphs, and deploying the plurality of sub-hybrid computation graphs to corresponding computing units based on the deployment location, wherein the deployment location is a location of each sub-hybrid computation graph in the plurality of computing units; In any target computing unit, the execution order of each sub-hybrid computing graph in the target computing unit is determined based on the topological relationship, and the execution order of the multiple sub-hybrid computing graphs in the target computing unit is deployed based on the execution order, wherein the target computing unit is a computing unit that contains sub-hybrid computing graphs with the same deployment location.

4. The method according to claim 1, wherein The deployment result includes an actual deployment location and execution order of each sub-hybrid computation graph in the multiple sub-hybrid computation graphs, the types of the sub-hybrid computation graphs include a first type and a second type, and deploying the multiple sub-hybrid computation graphs in multiple computing units based on the topological relationship includes: Splitting the multiple sub-hybrid computation graphs into multiple groups, determining virtual deployment locations of the multiple groups, and virtually deploying the multiple groups to corresponding computing units based on the virtual deployment locations, wherein each of the multiple groups includes one or more sub-hybrid computation graphs in the multiple sub-hybrid computation graphs; Simultaneously determining actual deployment locations and execution order of the sub-hybrid computation graphs of the first type in the multiple groups, the execution order of the sub-hybrid computation graphs of the first type being obtained based on the topological relationship; deploying the sub-hybrid computation graphs of the first type in the multiple computing units based on the actual deployment locations and execution order of the sub-hybrid computation graphs of the first type; and, Determine the virtual deployment location of the second-type sub-hybrid computing graph as the actual deployment location of the second-type sub-hybrid computing graph, and deploy the second-type sub-hybrid computing graph to the corresponding computing unit based on the actual deployment location of the second-type sub-hybrid computing graph; in any target computing unit, determine the execution order of each second-type sub-hybrid computing graph in the target computing unit based on the topological relationship, and deploy the execution order of each second-type sub-hybrid computing graph in the target computing unit based on the execution order, wherein the target computing unit is a computing unit that contains the second-type sub-hybrid computing graph with the same actual deployment location.

5. The method according to any one of claims 2 to 4, characterized in that: The execution order of the sub-hybrid computation graph is determined by the relaxation of the sub-hybrid computation graph, and the relaxation is used to represent the importance of the sub-hybrid computation graph in the hybrid computation graph based on the topological relationship.

6. The method according to any one of claims 1 to 5, characterized in that After automatically deploying the multiple sub-hybrid computation graphs within the multiple computing units based on the topological relationship between the multiple computation graphs, the method further includes: The deployment result is associated with the target scene and stored in the vehicle, wherein the target scene is the driving scene or road scene in which the vehicle is currently located.

7. The method according to claim 6, characterized in that Before constructing the plurality of computation graphs, the method further includes: Querying whether a deployment result corresponding to the target scenario is stored in the vehicle; The constructing of multiple computational graphs includes: If no deployment result corresponding to the target scenario is found in the vehicle, multiple computation graphs are constructed.

8. The method according to any one of claims 1 to 7, characterized in that The constructing of multiple computational graphs includes: Multiple computation graphs are constructed based on the values ​​of a first parameter and a second parameter, wherein the first parameter is used to characterize the computation delay of each node in any one of the multiple computation graphs, and the second parameter is used to characterize the amount of data transmission between two nodes in a computation graph.

9. The method according to any one of claims 1 to 8, characterized in that The constructing a hybrid computation graph based on the multiple computation graphs includes: Determine values ​​of a third parameter and a fourth parameter, wherein the third parameter is used to represent an execution cycle corresponding to any one of the multiple computation graphs, and the fourth parameter is used to represent the number of paths corresponding to any one of the multiple computation graphs, and the number of paths is used to represent the number of concurrent executions of the same computation graph; The plurality of computation graphs are fused into a hybrid computation graph based on values ​​of the third parameter and the fourth parameter.

10. The method according to claim 9, characterized in that The values ​​of the third parameter and the fourth parameter are determined by the scene in which the vehicle is currently located.

11. A driving decision-making device, characterized in that: It comprises one or more functional modules, and the one or more functional modules are used to implement the driving decision-making method according to any one of claims 1 to 10.

12. The device according to claim 11, characterized in that The driving decision-making device is a chip, or the driving decision-making device is a vehicle.

13. A driving decision-making device, characterized in that: include: A processor and a memory, wherein the memory is used to store a program; and the processor is used to run the program to implement the driving decision-making method according to any one of claims 1 to 10.

14. The device according to claim 13, characterized in that The driving decision-making device is a chip, or the driving decision-making device is a vehicle.

15. A readable storage medium, characterized in that The readable storage medium stores a program, and when the program is run on the driving decision-making device, the driving decision-making method according to any one of claims 1 to 10 is implemented.

16. A program, characterized in that When the program is executed on a processor of a driving decision-making device, the driving decision-making device is caused to execute the driving decision-making method according to any one of claims 1 to 10.

Images