Compilation method and device for intelligent driving neural network model, storage medium and electronic equipment

Abstract

The invention relates to the technical field of automobiles, and discloses a compiling method for an intelligent driving neural network model, and the method comprises the steps: recognizing a driving scene which needs to be deployed by the intelligent driving neural network model; selecting a target operator fusion strategy corresponding to the driving scene from an operator fusion strategy library; according to the target operator fusion strategy, constructing composite operators corresponding to a plurality of basic operators in the intelligent driving neural network model calculation graph; and compiling the intelligent driving neural network model calculation graph by using a composite operator to obtain a compiled intelligent driving neural network model. By introducing an operator fusion strategy selection mechanism of driving scene perception, the compiler can differentially balance the calculation efficiency and the model precision for different driving environments, so that the balance between the performance and the precision of intelligent driving neural network model compiling is realized on the premise of ensuring the driving safety and the perception reliability. The invention further discloses a compiling device for the intelligent driving neural network model, a computer readable storage medium and electronic equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobiles, for example to a compilation method and device for a smart driving neural network model, a storage medium and an electronic device. BACKGROUND

[0002] With the in-depth application of artificial intelligence technology in the field of automatic driving, the intelligent driving neural network model based on deep learning has become the core technical support for environment perception, decision planning and control execution. The intelligent driving neural network model usually has a complex structure, adopts advanced architectures such as bird's eye view perception network, multi-task learning model, time series prediction network, and the number of layers can reach hundreds of layers and the number of parameters can reach tens of millions or even hundreds of millions. When deployed in a real vehicle, the intelligent driving neural network model must complete high real-time inference on limited vehicle-mounted computing resources, usually requiring an end-to-end delay of less than 100 milliseconds, while ensuring extremely high computing accuracy and system reliability. In the prior art, when compiling and optimizing the intelligent driving neural network model, a “one-size-fits-all” operator fusion strategy is often used: the compiler predefines a fixed and universal operator fusion rule, and uses the same fusion mode for compilation and optimization.

[0003] In the process of implementing the embodiments of the present disclosure, it is found that at least the following problems exist in the related art: For the intelligent driving neural network model, the “one-size-fits-all” fusion strategy cannot achieve the optimal trade-off between performance and accuracy under diversified driving conditions, so that the compilation result is either insufficient in performance or has too high accuracy risk.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an overall description of the application, nor is it intended to determine key / important elements or delineate the scope of these embodiments, but as a prelude to the detailed description below.

[0006] The embodiments of the present disclosure provide a compilation method and device for an intelligent driving neural network model, a storage medium and an electronic device, to achieve the balance between performance and accuracy of the intelligent driving neural network model compilation by introducing an operator fusion strategy selection mechanism for driving scene perception.

[0007] In some embodiments, the compiling method for the intelligent driving neural network model comprises: identifying a driving scene required for deployment of the intelligent driving neural network model; selecting a target operator fusion strategy corresponding to the driving scene from an operator fusion strategy library; constructing a composite operator corresponding to a plurality of basic operators in a calculation graph of the intelligent driving neural network model according to the target operator fusion strategy; and compiling the calculation graph of the intelligent driving neural network model by using the composite operator to obtain a compiled intelligent driving neural network model.

[0008] Optionally, selecting the target operator fusion strategy corresponding to the driving scene from the operator fusion strategy library comprises: in the case of a high-speed scene, determining that the target operator fusion strategy is a first operator fusion strategy; in the case of an urban scene, determining that the target operator fusion strategy is a second operator fusion strategy; wherein the operator fusion degree in the second operator fusion strategy is less than the operator fusion degree in the first operator fusion strategy; and in the case of a parking scene, determining that the target operator fusion strategy is a third operator fusion strategy; wherein the operator fusion degree in the third operator fusion strategy is less than the operator fusion degree in the second operator fusion strategy.

[0009] Optionally, constructing the composite operator corresponding to the plurality of basic operators in the calculation graph of the intelligent driving neural network model according to the target operator fusion strategy comprises: selecting any node in the calculation graph of the intelligent driving neural network model as a candidate root node; starting from the candidate root node, traversing the graph structure of the calculation graph of the intelligent driving neural network model in reverse based on an operator rule library corresponding to the target operator fusion strategy, and taking a graph structure meeting a subgraph structure in the operator rule library as a candidate subgraph; and constructing the composite operator by using the candidate subgraph.

[0010] Optionally, taking the graph structure meeting the subgraph structure in the operator rule library as the candidate subgraph comprises one or more of the following ways: matching the graph structure of the calculation graph of the intelligent driving neural network model with the subgraph structure in the operator rule library, and taking a successfully matched graph structure as a candidate subgraph; in the case of a high-speed scene, taking all candidate subgraphs as the candidate subgraph; in the case of an urban scene, taking a candidate subgraph meeting a subgraph performance-to-price ratio condition as the candidate subgraph; and in the case of a parking scene, taking a candidate subgraph meeting a subgraph accuracy condition as the candidate subgraph.

[0011] Optionally, matching the graph structure of the calculation graph of the intelligent driving neural network model with the subgraph structure in the operator rule library, and taking a successfully matched graph structure as a candidate subgraph comprises: determining a target subgraph in the operator rule library subgraph structure that is the same as an output node type of the candidate root node; recursively and reversely matching a predecessor node according to a topological structure of the target subgraph to obtain a matching node in the calculation graph of the intelligent driving neural network model; and constructing the candidate subgraph by using all the matching nodes.

[0012] Optionally, the composite operator is constructed using the candidate sub-graph, including: performing composite verification on the candidate sub-graph; wherein the composite verification includes semantic verification and equivalence verification; and constructing the composite node operator using the candidate sub-graph that passes the composite verification.

[0013] Optionally, the semantic verification includes: checking whether the node attribute of each node in the candidate sub-graph satisfies a constraint condition defined in the target operator fusion strategy; and the equivalence verification includes: deriving a mathematical expression of an original calculation sequence of the candidate sub-graph and a calculation formula of the composite operator after fusion; and comparing whether the mathematical expression and the calculation formula are equal in mathematical definition.

[0014] Optionally, the intelligent driving neural network model calculation graph is compiled using the composite operator, including: removing an original node corresponding to the composite operator in the intelligent driving neural network model calculation graph, and creating a new composite operator node; connecting an input end of the new composite operator node to an external input node of an original node sub-graph, and connecting an output end of the new composite operator node to a successor node connected to an output node of the original node sub-graph.

[0015] In some embodiments, a compiling apparatus for an intelligent driving neural network model includes: an identification module configured to identify a driving scene required to be deployed by the intelligent driving neural network model, and select a target operator fusion strategy corresponding to the driving scene from an operator fusion strategy library; a conversion module in communication connection with the identification module, and configured to construct a composite operator corresponding to a plurality of basic operators in an intelligent driving neural network model calculation graph according to the target operator fusion strategy; and a compiling module in communication connection with the conversion module, and configured to compile the intelligent driving neural network model calculation graph using the composite operator, and obtain a compiled intelligent driving neural network model.

[0016] In some embodiments, a computer readable storage medium stores a computer program for executing the foregoing compiling method for an intelligent driving neural network model.

[0017] In some embodiments, an electronic device includes: a processor, a memory, and a computer program stored on the memory, wherein the processor, when executing the computer program, implements the foregoing compiling method for an intelligent driving neural network model.

[0018] The compiling method and apparatus for an intelligent driving neural network model, the storage medium, and the electronic device provided by the embodiments of the present disclosure can achieve the following technical effects: In the technical solution of the present disclosure, when compiling the intelligent driving neural network model, first, the driving scene required by the intelligent driving neural network model to be deployed is identified, a target operator fusion strategy corresponding to the driving scene is selected from an operator fusion strategy library, then a composite operator corresponding to multiple basic operators in the intelligent driving neural network model calculation graph is constructed according to the target operator fusion strategy, and the intelligent driving neural network model calculation graph is compiled by using the composite operator to obtain an optimized intelligent driving neural network model. In this way, by introducing the operator fusion strategy selection mechanism with driving scene perception, the compiler can balance the calculation efficiency and model accuracy differently for different driving environments, thereby balancing the performance and accuracy of the intelligent driving neural network model compilation under the premise of ensuring driving safety and perception reliability.

[0019] The foregoing general description and the following description are merely exemplary and explanatory, and are not intended to limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0020] One or more embodiments are exemplarily illustrated by corresponding drawings, which do not constitute limitations on the embodiments, elements with the same reference numerals in the drawings are shown as similar elements, the drawings do not constitute proportional limits, and wherein: Figure 1 is a flow diagram of a compilation method for an intelligent driving neural network model provided by an embodiment of the present disclosure; Figure 2 is a flow diagram of another compilation method for an intelligent driving neural network model provided by an embodiment of the present disclosure; Figure 3 is a flow diagram of another compilation method for an intelligent driving neural network model provided by an embodiment of the present disclosure; Figure 4 is a flow diagram of another compilation method for an intelligent driving neural network model provided by an embodiment of the present disclosure; Figure 5A is a schematic diagram of an original intelligent driving neural network model calculation graph provided by an embodiment of the present disclosure; Figure 5B is a schematic diagram of a compiled intelligent driving neural network model calculation graph provided by an embodiment of the present disclosure; Figure 6 is a structural schematic diagram of a compilation device for an intelligent driving neural network model provided by an embodiment of the present disclosure; Figure 7 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0022] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0023] Unless otherwise stated, the term "multiple" means two or more. In embodiments of this disclosure, the character " / " indicates that the preceding and following objects are in an "OR" relationship. For example, A / B means: A or B. The term "and / or" describes an association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B. The term "correspondence" can refer to an association or binding relationship; A corresponding to B means that there is an association or binding relationship between A and B.

[0024] The following explains some key technical terms used in the embodiments of this disclosure: Intelligent driving neural network models, also known as smart driving neural network models, are deep learning-based algorithmic models specifically designed for intelligent driving tasks such as environmental perception, decision-making and planning, and control execution. These models have complex, multi-layered structures, such as BEV (Bird's-eye View) perception networks, multi-task learning models, or temporal prediction models, and contain millions to hundreds of millions of trainable parameters. An intelligent driving neural network model is an abstract set of functions that defines the mathematical mapping from sensor inputs (such as images and point clouds) to driving-related outputs (such as object detection boxes and path planning).

[0025] The computational graph of an intelligent driving neural network model is an intermediate representation (IR) of the model. It is a data flow description of the computation process of the intelligent driving neural network model in the form of a directed acyclic graph. In the computational graph, nodes represent computational operations (operators), and edges represent multidimensional data tensors flowing between operations. The computational graph transforms the abstract model into a clear, traversable, and transformable graph structure.

[0026] Basic operators, also known as atomic operators or primal operators, are the fundamental, indivisible computational units that constitute the computational graph of an intelligent driving neural network model. Each basic operator corresponds to a specific computation that has a basic implementation (kernel) on the target hardware, such as two-dimensional convolution (Conv2D), batch normalization (BatchNorm), linear fully connected (Linear), activation functions (ReLU, SiLU), etc.

[0027] Composite operators are single, coarse-grained computational nodes generated during the compilation phase by fusing subgraphs of the computation graph of the intelligent driving neural network model that conform to a predefined pattern and consist of multiple adjacent basic operators, according to an operator fusion strategy. For example, the three basic operators Conv2D -> BatchNorm -> ReLU can be fused into a single Fused_Conv_BN_ReLU composite operator.

[0028] During compilation, the intelligent driving neural network model is converted into an intelligent driving neural network model computation graph. The computation graph initially consists of numerous fine-grained basic operators. Through scene-adaptive compilation optimization, the basic operator subgraphs that meet the conditions are replaced with high-performance composite operators, ultimately generating a deeply optimized compiled intelligent driving neural network model that is suitable for efficient execution on the target automotive chip.

[0029] Combination Figure 1 As shown in the embodiments of this disclosure, a compilation method for an intelligent driving neural network model is provided, including the following steps: S101 identifies the driving scenarios required for the deployment of the intelligent driving neural network model.

[0030] The driving scenarios required for the deployment of intelligent driving neural network models include highway scenarios, urban scenarios, and parking scenarios. Among these: High-speed scenarios refer to the situation where vehicles travel on closed, structured highways or expressways. In high-speed scenarios, lane lines are clear, the behavior of traffic participants (mainly other motor vehicles) is relatively regular and predictable, the traffic flow direction is unidirectional, there is almost no interference from pedestrians or non-motorized vehicles, and vehicles are in a state of continuous high-speed movement.

[0031] Urban scenarios refer to the conditions in which vehicles travel on open, semi-structured urban roads and streets. In urban scenarios, the driving environment includes intersections, traffic lights, a large number of traffic participants with uncertain behavior (motor vehicles, non-motor vehicles, and pedestrians), road construction, temporary roadblocks, etc., and vehicles are in a state of low to medium speed, frequent starts and stops, and intensive interactions.

[0032] Parking scenarios refer to situations where vehicles autonomously park themselves in and out of tight parking spaces while at low speeds (usually below 15 km / h) or stationary. In parking scenarios, the driving environment is extremely confined, requiring centimeter-level absolute accuracy, relying on high-resolution sensing (such as ultrasonic radar and surround-view fisheye cameras), and calculations are typically performed when the vehicle is static or at extremely low speeds.

[0033] In practical applications, the compiler (i.e., the compilation device) can infer the driving scenario to be deployed by analyzing the model structure and input / output of the intelligent driving neural network model. For example, after loading the computation graph of the intelligent driving neural network model, the compiler analyzes the structural characteristics of the model: if it contains a specialized module for fisheye camera distortion correction, the driving scenario is determined to be a parking scenario; if it contains a complex temporal fusion module (such as Transformer) to handle dynamic targets, the driving scenario is determined to be a highway scenario or an urban scenario. Similarly, analyzing the input / output of the intelligent driving neural network model: if the model relies solely on fisheye camera and ultrasonic radar inputs, and the output is parking space corner points and steering wheel angles, the driving scenario is determined to be a parking scenario; if the model primarily processes forward-looking cameras and long-range radar, and the output is lane lines and distant targets, the driving scenario is determined to be a highway scenario; if the model integrates multiple cameras and LiDAR, and outputs dense semantic segmentation maps and pedestrian key points, the driving scenario is determined to be an urban scenario.

[0034] S102, Select the target operator fusion strategy corresponding to the driving scenario from the operator fusion strategy library.

[0035] The operator fusion strategy is a complete set of rules that guides the compiler on how to discover, evaluate, verify, and execute operator fusion behavior in the computation graph. The operator fusion strategy includes an operator rule base, a matching and selection mechanism, and verification criteria and thresholds. Specifically: the operator rule base defines the subgraph structures that can be fused (such as Conv-BN-ReLU), as well as the applicable conditions and constraints for each subgraph structure; the matching and selection mechanism specifies how to find candidate subgraphs in the computation graph of the intelligent driving neural network model, and the algorithm for deciding among multiple conflicting candidates; the verification criteria and thresholds define the numerical precision required to determine whether a candidate subgraph is safe and acceptable.

[0036] Optionally, a target operator fusion strategy corresponding to the driving scenario is selected from the operator fusion strategy library, including: when the driving scenario is a highway scenario, the target operator fusion strategy is determined to be a first operator fusion strategy; when the driving scenario is an urban scenario, the target operator fusion strategy is determined to be a second operator fusion strategy, wherein the operator fusion degree in the second operator fusion strategy is less than the operator fusion degree in the first operator fusion strategy; when the driving scenario is a parking scenario, the target operator fusion strategy is determined to be a third operator fusion strategy, wherein the operator fusion degree in the third operator fusion strategy is less than the operator fusion degree in the second operator fusion strategy.

[0037] Here, operator fusion degree is a quantitative indicator that measures the aggressiveness of the operator fusion strategy. Operator fusion degree can be calculated using the following formula:

[0038] in, For operator fusion degree, To incorporate the breadth coefficient, For the fusion depth coefficient, α represents the risk tolerance coefficient, β and γ are the weighting factors of the fusion breadth coefficient, fusion depth coefficient and risk tolerance coefficient, respectively, and S is the operator fusion strategy.

[0039] In some possible implementations, the fusion breadth coefficient is calculated using the following formula:

[0040] in, Let S be the set of all fusionable subgraph patterns in the operator rule base of the operator fusion strategy S. This is the set of all subgraph patterns theoretically supported by the compilation system.

[0041] In some possible implementations, the fusion depth coefficient is calculated using the following formula:

[0042] in, Calculate the total number of fused nodes in graph G for the intelligent driving neural network model. Calculate the total number of nodes (basic operators) in graph G for the intelligent driving neural network model.

[0043] In some possible implementations, the risk tolerance coefficient is calculated using the following formula:

[0044] in, Let S be the maximum relative error allowed in the numerical equivalence verification of the operator fusion strategy S. This is the minimum error threshold.

[0045] The first operator fusion strategy includes: an operator rule base encompassing classic fusion (e.g., Conv-BN-ReLU) and complex cross-layer fusion (e.g., the entire residual block, small attention modules); a greedy matching mechanism that matches the most likely fusionable subgraphs; and verification criteria and thresholds that allow for errors smaller than a small amount (e.g., the relative error of the output tensor is on the order of 1e-3). The second operator fusion strategy includes: an operator rule base encompassing classic fusion (e.g., Conv-BN-ReLU), medium-complexity, and verified patterns (e.g., Conv-BN-Add); a selective matching mechanism that chooses the most cost-effective fusionable subgraphs; and verification criteria and thresholds that require high-precision matching (e.g., the relative error of the output tensor is on the order of 1e-5). The third operator fusion strategy includes: an operator rule base encompassing a few patterns whose mathematical equivalence has been rigorously proven (e.g., Conv-BN); a precise matching mechanism that matches absolutely safe fusionable subgraphs; and verification criteria and thresholds that pursue bit-level or near-bit-level equivalence (e.g., using high-precision reference comparisons).

[0046] By predefining a series of operator fusion strategies, ranging from aggressive to conservative and strictly bound to driving scenarios, the compiler is transformed from a static optimization tool into an adaptive optimization system that can understand application scenarios and make intelligent trade-offs. In this way, it can find the optimal solution for each specific intelligent driving task within the triangular constraints of "performance-accuracy-safety".

[0047] S103, Based on the target operator fusion strategy, construct composite operators corresponding to multiple basic operators in the computation graph of the intelligent driving neural network model.

[0048] In some possible implementations, based on the target operator fusion strategy, composite operators corresponding to multiple basic operators in the computation graph of the intelligent driving neural network model are constructed. Specifically, this includes: selecting any node in the computation graph of the intelligent driving neural network model as a candidate root node; based on the operator rule base corresponding to the target operator fusion strategy, traversing the graph structure of the computation graph of the intelligent driving neural network model backward from the candidate root node, and taking the graph structure that conforms to the subgraph structure in the operator rule base as a candidate subgraph; and using the candidate subgraphs to construct composite operators.

[0049] By starting from any node as a candidate root node and performing reverse traversal and precise matching based on the operator rule library corresponding to the driving scenario, the system can intelligently and efficiently identify fusionable graph structures in the computation graph of the intelligent driving neural network model that conform to the preset subgraph structure in the operator rule library. These structures are then constructed into composite operators, enabling the compiler to adaptively perform differentiated fusion for different driving scenarios. This significantly reduces the complexity of the computation graph of the intelligent driving neural network model, reduces memory access overhead and kernel startup times, while ensuring the semantic correctness and numerical equivalence of the model. Ultimately, this achieves a comprehensive improvement in the inference efficiency of the intelligent driving neural network model and efficient utilization of onboard computing resources.

[0050] S104, use composite operators to compile the computation graph of the intelligent driving neural network model to obtain the compiled intelligent driving neural network model.

[0051] In some possible implementations, the computation graph of the intelligent driving neural network model is compiled using composite operators to obtain the compiled intelligent driving neural network model. Specifically, this includes: removing the original node corresponding to the composite operator in the computation graph of the intelligent driving neural network model and creating a new composite operator node; connecting the input of the new composite operator node to the external input node of the original node subgraph; and connecting the output of the new composite operator node to the successor node connected to the output node of the original node subgraph.

[0052] Based on the intelligent driving neural network model computation graph rewriting technology, the original subgraph (composed of multiple basic operators) that has been verified to be fusionable is "cut out" from its overall intelligent driving neural network model computation graph, and a single composite operator node encapsulating equivalent computational semantics is "implanted" in situ. By precisely anchoring the input end of the new node to the external input source of the atomic graph, and seamlessly bridging its output end to the subsequent computation nodes driven by the original output of the atomic graph, the internal computational organization of the intelligent driving neural network model computation graph is completely changed while keeping the input-output interface and logic of the intelligent driving neural network model computation graph unchanged at the data flow level.

[0053] By replacing multiple basic operator nodes with a single composite operator node, the number of nodes and connection complexity of the computation graph of the intelligent driving neural network model are directly reduced, thereby significantly reducing runtime scheduling overhead and kernel startup times. Simultaneously, the fusion computation within the composite operator eliminates repeated storage and retrieval of intermediate results, optimizing memory access efficiency and reducing latency and power consumption. This transformation, while ensuring strict functional equivalence of the intelligent driving neural network model, brings substantial performance improvements to its deployment on automotive chips, enabling it to better meet the stringent requirements for real-time performance, energy efficiency, and reliability in different scenarios such as highways, urban areas, and parking.

[0054] The compilation method for intelligent driving neural network models provided in this disclosure first identifies the driving scenarios required for the deployment of the intelligent driving neural network model. Then, a target operator fusion strategy corresponding to the driving scenario is selected from the operator fusion strategy library. Following the target operator fusion strategy, composite operators corresponding to multiple basic operators in the computational graph of the intelligent driving neural network model are constructed. Finally, the computational graph of the intelligent driving neural network model is compiled using these composite operators to obtain an optimized intelligent driving neural network model. In this way, by introducing a driving scenario-aware operator fusion strategy selection mechanism, the compiler can differentiate between computational efficiency and model accuracy for different driving environments, thereby achieving a balance between performance and accuracy in the compilation of the intelligent driving neural network model while ensuring driving safety and perception reliability.

[0055] In some embodiments, combined with Figure 2 As shown, based on the target operator fusion strategy, a composite operator corresponding to multiple basic operators in the computation graph of the intelligent driving neural network model is constructed, including the following steps: S201, select any node in the computation graph of the intelligent driving neural network model as a candidate root node.

[0056] S202, based on the operator rule base corresponding to the target operator fusion strategy, traverse the graph structure of the intelligent driving neural network model computation graph in reverse from the candidate root node, and take the graph structure that conforms to the subgraph structure in the operator rule base as the candidate subgraph.

[0057] Optionally, a graph structure that conforms to the subgraph structure in the operator rule base is used as a candidate subgraph, including one or more of the following methods: matching the graph structure of the computation graph of the intelligent driving neural network model with the subgraph structure in the operator rule base, and using the successfully matched graph structure as a candidate subgraph; in the case of a high-speed driving scenario, using all candidate subgraphs as candidate subgraphs; in the case of an urban driving scenario, using candidate subgraphs that meet the subgraph cost-effectiveness condition as candidate subgraphs; in the case of a parking driving scenario, using candidate subgraphs that meet the subgraph accuracy condition as candidate subgraphs.

[0058] Here, the cost-effectiveness condition refers to the quantitative decision-making standard for evaluating whether a candidate subgraph is worth adopting in an urban scenario; the accuracy condition is the deterministic quantitative decision-making standard for evaluating whether a candidate subgraph is allowed to be adopted in a parking scenario.

[0059] In some possible implementations, the candidate subgraph is determined to meet the subgraph cost-effectiveness condition as follows: determine the cost-effectiveness score corresponding to the candidate subgraph; if the cost-effectiveness score is greater than the cost-effectiveness score threshold, the candidate subgraph is determined to meet the subgraph cost-effectiveness condition.

[0060] In practical applications, the cost-effectiveness score can be calculated using the following formula: Score=w1 PerfGain w2 Risk w3 Cost Wherein, Score is the cost-effectiveness score, PerfGain is the normalized performance gain, PerfGain is the normalized accuracy risk, PerfGain is the normalized resource cost, and w1, w2 and w3 are the weighting coefficients.

[0061] The subgraph cost-performance condition, through a quantified cost-benefit model and conflict resolution mechanism, ensures that the compiler automatically and reliably selects those solutions that can bring significant performance improvements and controllable accuracy risks from among many possible fusion schemes, which meets the fundamental requirement of urban scenarios to balance efficiency and reliability.

[0062] In some possible implementations, the candidate subgraph is determined to satisfy the subgraph accuracy condition as follows: if the candidate subgraph passes the mathematical equivalence test and the error test, the candidate subgraph is determined to satisfy the subgraph accuracy condition.

[0063] Among them, mathematical equivalence testing refers to the systematic verification process, within an ideal mathematical domain (such as the real number domain), verifying that the candidate subgraph and the original subgraph are mathematically identical through rigorous symbolic derivation and formal proof. Error testing refers to the verification process, in a real computing environment, using numerical computation and statistical methods to quantitatively evaluate the differences in output between the candidate subgraph and the original subgraph when executed on actual hardware.

[0064] The dual verification mechanism in the subgraph accuracy condition ensures the absolute reliability and determinism of the fusion operation in the parking scenario, providing the highest level of protection for safety-critical computation.

[0065] In some possible implementations, the graph structure of the intelligent driving neural network model computation graph is matched with the subgraph structure in the operator rule base, and the successfully matched graph structure is used as a candidate subgraph. This includes: determining the target subgraph in the subgraph structure of the operator rule base that has the same output node type as the candidate root node; recursively matching the predecessor node according to the topology of the target subgraph to obtain the matching node in the computation graph of the intelligent driving neural network model; and constructing the candidate subgraph using all the matching nodes.

[0066] In practical applications, when the compiler traverses to the candidate root node in the computation graph of the intelligent driving neural network model, it first uses the operator type (such as ReLU, Add) of the candidate root node as the key to query all target subgraphs in the operator rule base that have that type as the output node. For example, if the candidate root node is of type ReLU, then all patterns in the operator rule base that have ReLU as the last operator (such as Conv-BN-ReLU, Add-ReLU) will be extracted as target subgraphs.

[0067] For each target subgraph, a recursive matching process is initiated, using the candidate root node as the starting point for mapping the pattern output node, and then traversing in reverse order of the target subgraph's topology (i.e., from the output node to the input node). The process checks if the current actual node and the node in the target subgraph are of the same type; for each input edge of the current node in the target subgraph, the corresponding input node in the actual computation graph is found, and matching is performed recursively; if the target subgraph contains multiple input branches (e.g., the Add operator has two inputs), the predecessor subgraphs of both branches must be matched separately.

[0068] If all nodes of a target subgraph are successfully mapped to nodes in the actual computational graph, then these actual nodes and the edges between them are collected to form a complete candidate subgraph. The candidate subgraph is topologically isomorphic to the target subgraph, and its node attributes satisfy all constraints.

[0069] To illustrate this, let's take a specific example: Suppose there is a target subgraph in the operator rule base as: Conv → BN → ReLU (output node is ReLU). The current candidate root node is ReLU2 in the computation graph of the intelligent driving neural network model. Matching begins by mapping the target subgraph ReLU to ReLU2; then, the input of ReLU is matched in reverse. The input of ReLU in the target subgraph is BN. We check if the input node of ReLU2 is of type BN (assuming it's BN2) and if the attribute satisfies the constraint (e.g., eps=0.001). If so, the target subgraph BN is mapped to BN2. We continue matching the input of BN in reverse. The input of BN in the target subgraph is Conv. We check if the input node of BN2 is of type Conv (assuming it's Conv2) and if the attribute satisfies the constraint (e.g., groups=1). If so, the target subgraph Conv is mapped to Conv2. The target subgraph Conv is an input node with no predecessor, so the recursion terminates. Thus, the node set {Conv2, BN2, ReLU2} is successfully matched, and this is used to construct the candidate subgraph.

[0070] The technical solution for determining candidate subgraphs provides accurate and reliable candidate subgraph recognition capabilities for scene-adaptive operator fusion. Moreover, its efficient matching mechanism itself constitutes an important accelerator for the compilation and optimization process, and is a key technical support for the efficient and safe operation of the entire intelligent driving model compilation method.

[0071] S203, constructing composite operators using candidate subgraphs.

[0072] Optionally, constructing a composite operator using candidate subgraphs includes: performing composite verification on the candidate subgraphs; wherein the composite verification includes semantic verification and equivalence verification; and constructing a composite node operator using the candidate subgraphs that pass the composite verification.

[0073] Semantic verification performs a structural security check on the candidate subgraphs to ensure that fusion is feasible at the definition level. Equivalence verification proves the mathematical correctness of the candidate subgraphs to ensure that the computational results are exactly the same in the ideal mathematical domain before and after fusion.

[0074] In some possible implementations, semantic verification includes checking whether the node attributes of each node in the candidate subgraph satisfy the constraints defined in the target operator fusion strategy.

[0075] Each fusion rule comes with a set of scenario-specific constraints, defined in the operator rule library.

[0076] In practical applications, each node of the candidate subgraph is traversed, and all its runtime attributes (such as kernel_size, stride, padding, etc. of convolution) are extracted. The extracted attributes are then compared with the constraints defined under the current operator fusion strategy (highway / city / parking). If all node attributes fully satisfy all constraints, the semantic verification passes; otherwise, it fails.

[0077] In some possible implementations, equivalence verification includes: deriving the mathematical expression of the original computation sequence of the candidate subgraph and the computation formula of the fused composite operator; and comparing whether the mathematical expression and the computation formula are mathematically equivalent.

[0078] In practical applications, the original computation sequence of the candidate subgraph is expanded into a mathematical expression according to the operator order, and the computation formula of the fused composite operator is generated based on the mathematical proof. The mathematical expression and the computation formula are then compared to see if they are equal in mathematical definition.

[0079] Semantic verification ensures that only subgraphs that meet strict structural constraints are fused, avoiding incorrect fusion of special variants (such as dilated convolution and grouped convolution). Equivalence verification theoretically proves the equivalence of the fusion, completely eliminating model behavior deviations caused by mathematical transformation errors. The dual verification mechanism provides provable safety assurance for autonomous driving.

[0080] By traversing backwards from the output nodes and matching predefined operator rules, fusionable subgraph structures are accurately identified. After rigorous semantic and equivalence verification, composite operators are constructed, significantly reducing the number of nodes in the computation graph, memory access overhead, and kernel startup times. Through intelligent matching and verification of driver-scene-adaptive operator fusion strategies, a dynamic and optimal trade-off between performance and accuracy is achieved in compilation optimization: deep fusion can be performed in scenarios requiring extreme performance (such as high-speed scenarios) to maximize latency reduction, while in scenarios with stringent accuracy and safety requirements (such as parking scenarios), it can automatically switch to conservative fusion. Thus, in complex and ever-changing driving environments, it significantly improves the inference efficiency of intelligent driving neural network models while ensuring the reliability and safety of the intelligent driving system.

[0081] In some embodiments, combined with Figure 3 As shown, the computational graph of the intelligent driving neural network model is compiled using composite operators to obtain the compiled intelligent driving neural network model, including the following steps: S301, remove the original node corresponding to the composite operator in the computation graph of the intelligent driving neural network model, and create a new composite operator node.

[0082] In practical applications, before removing nodes, the input and output boundaries of the candidate subgraph are analyzed. The input boundary consists of all edges pointing from external nodes to internal nodes of the subgraph, with each input boundary edge corresponding to an input port of the composite operator. The output boundary consists of all edges pointing from internal nodes to external nodes of the subgraph, with each output boundary edge corresponding to an output port of the composite operator. All basic operator nodes contained in the candidate subgraph are removed from the node list of the intelligent driving neural network model's computational graph, and all internal connection edges between nodes are also removed, but the records of the input and output boundaries are retained.

[0083] According to the fusion rules, a new node is created, whose operator type is a predefined composite operator. The new weights, biases, and other parameters obtained by the fusion operator through fusion calculation are attached as attributes to this node, ensuring that it encapsulates all computational semantics of the original subgraph.

[0084] S302 connects the input of the new composite operator node to the external input node of the original node subgraph.

[0085] Traverse all input boundary edges and change the target node of each edge from the original input node inside the subgraph to the corresponding input port of the new composite operator node. If the subgraph has multiple inputs (such as the Add operation of the residual block having two inputs), the correct mapping should be performed according to the input order defined by the fusion rules.

[0086] S303 connects the output of the new composite operator node to the successor node to which the output node of the original node subgraph is connected.

[0087] Traverse all output boundary edges, changing the source node of each edge from the original output node inside the subgraph to the output port of the new composite operator node. If the subgraph has multiple outputs, the composite operator node must provide the corresponding number of output ports, ensuring that the connection relationship with the outside remains unchanged. Update the topological sorting of the computation graph of the intelligent driving neural network model, and update the intermediate representation information (IR) required for subsequent compilation processes.

[0088] By merging multiple nodes into one, the number of nodes in the computation graph of the intelligent driving neural network model is significantly reduced (e.g., from 3 to 1), lowering the management overhead of the compiler and runtime scheduler, and improving compilation and execution efficiency. Composite operators can invoke a single kernel deeply optimized for specific hardware (such as an NPU), replacing the sequential startup of multiple small kernels, reducing kernel startup overhead, improving the utilization of computing units, and thus significantly reducing inference latency.

[0089] Combination Figure 4 As shown, the compilation method for the intelligent driving neural network model includes the following steps: S401 identifies the driving scenarios required for the deployment of the intelligent driving neural network model.

[0090] S402, Select the target operator fusion strategy corresponding to the driving scenario from the operator fusion strategy library.

[0091] S403, select any node in the computation graph of the intelligent driving neural network model as a candidate root node.

[0092] S404: Based on the operator rule base corresponding to the target operator fusion strategy, the graph structure of the intelligent driving neural network model computation graph is traversed backward from the candidate root node, and the graph structure that conforms to the subgraph structure in the operator rule base is taken as the candidate subgraph.

[0093] S405, constructing composite operators using candidate subgraphs.

[0094] S406 Remove the original node corresponding to the composite operator in the computation graph of the intelligent driving neural network model and create a new composite operator node.

[0095] S407, connect the input of the new composite operator node to the external input node of the original node subgraph, and connect the output of the new composite operator node to the successor node connected to the output node of the original node subgraph.

[0096] S408, obtain the compiled intelligent driving neural network model.

[0097] By identifying the driving scenarios in which the intelligent driving neural network model is deployed, the corresponding operator fusion strategy is selected to achieve dynamic adaptation of the optimization target. In the computation graph of the intelligent driving neural network model, the predefined subgraph structure in the operator rule library is matched backward from the subsequent node to locate the fusionable subgraph. After verification, it is encapsulated into a composite operator, and the original multi-node subgraph is reconstructed into a single node through node replacement and edge reconnection. Finally, the optimized intelligent driving neural network model computation graph is generated, thereby reconstructing the computation structure of the intelligent driving neural network model under the condition of ensuring functional equivalence.

[0098] The compilation method for intelligent driving neural network models provided in this embodiment significantly improves the computational efficiency, resource utilization, and scenario applicability of the compiled intelligent driving neural network model through a driving scenario-adaptive operator fusion strategy and precise graph structure rewriting. It effectively reduces the computational graph complexity and runtime scheduling overhead of the intelligent driving neural network model, optimizes memory access patterns, thereby shortening inference latency and reducing power consumption. Simultaneously, the ability to differentiate performance and accuracy according to different driving scenarios ensures that the optimized intelligent driving neural network model, while improving efficiency, still meets the stringent requirements of safety, real-time performance, and determinism in various driving scenarios, maximizing the overall benefits of model deployment in in-vehicle environments.

[0099] The following is a specific example, illustrated in detail with diagrams, illustrating the complete implementation process of constructing a composite operator: The compiler identifies the driving scenario required for the deployment of the intelligent driving neural network model as an urban scenario, and automatically selects the second operator fusion strategy. The operator rule base of the second operator fusion strategy includes the following typical subgraph patterns: Pattern A: Conv → BatchNorm → ReLU, Pattern B: Conv → BatchNorm, Pattern C: BatchNorm → ReLU, Pattern D: Conv → BatchNorm → Add, Pattern E: Conv → BatchNorm → Add → ReLU.

[0100] Combined with appendix Figure 5A As shown, the original intelligent driving neural network model computation graph contains 8 nodes (excluding Input / Output): Conv1, BatchNorm1, ReLU1, Conv2, BatchNorm2, Identity, Add, and ReLU2. The compiler performs a reverse topological traversal starting from the output node, attempting matches sequentially, as follows: In the first round, ReLU2 is used as the candidate root node: The matching process attempts to match pattern A (Conv → BatchNorm → ReLU), moving backwards from ReLU2 to Add. Add does not conform to the BatchNorm type, so the match fails. The process then attempts pattern C (BatchNorm → ReLU), which also does not conform to the BatchNorm type, resulting in another failure. Finally, the process attempts pattern E (Conv → BatchNorm → Add → ReLU), which conforms to the BatchNorm type, but Add has two inputs that need to be matched separately: the main path is Conv2→BatchNorm2→Add (conforming to Conv→BatchNorm), and the jump path is Identity→Add (a direct connection, not matching Conv→BatchNorm), leading to another failure.

[0101] The second round uses Add as the candidate root node: It attempts pattern D (Conv → BatchNorm → Add), working backwards from Add. There are two input branches: Branch 1 is Conv2 → BatchNorm2 → Add (matching Conv → BatchNorm). If the match is successful, a candidate node is formed. Figure 1 That is, Conv2→BatchNorm2→Add; branch 2 is Identity→Add (direct connection), matching failed.

[0102] In the third round, BatchNorm2 is selected as the candidate root node: Pattern B (Conv → BatchNorm) is attempted, matching Conv2 → BatchNorm2. If the match is successful, candidate nodes are formed. Figure 2 That is, Conv2→BatchNorm2; Try mode A (Conv →BatchNorm → ReLU), but the match fails because the ReLU suffix is ​​required.

[0103] In the fourth round, ReLU1 is used as the candidate root node: Pattern A (Conv → BatchNorm → ReLU) is attempted, matching Conv1→BatchNorm1→ReLU1. If a match is successful, candidate nodes are formed. Figure 3 That is, Conv1→BatchNorm1→ReLU1.

[0104] Fifth round: Continue matching using BatchNorm1, Conv1, etc. as candidate root nodes... Candidate Figure 1 Candidate Figure 2 and candidate Figure 3 In the process, candidate sub-subs that pass semantic and security verification are selected. Figure 2 and candidate Figure 3 Construct composite operators. For candidate operators...Figure 3 Remove Conv1, BatchNorm1, and ReLU1 nodes, create a composite operator node Fused_CBR1, connect Input → Fused_CBR1 (the original Conv1 input), and connect Fused_CBR1 → Conv2 (the original ReLU1 output to Conv2 edge); for candidate nodes... Figure 2 Remove Conv2 and BatchNorm2 nodes, create a composite operator node Fused_CB2, connect Fused_CBR1 → Fused_CB2 (original Conv2 input), and connect Fused_CB2 → Add (original BatchNorm2 output to Add edge).

[0105] Combined with appendix Figure 5B As shown, the encoded intelligent driving neural network model computation graph is formed, which contains 5 nodes (excluding Input / Output): Fused_CBR1, Fused_CBR2, Identity, Add, and ReLU2.

[0106] The compiler used in the above-mentioned intelligent driving neural network model successfully transformed the compilation optimization of scene perception from theory into executable engineering practice, achieving a significant performance improvement of the intelligent driving system while ensuring the required accuracy in urban scenarios.

[0107] In this embodiment, an optimal balance between performance and safety in urban driving environments is achieved through driving scene recognition and adaptive operator fusion strategy selection. The fusionable subgraph is accurately located via reverse traversal and rule matching, and a composite operator is constructed after rigorous verification. Multiple basic operators are then safely replaced with a single composite operator through graph rewriting. The compiled intelligent driving neural network model shows significant optimizations in computational graph complexity, memory access efficiency, and kernel scheduling overhead. This maintains a balance between accuracy and performance in urban scenarios under limited onboard computing power, comprehensively improving the real-time performance, energy efficiency, and reliability of the intelligent driving system in urban driving environments.

[0108] Combination Figure 6As shown in the figure, this disclosure provides a compilation device 600 for an intelligent driving neural network model, including an identification module 610, a conversion module 620, and a compilation module 630. The identification module 610 is configured to identify the driving scenario required for the deployment of the intelligent driving neural network model and select a target operator fusion strategy corresponding to the driving scenario from an operator fusion strategy library. The conversion module 620 is communicatively connected to the identification module 610 and is configured to construct composite operators corresponding to multiple basic operators in the computational graph of the intelligent driving neural network model according to the target operator fusion strategy. The compilation module 630 is communicatively connected to the conversion module 620 and is configured to compile the computational graph of the intelligent driving neural network model using the composite operators to obtain the compiled intelligent driving neural network model.

[0109] The compilation apparatus for intelligent driving neural network models provided in this disclosure first identifies the driving scenario to be deployed in the intelligent driving neural network model when compiling the model. Then, it selects a target operator fusion strategy corresponding to the driving scenario from the operator fusion strategy library. Next, based on the target operator fusion strategy, it constructs composite operators corresponding to multiple basic operators in the computational graph of the intelligent driving neural network model. Finally, it uses these composite operators to compile the computational graph of the intelligent driving neural network model, resulting in an optimized intelligent driving neural network model. In this way, by introducing a driving scenario-aware operator fusion strategy selection mechanism, the compiler can differentiate between computational efficiency and model accuracy for different driving environments, thereby achieving a balance between performance and accuracy in the compilation of the intelligent driving neural network model while ensuring driving safety and perception reliability.

[0110] Combination Figure 7 As shown, this embodiment of the disclosure provides an electronic device (e.g., a computer, controller, etc.) 700, which includes a memory 701, a processor 702, a communication interface 703, and a bus 704. The memory 701, processor 702, and communication interface 703 are interconnected via the bus 704.

[0111] The memory 701 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).

[0112] The memory 701 can store a program. When the program stored in the memory 701 is executed by the processor 702, the processor 702 and the communication interface 703 are used to execute the various steps of the compilation method for the intelligent driving neural network model of the present application embodiments.

[0113] Processor 702 is a circuit with signal processing capabilities. In one implementation, processor 702 can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, processor 702 can implement certain functions through the logical relationships of hardware circuits. These logical relationships of hardware circuits are fixed or reconfigurable. For example, processor 702 can be a hardware circuit implemented as an ASIC or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above modules. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as a type of ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), or deep learning processing unit (DPU). The processor 702 is used to execute related programs to implement the functions required by the units in the compilation apparatus for the intelligent driving neural network model in the embodiments of this application, or to execute the compilation method for the intelligent driving neural network model in the method embodiments of this application.

[0114] As can be seen, each module in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor types.

[0115] Furthermore, the modules in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these modules are integrated together as a system-on-a-chip (SOC). The SOC may include at least one processor for implementing any of the above methods or for implementing the functions of the modules of the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0116] The communication interface 703 uses transceiver devices, such as, but not limited to, transceivers, to enable communication between the device 700 and other devices or communication networks. For example, data can be acquired through the communication interface 703.

[0117] Bus 704 may include a pathway for transmitting information between various components of device 700 (e.g., memory 701, processor 702, communication interface 703).

[0118] It should be noted that, although Figure 7 The illustrated compilation device 700 for intelligent driving neural network models only shows the memory, processor, and communication interface. However, those skilled in the art should understand that in specific implementations, device 700 also includes other devices necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that device 700 may also include hardware devices for implementing other additional functions. Moreover, those skilled in the art should understand that device 700 may only include the devices necessary for implementing the embodiments of this application, and may not necessarily include... Figure 7 All the devices shown.

[0119] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps of any of the above methods.

[0120] This application also provides a computer program product containing instructions. When the computer program product is run on a computer or processor, it causes the computer or processor to perform one or more steps of any of the methods described above.

[0121] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0122] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0123] The foregoing description and accompanying drawings fully illustrate embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included or substituted for parts and features of other embodiments. The scope of the embodiments of this disclosure includes the entire scope of the claims and all available equivalents of the claims. While the terms “first,” “second,” etc., may be used in this application to describe elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be called a second element without changing the meaning of the description, and similarly, a second element may be called a first element, provided that all occurrences of “first element” are consistently renamed and all occurrences of “second element” are consistently renamed. First and second elements are both elements, but may not be the same element. Moreover, the terminology used in this application is only for describing embodiments and is not intended to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. Similarly, the term “and / or” as used herein means including one or more of the associated listed any and all possible combinations. Additionally, when used herein, the terms “comprise” and its variations “comprises” and / or “comprising” refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase “comprising an…” does not exclude the presence of additional identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0124] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0125] The methods and products disclosed in the embodiments herein (including but not limited to devices and equipment) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A compilation method for intelligent driving neural network models, characterized in that, include: Identify the driving scenarios required for the deployment of intelligent driving neural network models; Select the target operator fusion strategy corresponding to the driving scenario from the operator fusion strategy library; Based on the target operator fusion strategy, construct composite operators corresponding to multiple basic operators in the computation graph of the intelligent driving neural network model; The computation graph of the intelligent driving neural network model is compiled using composite operators to obtain the compiled intelligent driving neural network model.

2. The compilation method according to claim 1, characterized in that, Select the target operator fusion strategy corresponding to the driving scenario from the operator fusion strategy library, including: In the case of a high-speed driving scenario, the target operator fusion strategy is determined to be the first operator fusion strategy; When the driving scenario is an urban scenario, the target operator fusion strategy is determined to be the second operator fusion strategy; wherein, the operator fusion degree in the second operator fusion strategy is less than the operator fusion degree in the first operator fusion strategy; When the driving scenario is a parking scenario, the target operator fusion strategy is determined to be the third operator fusion strategy; wherein, the operator fusion degree in the third operator fusion strategy is less than the operator fusion degree in the second operator fusion strategy.

3. The compilation method according to claim 1, characterized in that, Based on the target operator fusion strategy, composite operators corresponding to multiple basic operators in the computation graph of the intelligent driving neural network model are constructed, including: Select any node in the computation graph of the intelligent driving neural network model as a candidate root node; Based on the operator rule base corresponding to the target operator fusion strategy, the graph structure of the intelligent driving neural network model computation graph is traversed backward from the candidate root node, and the graph structure that conforms to the subgraph structure in the operator rule base is taken as the candidate subgraph. Construct composite operators using candidate subgraphs.

4. The compilation method according to claim 3, characterized in that, Graph structures that conform to the subgraph structure in the operator rule base are considered as candidate subgraphs, including one or more of the following methods: The graph structure of the computation graph of the intelligent driving neural network model is matched with the subgraph structure in the operator rule base, and the successfully matched graph structure is taken as the candidate subgraph. In the case of a high-speed driving scenario, all candidate subgraphs will be used as candidate subgraphs. In the case of urban driving scenarios, candidate subgraphs that meet the cost-effectiveness criteria will be selected as candidate subgraphs. When the driving scenario is a parking scenario, the candidate subgraphs that meet the subgraph accuracy conditions are selected as candidate subgraphs.

5. The compilation method according to claim 4, characterized in that, The graph structure of the computation graph of the intelligent driving neural network model is matched with the subgraph structures in the operator rule base, and the successfully matched graph structures are used as candidate subgraphs, including: Determine the target subgraph in the subgraph structure of the operator rule base that has the same output node type as the candidate root node; By recursively matching the predecessor nodes in reverse according to the topology of the target subgraph, the matching nodes in the computation graph of the intelligent driving neural network model are obtained. Construct a candidate subgraph using all matching nodes.

6. The compilation method according to claim 3, characterized in that, Constructing composite operators using candidate subgraphs includes: The candidate subgraphs are subjected to composite verification, which includes semantic verification and equivalence verification. Composite node operators are constructed using candidate subgraphs that pass composite verification.

7. The compilation method according to claim 6, characterized in that, Semantic verification includes checking whether the node attributes of each node in the candidate subgraph satisfy the constraints defined in the target operator fusion strategy. Equivalence verification includes: deriving the mathematical expression of the original computation sequence of the candidate subgraph and the computation formula of the composite operator after fusion; and comparing whether the mathematical expression and the computation formula are equal in mathematical definition.

8. The compilation method according to any one of claims 1 to 7, characterized in that, The computational graph of the intelligent driving neural network model is compiled using composite operators, including: Remove the original nodes corresponding to the composite operators in the computation graph of the intelligent driving neural network model and create new composite operator nodes; Connect the input of the new composite operator node to the external input node of the original node subgraph, and connect the output of the new composite operator node to the successor node to which the output node of the original node subgraph is connected.

9. A compilation device for an intelligent driving neural network model, comprising: The recognition module is configured to recognize the driving scenarios required for the deployment of the intelligent driving neural network model, and select the target operator fusion strategy corresponding to the driving scenario from the operator fusion strategy library; The conversion module, which communicates with the recognition module, is configured to construct composite operators corresponding to multiple basic operators in the calculation graph of the intelligent driving neural network model according to the target operator fusion strategy. The compilation module, which communicates with the conversion module, is configured to compile the computation graph of the intelligent driving neural network model using composite operators to obtain the compiled intelligent driving neural network model.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the compilation method for the intelligent driving neural network model as described in any one of claims 1 to 8.

11. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the compilation method for an intelligent driving neural network model as described in any one of claims 1 to 8.

Images