Methods, systems, devices, and computer program products that do not support operator adaptation for deploying point cloud semantic segmentation models for intelligent driving
By adopting a systematic method for adapting operators that do not support them, risky operators are automatically identified and adapted in a hierarchical and classified manner. This solves the problems of low efficiency and lack of accuracy in operator adaptation during the deployment of the intelligent driving point cloud semantic segmentation model, and achieves efficient and reliable model deployment.
Patent Information
- Application Number
- CN202611619962.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-20
- Publication Date
- 2026-11-17
AI Technical Summary
The existing intelligent driving point cloud semantic segmentation model deployment suffers from a lack of systematic process for operator adaptation, reliance on manual patching, missing accuracy verification loop, low deployment efficiency, and poor reusability of adaptation results.
It provides a systematic method for adapting operators that do not support them. It identifies risky operators through computation graph analysis, performs hierarchical classification and adaptation, and constructs a hierarchical verification closed loop at the subgraph level and the whole network level to ensure the consistency of model accuracy and deployment quality.
It achieves automated and rapid identification of unsupported operators, improves problem localization efficiency, reduces invasiveness to the inference backend, ensures the model's deployment portability and engineering deployment efficiency, and meets the high reliability and low latency requirements of intelligent driving vehicle platforms.
Smart Images

Figure CN123114494A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent driving perception model deployment technology, specifically to a method, system, device, and computer program product that does not support operator adaptation for intelligent driving point cloud semantic segmentation model deployment. Background Technology
[0002] With the large-scale deployment of intelligent driving technology, LiDAR point cloud semantic segmentation has become a core technology for environmental perception. It enables point-level semantic recognition of targets such as roads, vehicles, pedestrians, and traffic facilities, providing high-precision environmental input for decision-making and planning modules. These deep learning models are typically trained in GPU training frameworks and then converted into deployment models adapted to in-vehicle inference engines through intermediate representation formats to meet the low latency, high throughput, and stable operation requirements of automotive embedded platforms.
[0003] In practical engineering deployments, a semantic gap exists between the training framework, intermediate representation standards, and the vehicle-side inference engine: the training framework provides rich operator implementations to balance flexibility and development efficiency, but some custom operations and high-level encapsulated operators cannot be directly mapped by the intermediate representation format, nor are they natively supported by hardware-accelerated inference backends. Compared to 2D vision models such as image classification and object detection, point cloud semantic segmentation models involve a large number of dynamic point inputs, index mapping, spatial aggregation, reduction statistics, interpolation post-processing, and other special operations. The operators are more complex in form and more flexible in shape constraints, resulting in a significantly higher probability of compatibility issues.
[0004] Current mainstream operator adaptation solutions largely rely on passive, experience-driven patching: when model export errors occur, code is checked line by line and the model structure is modified; when inference construction fails, custom plugins are temporarily written; and when accuracy anomalies occur, problems are manually compared layer by layer to pinpoint the issue. This fragmented approach has several drawbacks: First, problem localization lacks a systematic process, making it difficult to distinguish the root causes of problems in the export, intermediate representation, backend construction, and runtime stages, resulting in low troubleshooting efficiency. Second, there is a lack of unified rules for classifying and adapting unsupported operators, which can easily lead to premature introduction of highly invasive backend plugin development or large-scale unnecessary modifications to the model, damaging its original semantics. Third, there is a lack of standardized verification loops, often resulting in situations where some operators are runnable but the overall network accuracy is abnormal, making it difficult to guarantee deployment quality. Fourth, the reusability of adaptation results is poor, requiring repeated adaptation work when deploying similar models, leading to high engineering maintenance costs. Therefore, a systematic method for layered adaptation of unsupported operators is urgently needed to transform the deployment process of point cloud semantic segmentation models from passive error reporting and patching to a fully controllable solution involving proactive identification, classification, adaptation, and verification. Summary of the Invention
[0005] To address the aforementioned technical shortcomings, this invention provides a method, system, device, and computer program product for deploying intelligent driving point cloud semantic segmentation models that does not support operator adaptation. It aims to solve the technical problems of existing intelligent driving point cloud model deployment relying on manual repair, lack of a systematic process for operator adaptation, missing accuracy verification closed loop, and low deployment efficiency.
[0006] This invention is achieved through the following technical solution: A method for adapting to unsupported operators in the deployment of point cloud semantic segmentation models for intelligent driving is provided. The method includes the following steps: Step S10: Obtain the trained point cloud semantic segmentation model and a representative point cloud input sample set, perform model export and benchmark inference, generate an intermediate representation model with validation benchmark data, and output the intermediate representation model with benchmark data. Step S20: Perform computation graph analysis based on the intermediate representation model with benchmark data output in step S10, extract operator nodes, tensor attributes and topological connectivity, identify unsupported operators and high-risk operators by combining the operator support list of the target inference backend, and output the risk operator set, node topology information and supporting benchmark data. Step S30: Based on the risk operator set and node topology information output in step S20, classify them hierarchically according to operator functional attributes, input and output constraints and deployment risk levels, determine the adaptation category corresponding to each risk operator, and output the classification results, adaptation strategy mapping table and supporting benchmark data. Step S40: Based on the classification results and adaptation strategy mapping table output in step S30, perform mapping supplementation, equivalent subgraph replacement, local rewriting of computation graph or backend extension adaptation on each risk operator to generate an adapted intermediate representation model with corresponding verification benchmarks. Step S50: Perform subgraph-level and whole-network-level hierarchical verification on the adapted intermediate representation model output from step S40. After the verification is passed, generate the vehicle-side inference deployment model.
[0007] Preferably, the step S10, which generates an intermediate representation model with accompanying verification benchmark data and outputs the intermediate representation model with benchmark data, specifically includes: Step S101: Obtain the network structure, weight parameters and configuration file of the point cloud semantic segmentation model, and collect point cloud samples covering different point scales, scene densities and obstacle distributions to construct a benchmark input sample set; Step S102: Based on the export interface of the training framework, convert the training-state model into an inference-state computation graph, remove the training-specific nodes, perform constant folding optimization, and generate a standard intermediate representation model. Step S103: Using the benchmark input sample set as input, run the original training model and the initial intermediate representation model respectively, record the output tensors of each layer as verification benchmark data, and integrate them into the supporting information of the intermediate representation model.
[0008] Preferably, the step S20, which outputs the risk operator set, node topology information, and supporting baseline data, specifically includes: Step S201: Traverse the computation graph nodes of the intermediate representation model, read the type name, input and output tensor shape, data type and topological connection relationship of each operator, and construct a complete node attribute table; Step S202: Match the operators in the node attribute table with the list of supported operators in the target inference backend one by one, and calculate the single operator compatibility. The calculation formula is as follows:
[0009] in For the first The compatibility of the operators Score for operator type matching. For tensor shape constraint matching score, Scoring is given based on data type matching. , and These are the weighting coefficients for the corresponding items; Step S203: Mark operators with compatibility below a preset threshold as unsupported operators, and mark operators with compatibility that meet the standard but have risks of dynamic shape or numerical divergence as high-risk operators. Summarize them to form a set of risk operators, and output them together with node topology information and supporting benchmark data.
[0010] Preferably, the step S30, which outputs the classification results, the adaptation strategy mapping table, and the corresponding benchmark data, specifically includes: Step S301: Based on the operator's functional type, computational semantic complexity, and backend modification intrusion, risk operators are divided into five adaptation categories: directly mappable, equivalently rewriteable, graph structure sensitive, numerical sensitive, and backend extension. Step S302: According to the preset principle, match the corresponding basic adaptation strategy for each type of operator and generate an adaptation strategy mapping table, such as "prioritize intermediate representation layer adaptation and backend adaptation as a supplement"; Step S303: Combine the node topology information to mark the range of the associated subgraph of each risk operator, determine the boundary of the adaptation operation, and output the classification results, adaptation strategy mapping table and supporting benchmark data together.
[0011] Preferably, in step S301, a local numerical protection strategy is implemented during the adaptation process for numerically sensitive operators: The numerical risk propagation path is located from the node topology information output in step S30. Local high-precision calculation, outlier clamping, or replacement with numerically stable equivalent operators are used for the risk area. At the same time, the protection strategy is verified through subgraph to confirm that it does not affect the overall network segmentation accuracy.
[0012] Preferably, step S40, which involves performing mapping supplementation, equivalent subgraph replacement, local rewriting of the computation graph, or backend extension adaptation on each risk operator to generate an adapted intermediate representation model with corresponding verification benchmarks, specifically includes: Step S401: Extract the compatibility, subgraph influence range coefficient, and accuracy risk level coefficient of each risk operator from the adaptation strategy mapping table, and calculate the adaptation priority of each risk operator. The calculation formula is as follows:
[0013] in For the first The adaptation priority of each operator For operator compatibility, This is the subplot influence range coefficient. For the accuracy risk level coefficient, , , These are the weighting coefficients; Step S402: According to the order of priority from high to low, supplement the derived mapping relationship for operators that can be directly mapped, replace the operators that can be equivalently rewritten with equivalent subgraphs composed of basic operators, and perform local computation graph rewriting for operators that are sensitive to graph structure. Step S403: For backend extended operators that cannot be adapted through the intermediate representation layer, the extended implementation module is connected to the inference backend to complete the adaptation and optimization of all risk operators, generate the adapted intermediate representation model, and synchronously associate it with the corresponding verification benchmark data.
[0014] Preferably, step S50, which involves performing subgraph-level and whole-network-level hierarchical verification on the adapted intermediate representation model output in step S40, and generating the vehicle-side inference deployment model after successful verification, specifically includes: Step S501: Extract the local subgraph corresponding to each risk operator, extract the corresponding input tensor from the validation benchmark attached to the adapted model, compare the error of the subgraph output before and after adaptation, and complete the subgraph-level validation. Step S502: Perform end-to-end inference on the complete model. Using the original model output in the validation benchmark as a reference, compare the point cloud semantic segmentation output of the original model and the adapted model, calculate the class consistency rate and the intersection-union ratio of key classes, and complete the whole network-level validation. Step S503: If the verification is successful, a deployment model that can run on the vehicle-side inference engine is generated. If the verification fails, the process returns to the corresponding stage for re-adaptation.
[0015] Furthermore, to achieve the above objectives, this invention also proposes a non-operator adaptation system for deploying intelligent driving point cloud semantic segmentation models. This non-operator adaptation system for deploying intelligent driving point cloud semantic segmentation models includes: Model export module: used to obtain the trained point cloud semantic segmentation model and representative point cloud input sample set, perform model export and benchmark inference, generate an intermediate representation model with verification benchmark data, and output the intermediate representation model with benchmark data; Operator identification module: It is used to perform computation graph analysis based on the intermediate representation model with benchmark data output by the model export module, extract operator nodes, tensor attributes and topological connectivity, identify unsupported operators and high-risk operators by combining the operator support list of the target inference backend, and output the risk operator set, node topology information and supporting benchmark data. The classification and mapping module is used to classify risk operators according to their functional attributes, input and output constraints and deployment risk levels based on the risk operator set and node topology information output by the operator identification module. It determines the adaptation category corresponding to each risk operator and outputs the classification results, adaptation strategy mapping table and supporting benchmark data. The adaptation execution module is used to perform mapping supplementation, equivalent subgraph replacement, computation graph local rewriting, or backend extension adaptation on each risk operator according to the classification results output by the classification mapping module and the adaptation strategy mapping table, and generate an adapted intermediate representation model with corresponding verification benchmarks. Layered verification module: Used to perform subgraph-level and whole-network-level layered verification on the adapted intermediate representation model output by the adaptation execution module. After the verification is passed, the vehicle-side inference deployment model is generated.
[0016] Furthermore, to achieve the above objectives, the present invention also proposes a device for adapting to non-operators in the deployment of a semantic segmentation model for intelligent driving point cloud. The device includes: a memory, a processor, and a program for adapting to non-operators in the deployment of a semantic segmentation model for intelligent driving point cloud, which is stored in the memory and can run on the processor. The program for adapting to non-operators in the deployment of a semantic segmentation model for intelligent driving point cloud consists of the steps for implementing the method, system, device, and computer program product for adapting to non-operators in the deployment of a semantic segmentation model for intelligent driving point cloud as described above.
[0017] Furthermore, to achieve the above objectives, the present invention also provides a computer program product, which includes programs such as those for deploying a semantic segmentation model for intelligent driving point cloud, which do not support operator adaptation. When the programs such as those for deploying a semantic segmentation model for intelligent driving point cloud are executed by a processor, they implement the method, system, device, and computer program product for deploying a semantic segmentation model for intelligent driving point cloud that do not support operator adaptation as described above.
[0018] The advantages and effects of this invention are: This invention, through systematic computational graph analysis and operator compatibility evaluation, can automatically identify unsupported and high-risk operators in point cloud semantic segmentation models, replacing manual line-by-line checks and significantly improving problem localization efficiency. Simultaneously, it employs a hierarchical and categorized adaptation strategy, prioritizing adaptation through intermediate representation layer mapping supplementation, equivalent subgraph replacement, and local graph rewriting, only introducing backend extensions when necessary. This minimizes invasiveness to the inference backend and ensures the model's original semantics and deployment portability.
[0019] This invention further constructs a hierarchical verification closed loop from the subgraph level to the entire network level. The verification benchmark is passed down level by level along with the model data, which can verify the consistency of accuracy at each stage during the adaptation process. This avoids the problem of local operators being usable but the overall network result being abnormal, ensuring that the accuracy of the deployed model is consistent with the original training model. In addition, the standardized adaptation process can be reused for the vehicle-side deployment of various point cloud perception models, reducing redundant development work, significantly improving engineering deployment efficiency and operational stability, and meeting the high reliability and low latency requirements of intelligent driving vehicle platforms for model deployment. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is a flowchart of the non-operator adaptation method for deploying a point cloud semantic segmentation model for intelligent driving according to the present invention. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] like Figure 1 As shown, in one embodiment of the present invention, a method for adapting to non-supporting operators for deployment of intelligent driving point cloud semantic segmentation models includes the following steps: Step S10: Obtain the trained point cloud semantic segmentation model and a representative point cloud input sample set, perform model export and benchmark inference, generate an intermediate representation model with validation benchmark data, and output the intermediate representation model with benchmark data.
[0024] It should be noted that in step S10, obtaining the point cloud semantic segmentation model and representative samples and exporting the intermediate representation model refers to collecting the trained point cloud segmentation model file, weights, and configurations, while selecting typical point cloud data covering multiple scenarios such as urban roads, highways, and parks, containing different numbers of points and obstacle distributions, as benchmark test samples. Using the model export tool of the training framework, the dynamic computation graph of the training state is converted into a static inference computation graph, removing training-specific nodes such as dropout and gradient updates, and performing basic optimizations such as constant folding to generate a standard format intermediate representation model. Simultaneously, the benchmark samples are run in the original training environment, and the tensor outputs of each layer are recorded as validation benchmark data. This benchmark data is then integrated into the supporting information of the intermediate representation model and output together.
[0025] Understandably, this step completes the input preparation for deployment and adaptation. On the one hand, it transforms the training model into an intermediate representation that can be further analyzed. On the other hand, it constructs a unified benchmark sample and accuracy benchmark, providing a unified standard for subsequent full-process verification and comparison.
[0026] It should be understood that traditional adaptation processes often neglect the construction of benchmark samples, resulting in a lack of comparative data when accuracy issues arise, making it difficult to pinpoint the source of error. This step completes the benchmark data collection and binds it to the model output beforehand, ensuring that each subsequent adaptation step has a clear accuracy reference and guaranteeing the traceability of the adaptation process.
[0027] For example, a vehicle-mounted point cloud semantic segmentation model is trained in the PyTorch framework, using 100 frames of typical urban scene point clouds as benchmark samples. The system converts the model into an intermediate representation model in ONNX format using an export tool, and runs all benchmark samples in the PyTorch environment. The output tensors of each layer of the model are saved as validation benchmarks, and the benchmark data is bound to the ONNX model as supporting information and output together to the next stage.
[0028] Step S20: Use the intermediate representation model with benchmark data output in step S10 to perform computation graph analysis, extract operator nodes, tensor attributes and topological connectivity, and identify unsupported operators and high-risk operators by combining the operator support list of the target inference backend, and output the risk operator set, node topology information and supporting benchmark data.
[0029] It should be noted that in step S20, computation graph analysis and operator identification refer to reading the intermediate representation model output from the previous step, traversing all computation nodes, extracting attributes such as the type of each operator, the shape and data type of the input and output tensors, and the connection relationships between nodes, and constructing a complete list of node attributes. Each operator is then matched against the official list of supported operators in the target vehicle's inference backend in multiple dimensions, calculating a compatibility score based on operator type, shape constraints, and data type. Operators with scores below a threshold are deemed unsupported, while those with scores meeting the threshold but exhibiting risks related to dynamic input shapes or low-precision values are classified as high-risk operators. Finally, a set of high-risk operators is compiled, and the node topology information, along with the corresponding baseline data, is passed down to the next level.
[0030] Understandably, this step enables automated and systematic identification of operators that do not support the model, replacing the traditional method of manual node-by-node inspection. It can comprehensively cover all operators in the model and avoid missing any hidden risks.
[0031] It should be understood that traditional methods often use the successful construction of the model as the criterion, failing to identify operators that can be constructed but pose accuracy risks. This step, through multi-dimensional compatibility evaluation, covers both explicitly unsupported operators and implicitly high-risk operators, resulting in a more comprehensive identification.
[0032] For example, if a target inference backend is TensorRT, the system parses the ONNX model output from the previous step, resulting in 327 computation nodes. After matching calculations, three custom index aggregation operators have a compatibility of 0 and are therefore deemed unsupported; five reduction operators containing dynamic point count dimensions have a compatibility of 0.6 and are therefore deemed high-risk operators. This results in a risk set containing eight operators, with each operator's upstream and downstream connection nodes marked, and output along with the corresponding baseline data.
[0033] Step S30: Based on the risk operator set and node topology information output in step S20, classify them hierarchically according to operator functional attributes, input and output constraints and deployment risk levels, determine the adaptation category corresponding to each risk operator, and output the classification results, adaptation strategy mapping table and supporting benchmark data.
[0034] It should be noted that in step S30, hierarchical classification refers to classifying risk operators into different adaptation categories based on the attributes of the risk operators output in the previous step, considering dimensions such as semantic complexity, rewriting feasibility, and backend modification invasiveness. The categories are: directly mappable operators (those that meet semantic standards but lack derived mappings); equivalently rewriteable operators (high-level operators that can be decomposed into multiple basic standard operators); graph structure sensitive operators (operators supported by the operator itself but whose dynamic shape or topological order does not meet backend requirements); numerically sensitive operators (operators that are runnable but prone to numerical anomalies at low precision); and backend extension operators (operators that cannot be equivalently expressed through standard operators and must be implemented through backend plugins). Simultaneously, a corresponding basic adaptation strategy is matched for each category, forming a strategy mapping table. Finally, the classification results, strategy mapping table, and supporting benchmark data are output together.
[0035] Understandably, this step avoids the high-cost approach of developing backend plugins for all problems by classifying them into different operators and matching them with differentiated adaptation solutions, thus achieving an optimal balance between adaptation cost and effectiveness.
[0036] It should be understood that traditional adaptation methods lack a classification decision-making mechanism, which can easily lead to the premature introduction of highly invasive plugin development, increasing workload and reducing model portability. This step follows the principle of "lightweight adaptation first, heavy-duty adaptation as a backup," maximizing the use of low-intrusion solutions.
[0037] For example, among the aforementioned eight risk operators, one operator belongs to the directly mappable class, two operators belong to the equivalent rewrite class, two operators belong to the graph structure sensitive class, two operators belong to the numerical sensitive class, and one operator belongs to the backend extension class. The system generates five corresponding strategies: mapping supplementation, subgraph replacement, local rewriting, numerical protection, and backend extension, forming an adaptation strategy mapping table, which is passed down along with node topology information and baseline data.
[0038] Step S40: Based on the classification results and adaptation strategy mapping table output in step S30, perform mapping supplementation, equivalent subgraph replacement, local rewriting of computation graph or backend extension adaptation on each risk operator to generate an adapted intermediate representation model with corresponding verification benchmarks.
[0039] It should be noted that in step S40, operator adaptation execution refers to performing corresponding adaptation operations on various risk operators in descending order of priority, based on the adaptation strategy mapping table output in the previous step. Priority is calculated comprehensively based on the degree of operator incompatibility, the scope of impact, and accuracy risk; operators with greater impact and higher risk are processed first. During the adaptation process, the operation scope is strictly limited to the operator association subgraph, without altering the overall model topology. After all operator adaptations are completed, the corresponding validation benchmark data is associated with the adapted model to generate an optimized intermediate representation model.
[0040] Understandably, this step follows a hierarchical strategy to adapt all risk operators, balancing adaptation efficiency and model stability while minimizing intrusion into the original model.
[0041] It should be understood that traditional adaptation methods often involve a one-time, full-scale modification, which can easily introduce new compatibility issues. This approach prioritizes and adapts step by step, with each step verifiable, making it easier to pinpoint new problems introduced during the adaptation process.
[0042] For example, the system first processes the highest-priority backend extension operators, registering corresponding custom plugins in the inference backend; then it processes directly mappable operators, supplementing the exported mapping rules; next, it decomposes two high-level aggregation operators into equivalent subgraphs of basic operator combinations; it performs local graph rewriting on dynamic shape operators, adding dynamic shape range, shape configuration, or controlled boundary constraints; finally, it adds precision protection for numerically sensitive operators. After all this is completed, it generates an adapted ONNX model and binds the validation baseline data to the model for output.
[0043] Step S50: Perform subgraph-level and whole-network-level hierarchical verification on the adapted intermediate representation model output from step S40. After the verification is passed, generate the vehicle-side inference deployment model.
[0044] It should be noted that in step S50, hierarchical verification refers to reading the adapted model and its accompanying verification benchmark data from the previous step, first verifying the correctness of the output of each adaptation operator from the local subgraph dimension, and then verifying the end-to-end semantic segmentation accuracy of the model from the whole network dimension. Subgraph verification extracts the local computation graph of each risk operator, extracts the corresponding input tensor from the supporting benchmark, and compares whether the error of the output before and after adaptation is within the allowable range. Whole network verification inputs complete benchmark point cloud samples, using the original model output in the benchmark as a reference, and compares and calculates indicators such as class consistency rate and intersection-union ratio. After all verifications pass, a deployment model that can be directly loaded and run on the vehicle-side inference engine is generated.
[0045] Understandably, this step constructs a dual verification loop from local to global, ensuring that no operator adaptation introduces accuracy deviations and guaranteeing that the final deployed model's functionality is completely consistent with the original model.
[0046] It should be understood that traditional adaptation often only verifies whether the model can run successfully, lacking systematic accuracy verification, which can easily lead to a decrease in segmentation accuracy after deployment. This step uses layered verification to intercept accuracy anomalies in advance, avoiding functional defects after deployment.
[0047] For example, the system first retrieves the corresponding tensors from the model's baseline and verifies the local subgraphs of the eight risk operators one by one. All subgraph output errors are less than 1e-5, indicating successful subgraph verification. Then, it performs full-network verification, achieving a semantic category consistency rate of 99.2% for 100 baseline samples. The intersection-union ratio (IUU) for key categories such as vehicles and pedestrians decreases by less than 0.5%, meeting deployment requirements. Finally, it generates a TensorRT deployment model that can run on the vehicle-side Orin platform.
[0048] Furthermore, in one embodiment of the present invention, a system that does not support operator adaptation for deployment of intelligent driving point cloud semantic segmentation models is proposed. The system includes: Model export module: used to obtain the trained point cloud semantic segmentation model and representative point cloud input sample set, perform model export and benchmark inference, generate an intermediate representation model with verification benchmark data, and output the intermediate representation model with benchmark data; Operator identification module: It is used to perform computation graph analysis based on the intermediate representation model with benchmark data output by the model export module, extract operator nodes, tensor attributes and topological connectivity, identify unsupported operators and high-risk operators by combining the operator support list of the target inference backend, and output the risk operator set, node topology information and supporting benchmark data. The classification and mapping module is used to classify risk operators according to their functional attributes, input and output constraints and deployment risk levels based on the risk operator set and node topology information output by the operator identification module. It determines the adaptation category corresponding to each risk operator and outputs the classification results, adaptation strategy mapping table and supporting benchmark data. The adaptation execution module is used to perform mapping supplementation, equivalent subgraph replacement, computation graph local rewriting, or backend extension adaptation on each risk operator according to the classification results output by the classification mapping module and the adaptation strategy mapping table, and generate an adapted intermediate representation model with corresponding verification benchmarks. Layered verification module: Used to perform subgraph-level and whole-network-level layered verification on the adapted intermediate representation model output by the adaptation execution module. After the verification is passed, the vehicle-side inference deployment model is generated.
[0049] The non-operator adaptation system for deploying intelligent driving point cloud semantic segmentation models provided in this application adopts the non-operator adaptation method for deploying intelligent driving point cloud semantic segmentation models as described in the above embodiments. This system can solve the technical problems in existing point cloud model deployments, such as difficulty in operator identification, low adaptation efficiency, lack of accuracy guarantee, and poor reusability. Compared with existing technologies, the beneficial effects of the non-operator adaptation system for deploying intelligent driving point cloud semantic segmentation models provided in this application are the same as those of the non-operator adaptation method for deploying intelligent driving point cloud semantic segmentation models as described in the above embodiments. Furthermore, other technical features of the non-operator adaptation system for deploying intelligent driving point cloud semantic segmentation models are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.
[0050] This application provides an unsupported operator adaptation device for deploying a point cloud semantic segmentation model for intelligent driving. The unsupported operator adaptation device for deploying a point cloud semantic segmentation model for intelligent driving includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the unsupported operator adaptation method for deploying a point cloud semantic segmentation model for intelligent driving described in Embodiment 1 above.
[0051] In one embodiment of the present invention, the device that does not support operator adaptation for deployment of the intelligent driving point cloud semantic segmentation model may include, but is not limited to, mobile terminals such as in-vehicle computing platforms, domain controllers, and in-vehicle AI acceleration devices. The described device that does not support operator adaptation for deployment of the intelligent driving point cloud semantic segmentation model is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0052] The non-operator-adaptive device deployed for the intelligent driving point cloud semantic segmentation model may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) or a program loaded from a storage device into machine-readable storage medium (RAM). The RAM also stores various programs and data required for the operation of the non-operator-adaptive device deployed for the intelligent driving point cloud semantic segmentation model. The processor, ROM, and machine-readable storage medium are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus. Typically, the following systems can be connected to the I / O interface: input devices including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices including, for example, magnetic tapes, hard disks, etc.; and a communication unit. The communication unit allows the non-operator-adaptive device deployed for the intelligent driving point cloud semantic segmentation model to exchange data with other devices wirelessly or via wired communication.
[0053] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication unit, or installed from a storage device, or installed from a read-only memory. When the computer program is executed by a processor, it performs the functions defined in the methods of the embodiments disclosed in this application.
[0054] The non-operator adaptation device for deploying intelligent driving point cloud semantic segmentation models provided in this application adopts the non-operator adaptation method for deploying intelligent driving point cloud semantic segmentation models as described in the above embodiments. This solves the technical problems in existing technologies such as difficulty in operator identification, low adaptation efficiency, lack of accuracy guarantee, and poor reusability in point cloud model deployment. Compared with existing technologies, the beneficial effects of the non-operator adaptation device for deploying intelligent driving point cloud semantic segmentation models provided in this application are the same as the beneficial effects of the non-operator adaptation method for deploying intelligent driving point cloud semantic segmentation models as described in the above embodiments. Furthermore, other technical features of this non-operator adaptation device for deploying intelligent driving point cloud semantic segmentation models are the same as those disclosed in the previous embodiment method, and will not be repeated here.
[0055] The various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.
[0056] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for deploying a semantic segmentation model for intelligent driving point cloud without supporting operator adaptation.
[0057] The computer program product provided in this application can solve the technical problems of difficult operator recognition, low adaptation efficiency, lack of accuracy guarantee, and poor reusability in point cloud model deployment in the prior art. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the non-operator adaptation method for intelligent driving point cloud semantic segmentation model deployment provided in the above embodiments, and will not be repeated here.
[0058] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A method for deploying point cloud semantic segmentation models for intelligent driving that does not support operator adaptation, characterized in that, The method includes the following steps: Step S10: Obtain the trained point cloud semantic segmentation model and a representative point cloud input sample set, perform model export and benchmark inference, generate an intermediate representation model with validation benchmark data, and output the intermediate representation model with benchmark data. Step S20: Perform computation graph analysis based on the intermediate representation model with benchmark data output in step S10, extract operator nodes, tensor attributes and topological connectivity, identify unsupported operators and high-risk operators by combining the operator support list of the target inference backend, and output the risk operator set, node topology information and supporting benchmark data. Step S30: Based on the risk operator set and node topology information output in step S20, classify them hierarchically according to operator functional attributes, input and output constraints and deployment risk levels, determine the adaptation category corresponding to each risk operator, and output the classification results, adaptation strategy mapping table and supporting benchmark data. Step S40: Based on the classification results and adaptation strategy mapping table output in step S30, perform mapping supplementation, equivalent subgraph replacement, local rewriting of computation graph or backend extension adaptation on each risk operator to generate an adapted intermediate representation model with corresponding verification benchmarks. Step S50: Perform subgraph-level and whole-network-level hierarchical verification on the adapted intermediate representation model output from step S40. After the verification is passed, generate the vehicle-side inference deployment model.
2. The method for non-operator adaptation for deploying intelligent driving point cloud semantic segmentation models according to claim 1, characterized in that, The step S10, which generates an intermediate representation model with verification benchmark data and outputs the intermediate representation model with benchmark data, specifically includes: Step S101: Obtain the network structure, weight parameters and configuration file of the point cloud semantic segmentation model, and collect point cloud samples covering different point scales, scene densities and obstacle distributions to construct a benchmark input sample set; Step S102: Based on the export interface of the training framework, convert the training-state model into an inference-state computation graph, remove the training-specific nodes, perform constant folding optimization, and generate a standard intermediate representation model. Step S103: Using the benchmark input sample set as input, run the original training model and the initial intermediate representation model respectively, record the output tensors of each layer as verification benchmark data, and integrate them into the supporting information of the intermediate representation model.
3. The method for non-operator adaptation for deploying intelligent driving point cloud semantic segmentation models according to claim 1, characterized in that, The steps in step S20, which output the risk operator set, node topology information, and supporting baseline data, specifically include: Step S201: Traverse the computation graph nodes of the intermediate representation model, read the type name, input and output tensor shape, data type and topological connection relationship of each operator, and construct a complete node attribute table; Step S202: Match the operators in the node attribute table with the list of supported operators in the target inference backend one by one, and calculate the single operator compatibility. The calculation formula is as follows: in For the first The compatibility of the operators Score for operator type matching. For tensor shape constraint matching score, Scoring is given based on data type matching. , and These are the weighting coefficients for the corresponding items; Step S203: Mark operators with compatibility below a preset threshold as unsupported operators, and mark operators with compatibility that meet the standard but have risks of dynamic shape or numerical divergence as high-risk operators. Summarize them to form a set of risk operators, and output them together with node topology information and supporting benchmark data.
4. The method for non-operator adaptation for deploying intelligent driving point cloud semantic segmentation models according to claim 1, characterized in that, The steps in step S30, which output the classification results, the adaptation strategy mapping table, and the corresponding benchmark data, specifically include: Step S301: Based on the operator's functional type, computational semantic complexity, and backend modification intrusion, risk operators are divided into five adaptation categories: directly mappable, equivalently rewriteable, graph structure sensitive, numerical sensitive, and backend extension. Step S302: According to preset principles, match the corresponding basic adaptation strategy for each type of operator and generate an adaptation strategy mapping table; Step S303: Combine the node topology information to mark the range of the associated subgraph of each risk operator, determine the boundary of the adaptation operation, and output the classification results, adaptation strategy mapping table and supporting benchmark data together.
5. The method for non-operator adaptation for deploying intelligent driving point cloud semantic segmentation models according to claim 4, characterized in that, In step S301, a local numerical protection strategy is implemented during the adaptation process for numerically sensitive operators: The numerical risk propagation path is located from the node topology information output in step S30. Local high-precision calculation, outlier clamping, or replacement with numerically stable equivalent operators are used for the risk area. At the same time, the protection strategy is verified through subgraph to confirm that it does not affect the overall network segmentation accuracy.
6. The method for non-operator adaptation for deploying intelligent driving point cloud semantic segmentation models according to claim 1, characterized in that, Step S40, which involves performing mapping supplementation, equivalent subgraph replacement, local rewriting of the computation graph, or backend extension adaptation on each risk operator to generate an adapted intermediate representation model with a corresponding verification benchmark, specifically includes: Step S401: Extract the compatibility, subgraph influence range coefficient, and accuracy risk level coefficient of each risk operator from the adaptation strategy mapping table, and calculate the adaptation priority of each risk operator. The calculation formula is as follows: in For the first The adaptation priority of each operator For operator compatibility, This is the subplot influence range coefficient. For the accuracy risk level coefficient, , , These are the weighting coefficients; Step S402: According to the order of priority from high to low, supplement the derived mapping relationship for operators that can be directly mapped, replace the operators that can be equivalently rewritten with equivalent subgraphs composed of basic operators, and perform local computation graph rewriting for operators that are sensitive to graph structure. Step S403: For backend extended operators that cannot be adapted through the intermediate representation layer, the extended implementation module is connected to the inference backend to complete the adaptation and optimization of all risk operators, generate the adapted intermediate representation model, and synchronously associate it with the corresponding verification benchmark data.
7. The method for non-operator adaptation for deploying intelligent driving point cloud semantic segmentation models according to claim 1, characterized in that, The step S50, which involves performing subgraph-level and whole-network-level hierarchical verification on the adapted intermediate representation model output from step S40, and generating the vehicle-side inference deployment model after successful verification, specifically includes: Step S501: Extract the local subgraph corresponding to each risk operator, extract the corresponding input tensor from the validation benchmark attached to the adapted model, compare the error of the subgraph output before and after adaptation, and complete the subgraph-level validation. Step S502: Perform end-to-end inference on the complete model. Using the original model output in the validation benchmark as a reference, compare the point cloud semantic segmentation output of the original model and the adapted model, calculate the class consistency rate and the intersection-union ratio of key classes, and complete the whole network-level validation. Step S503: If the verification is successful, a deployment model that can run on the vehicle-side inference engine is generated. If the verification fails, the process returns to the corresponding stage for re-adaptation.
8. A system that does not support operator adaptation for deploying point cloud semantic segmentation models for intelligent driving, characterized in that: The system executes the non-supported operator adaptation method for deploying intelligent driving point cloud semantic segmentation models as described in claim 1, including: Model export module: used to obtain the trained point cloud semantic segmentation model and representative point cloud input sample set, perform model export and benchmark inference, generate an intermediate representation model with verification benchmark data, and output the intermediate representation model with benchmark data; Operator identification module: It is used to perform computation graph analysis based on the intermediate representation model with benchmark data output by the model export module, extract operator nodes, tensor attributes and topological connectivity, identify unsupported operators and high-risk operators by combining the operator support list of the target inference backend, and output the risk operator set, node topology information and supporting benchmark data. The classification and mapping module is used to classify risk operators according to their functional attributes, input and output constraints and deployment risk levels based on the risk operator set and node topology information output by the operator identification module. It determines the adaptation category corresponding to each risk operator and outputs the classification results, adaptation strategy mapping table and supporting benchmark data. The adaptation execution module is used to perform mapping supplementation, equivalent subgraph replacement, computation graph local rewriting, or backend extension adaptation on each risk operator according to the classification results output by the classification mapping module and the adaptation strategy mapping table, and generate an adapted intermediate representation model with corresponding verification benchmarks. Layered verification module: Used to perform subgraph-level and whole-network-level layered verification on the adapted intermediate representation model output by the adaptation execution module. After the verification is passed, the vehicle-side inference deployment model is generated.
9. A device that does not support operator adaptation for deploying a point cloud semantic segmentation model for intelligent driving, characterized in that: include: The system includes a memory, a processor, and an adaptation program for deploying a semantic segmentation model for intelligent driving point cloud that is not supported by operators, which is stored in the memory and can run on the processor. When the adaptation program for deploying a semantic segmentation model for intelligent driving point cloud that is not supported by operators is executed by the processor, it implements the adaptation method for deploying a semantic segmentation model for intelligent driving point cloud that is not supported by operators as described in any one of claims 1 to 7.
10. A computer program product, characterized in that, The computer program product includes an adaptation program for deploying a semantic segmentation model for intelligent driving point cloud, which does not support operators. When the adaptation program for deploying a semantic segmentation model for intelligent driving point cloud is executed by the processor, it implements the adaptation method for deploying a semantic segmentation model for intelligent driving point cloud as described in any one of claims 1 to 7.
