Self-closed-loop optimized point cloud fusion data labeling method and system

Abstract

The invention provides a self-closed-loop optimized point cloud fusion data labeling method and system. The method comprises the following steps: receiving multi-modal data and carrying out preprocessing format standardization, multi-modal data space-time alignment, multi-source point cloud data splicing, data enhancement and noise filtering; performing 3D detection on the preprocessed multi-modal data based on a pre-trained deep learning model to obtain an initial labeling result, and for the initial labeling result, analyzing a difference value between the initial labeling result and a real labeling result obtained through analysis based on a preset labeling rule; after an initial annotation structure is subjected to annotation correction operation, the annotation correction operation is converted into a structured difference feature vector, and a weighted distillation loss function is constructed to train a deep learning model; and performing parameter iteration updating by taking minimization of the loss function as a target until the preset iteration times are reached, and then obtaining the updated target deep learning model, so that the labeling precision of the point cloud fusion data can be improved, and the labeling efficiency and accuracy can be improved.

Description

Technical Field

[0001] This invention relates to the field of data annotation, and in particular to a self-closed-loop optimized point cloud fusion data annotation method and system. Background Technology

[0002] With the deepening development of 3D vision applications such as autonomous driving, intelligent robots, and smart cities, point cloud data annotation is the foundation supporting their development. In autonomous driving, vehicles rely on point cloud data annotation to accurately identify roads, lanes, traffic signals, and various obstacles. In the field of intelligent robots, intelligent robots need to use point cloud data annotation to understand their surrounding environment, identify various objects and targets, and thus complete operational tasks. The construction of smart cities involves using point cloud data annotation to build high-precision 3D models of cities, such as buildings, roads, bridges, and green spaces, providing comprehensive and accurate data support for urban planning, traffic management, environmental monitoring, and public safety.

[0003] In existing technologies, point cloud data processing places heterogeneous demands on CPUs, GPUs, FPGAs, and large amounts of memory; automatic annotation models lack accuracy in complex scenes, especially in scenarios with occlusion, small targets, and blurred boundaries, while purely manual annotation is inefficient and costly; manual correction and model training cannot form an effective closed loop; dedicated hardware is costly and lacks a unified scheduling and management mechanism; point cloud data often needs to be fused with multimodal data such as images and IMUs, but existing systems lack effective fusion annotation capabilities, lack deep perception and adaptive optimization capabilities for point cloud task features, and cannot achieve a balanced optimization of quality, efficiency, and resource utilization. Summary of the Invention

[0004] This invention provides a self-closed-loop optimized point cloud fusion data annotation method and system, the main purpose of which is to solve the problem that the existing technology cannot achieve a balanced optimization of quality, efficiency and resource utilization in the point cloud annotation process.

[0005] To achieve the above objectives, this invention provides a point cloud fusion data annotation method for self-closed-loop optimization, comprising:

[0006] The system receives and preprocesses multimodal data, including point cloud data, image data, and raw inertial data. Preprocessing includes format standardization, spatiotemporal alignment of multimodal data, stitching of multi-source point cloud data, data augmentation, and noise filtering.

[0007] The pre-trained deep learning model is used to perform 3D detection on the pre-processed multimodal data to obtain initial annotation results. The initial annotation results are then used for instance segmentation and semantic segmentation. Based on the preset annotation rules, the pre-processed multimodal data is analyzed to obtain the true annotation results.

[0008] Analyze the differences between the initial annotation results and the actual annotation results, and perform annotation correction operations on the initial annotation structure according to the differences. Then, transform the annotation correction operations into structured difference feature vectors.

[0009] The deep learning model is trained by constructing a weighted distillation loss function based on the structured difference feature vectors. The parameters are iteratively updated with the goal of minimizing the loss function until a preset number of iterations is reached to obtain the updated target deep learning model.

[0010] Optionally, the method further includes:

[0011] Multimodal data is stored in a high-performance distributed file system, initial annotation results are stored in a local cache, and parameters of deep learning models are persisted to object storage.

[0012] Optionally, the process of training the deep learning model by constructing a weighted distillation loss function based on the structured difference feature vectors also includes:

[0013] Based on the difference value and the frequency of annotation correction operations, the multimodal data is divided into high-discrepancy regions and low-discrepancy regions.

[0014] During the training of the deep learning model, a weighted distillation loss function is constructed based on the structured difference feature vectors. This increases the sampling weight and iteration rounds in high-difference regions and decreases the sampling weight and iteration rounds in low-difference regions.

[0015] To address the aforementioned problems, this invention also provides a self-closed-loop optimized point cloud fusion data annotation system, which includes:

[0016] The unified resource management module is used to manage all task nodes in a unified manner. Task nodes include point cloud preprocessing nodes, deep learning training nodes, and interactive annotation nodes.

[0017] The intelligent annotation module is used to receive multimodal data and preprocess it. Based on the pre-trained deep learning model, it performs 3D detection on the preprocessed multimodal data to obtain initial annotation results and performs instance segmentation and semantic segmentation on the initial annotation results. Based on the preset annotation rules, it analyzes the preprocessed multimodal data to obtain the real annotation results.

[0018] The self-closed-loop optimization module is used to calculate the difference between the initial annotation results and the actual annotation results, and then perform annotation correction operations on the initial annotation structure according to the difference values, and transform the annotation correction operations into structured difference feature vectors. Based on the structured difference feature vectors, a weighted distillation loss function is constructed to train the deep learning model. The parameters are iteratively updated with the goal of minimizing the loss function until the preset number of iterations is reached to obtain the updated target deep learning model.

[0019] Optionally, the point cloud preprocessing node receives multimodal data and performs preprocessing through the central processing unit, the deep learning training node trains the deep learning model through the image processor, and the interactive annotation node receives the initial annotation results and the real annotation results through memory and analyzes the difference values.

[0020] Optionally, the system also includes an operation and maintenance monitoring module for full-process monitoring and fault detection. When a fault occurs, it detects available task processing nodes and performs task migration, and records the data processing process and model training process as logs.

[0021] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0022] At least one processor; and,

[0023] A memory communicatively connected to the at least one processor; wherein,

[0024] The memory has a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the self-closed-loop optimized point cloud fusion data annotation method described above.

[0025] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the self-closed-loop optimized point cloud fusion data annotation method described above.

[0026] The beneficial effects of this invention are as follows:

[0027] This invention provides a point cloud fusion data annotation method and system with self-closed-loop optimization. Through multimodal optimization and self-closed-loop optimization, the accuracy of point cloud fusion data annotation is improved; intelligent annotation and correction are combined to improve the efficiency and accuracy of annotation; real-time monitoring of the annotation process allows for flexible scheduling of each node, improving resource utilization and saving costs.

[0028] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0029] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating a self-closed-loop optimized point cloud fusion data annotation method according to an embodiment of the present invention.

[0031] Figure 2 This is a schematic diagram of the structure of a self-closed-loop optimized point cloud fusion data annotation system according to an embodiment of the present invention.

[0032] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0034] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0035] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0036] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0037] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0038] Figure 1 This is a flowchart illustrating a self-loop optimized point cloud fusion data annotation method according to an embodiment of the present invention. This application provides a self-loop optimized point cloud fusion data annotation method, which includes the following steps S101~S104:

[0039] Step S101: Receive multimodal data and perform preprocessing; the multimodal data includes point cloud data, image data and raw inertial data; the preprocessing includes format standardization, spatiotemporal alignment of multimodal data, stitching of multi-source point cloud data, data augmentation and noise filtering.

[0040] Step S102: Based on the pre-trained deep learning model, perform 3D detection on the pre-processed multimodal data to obtain initial annotation results, and perform instance segmentation and semantic segmentation on the initial annotation results. Analyze the pre-processed multimodal data based on preset annotation rules to obtain real annotation results.

[0041] Step S103: Analyze the difference between the initial annotation results and the actual annotation results, and perform annotation correction operations on the initial annotation structure according to the difference values, and then convert the annotation correction operations into structured difference feature vectors.

[0042] Step S104: Construct a weighted distillation loss function based on the structured difference feature vector to train the deep learning model. Iterate the parameters to minimize the loss function until the preset number of iterations is reached to obtain the updated target deep learning model.

[0043] In step S101, multimodal data preprocessing in the central processing unit (CPU) also includes point cloud filtering, downsampling, and coordinate transformation; the multimodal data includes point cloud data, image data, and raw inertial data (IMU); spatiotemporal alignment and fusion are achieved by extrinsic parameter calibration and timestamp matching of point cloud data, image data, and raw inertial data through multiple image processors (GPUs) and high-speed I / O interfaces; large-scale point cloud reconstruction is achieved through feature matching and optimization algorithms; data augmentation includes point cloud completion, denoising, and enhancement to improve data quality; furthermore, a unified data lake architecture is constructed to uniformly manage point cloud data, image data, and raw inertial data.

[0044] In steps S102 and S103, initial detection boxes are generated and initial annotation results are obtained through point-based and voxel-based algorithms; instance segmentation and semantic segmentation are used to classify point cloud data, instance segmentation is performed using the PointGroup algorithm, and semantic segmentation is performed using the PointNet++ algorithm. Furthermore, the accuracy and completeness of the initial annotation results are checked by combining image information; preset annotation rules are manually set based on historical data and scene feature patterns.

[0045] Furthermore, the difference values ​​include, but are not limited to, mislabeled values ​​and missing values. Mislabeled values ​​are data that differ from the initial annotation results and the actual annotation results. Missing values ​​are data that are annotated in the actual annotation results but not in the initial annotation results. The process of converting the annotation correction operation into a structured difference feature vector includes: obtaining the difference type, correction magnitude and spatial features, assigning values ​​to the difference type, correction magnitude and spatial features and arranging them in a preset order to obtain a structured difference feature vector.

[0046] In step S104, the deep learning model is trained using an image processor, and sampling weights and training rounds are increased for regions with large discrepancies and numerous annotation correction operations. In this embodiment of the invention, the process of training the deep learning model by constructing a weighted distillation loss function based on the structured discrepancy feature vector further includes steps S1041 to S1042:

[0047] Step S1041: Divide the multimodal data into regions based on the difference value and the frequency of annotation correction operations to obtain high difference regions and low difference regions.

[0048] Step S1042: During the training of the deep learning model, a weighted distillation loss function is constructed based on the structured difference feature vector. In this process, the sampling weight and iteration rounds of high difference regions are increased, while the sampling weight and iteration rounds of low difference regions are decreased.

[0049] Furthermore, to suppress catastrophic forgetting during model training, a continuous learning algorithm is used, including Elastic Weight Consolidation (EWC) and Learning without Forgetting (LwF). EWC adds a weight penalty term to the weighted distillation loss function, while LwF incorporates distillation loss during training on new data to constrain the output of the new model and ensure consistency with the old model. The model updated in each iteration is recorded and stored. The learning rate is dynamically adjusted according to the annotation instructions, and the batch size is adjusted according to the task complexity.

[0050] In this embodiment of the invention, the method further includes:

[0051] Multimodal data is stored in a high-performance distributed file system, initial annotation results are stored in a local cache, and parameters of deep learning models are persisted to object storage.

[0052] Figure 2 This is a schematic diagram of the structure of a self-closed-loop optimized point cloud fusion data annotation system according to an embodiment of the present invention. On the other hand, the present invention also provides a self-closed-loop optimized point cloud fusion data annotation system, which includes:

[0053] A unified resource management module is used to manage all task nodes, including point cloud preprocessing nodes, deep learning training nodes, and interactive annotation nodes. Specifically, RoCE (RDMA over Converged Ethernet) networking is used for transmission between nodes to reduce transmission latency and improve efficiency; the intelligent network interface controller receives annotation data and performs network task processing and data preprocessing; data synchronization between multiple clusters is achieved through dedicated lines or VPNs. In this embodiment of the invention, the point cloud preprocessing node receives and preprocesses multimodal data through a central processing unit, the deep learning training node trains the deep learning model through an image processor, and the interactive annotation node receives initial annotation results and real annotation results through memory and analyzes the differences.

[0054] The intelligent annotation module is used to receive multimodal data and preprocess it. Based on the pre-trained deep learning model, it performs 3D detection on the preprocessed multimodal data to obtain initial annotation results and performs instance segmentation and semantic segmentation on the initial annotation results. Based on the preset annotation rules, it analyzes the preprocessed multimodal data to obtain the real annotation results.

[0055] The self-closed-loop optimization module is used to analyze the difference between the initial annotation results and the actual annotation results, and then perform annotation correction operations on the initial annotation structure according to the difference values, and then transform the annotation correction operations into structured difference feature vectors. Based on the structured difference feature vectors, a weighted distillation loss function is constructed to train the deep learning model, and the sampling weights and training rounds are enhanced for high difference regions. The parameters are iteratively updated with the goal of minimizing the loss function until the preset number of iterations is reached.

[0056] In this embodiment of the invention, the system also includes an operation and maintenance monitoring module, which is used for full-process monitoring and fault detection. When a fault occurs, it detects available task processing nodes and performs task migration, and records the data processing process and model training process as logs.

[0057] On the other hand, embodiments of the present invention also provide an electronic device, the electronic device including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory has a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute to implement the above-described self-closed-loop optimized point cloud fusion data annotation method.

[0058] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the above-described self-closed-loop optimized point cloud fusion data annotation method.

[0059] The present invention will now be described with reference to a specific embodiment:

[0060] This invention proposes a self-closed-loop optimized point cloud fusion data annotation method and system.

[0061] Step 1: The system architecture deployment adopts a three-tier architecture design that integrates cloud, edge, and endpoint collaboration.

[0062] (1) Edge layer deployment. Lightweight inference nodes are deployed at data collection points, equipped with NVIDIA Jetson series embedded devices; point cloud preprocessing and initial annotation generation modules are integrated to support real-time data processing; model inference services are deployed to provide low-latency annotation capabilities.

[0063] (2) Cloud layer deployment. Build a Kubernetes container orchestration cluster to manage heterogeneous computing resources; deploy a model training cluster equipped with NVIDIA A100 / H100 GPU servers; implement a distributed storage system to support the management of massive point cloud data.

[0064] (3) Terminal layer deployment. Provide a web-based annotation workbench that supports concurrent access by multiple users; integrate 3D visualization components to realize interactive annotation of point cloud data; deploy behavior analysis services to collect annotation interaction data.

[0065] Step 2, Core Component Configuration:

[0066] (1) Construction of heterogeneous resource pool and task classification and scheduling. The point cloud preprocessing node is equipped with a high-frequency CPU and a large-capacity memory, and is responsible for point cloud filtering, downsampling and coordinate transformation; the model training node is equipped with a high-performance GPU, and is responsible for point cloud detection and segmentation model training; the inference acceleration node uses FPGA to achieve hardware acceleration, and realizes point cloud feature extraction and model inference acceleration; the multimodal fusion node is configured with multiple GPUs and high-speed I / O, and supports spatiotemporal alignment and fusion of point cloud and image and IMU data; the interactive annotation node is equipped with a large-memory CPU and high-speed storage, and supports multi-user concurrent annotation and real-time rendering.

[0067] The compute nodes use RDMA over Converged Ethernet (RoCE) networking to reduce data transmission latency; labeled data streams are offloaded from network functions through smart network cards to improve concurrent processing capabilities; efficient communication and traffic management between microservices are achieved through service mesh technology; and secure and reliable data synchronization between multiple clusters is achieved through dedicated lines or VPNs.

[0068] Point cloud preprocessing tasks are scheduled to high-frequency CPU nodes, scaling elastically based on CPU utilization; model training tasks are scheduled to GPU clusters, scaling based on GPU utilization and memory usage; inference service tasks are scheduled to FPGA / ASIC nodes, scaling based on inference latency and throughput; and annotation interaction tasks are scheduled to large-memory nodes, scaling based on concurrent users and response time. Unified resource management and scheduling across public cloud, private cloud, and edge nodes are supported. Automatic scaling to the public cloud is implemented during traffic surges to ensure service quality, and resource optimization is achieved through consistent hashing and intelligent load balancing.

[0069] (2) Hot data uses NVMe SSD local caching; warm and cold data are stored in Ceph distributed storage; point cloud raw data adopts high-performance distributed file system to support efficient reading and writing of massive point cloud data; annotation intermediate results use local caching to ensure data access efficiency during the annotation process; model parameters and logs are persisted to object storage and intelligent migration is achieved through lifecycle management; multimodal data warehouse builds a unified data lake architecture to support unified management of multi-source data such as point cloud, image, and IMU; task migration and load balancing between multiple clusters, intelligent task distribution based on network topology, and collaborative processing and optimization of cross-regional data.

[0070] (3) Network architecture design: RDMA high-speed network is used between computing nodes; service mesh technology is used for communication between services; edge-cloud is connected via dedicated line or VPN.

[0071] (4) Task-aware elastic scheduling includes intelligent task classification and scheduling, unified management of hybrid cloud resources, and cross-cluster collaborative scheduling:

[0072] (5) Intelligent high availability and disaster recovery include multi-active zone deployment architecture, core components deployed across 3 or more availability zones, metadata consistency based on RAFT protocol, and data-level disaster recovery through off-site disaster recovery center.

[0073] (6) Fault prediction and self-healing include node health prediction based on time-series data, automatic fault detection and task migration, and service mesh-level traffic cut-off and circuit breaker protection.

[0074] (7) Intrinsic security and unified governance include zero-trust security architecture, data lifecycle security and unified identity and access management; among which, zero-trust security architecture includes mandatory mTLS two-way authentication for inter-service communication, identity-based dynamic access control policies, network micro-segmentation and security group policies; data lifecycle security includes end-to-end encryption of point cloud data transmission, static data encrypted storage, key rotation and security management; unified identity and access management includes unified identity authentication based on OIDC / LDAP, fine-grained RBAC permission control and operation audit and compliance reporting.

[0075] (8) AIOps intelligent operation and maintenance includes end-to-end observability, GitOps continuous delivery and intelligent operation and maintenance decision-making; end-to-end observability includes the fusion of three elements: metrics, logs and traces, point cloud task-specific monitoring metrics, and AI-based anomaly detection and root cause analysis; GitOps continuous delivery includes infrastructure as code, application configuration version management and automated deployment and rollback; intelligent operation and maintenance decision-making includes reinforcement learning-based resource optimization decision-making, capacity planning and prediction and automated fault handling and recovery.

[0076] Step 3, Workflow Implementation:

[0077] (1) The data preprocessing stage includes point cloud data access and format standardization, spatiotemporal alignment of multimodal data, data quality enhancement and noise filtering. Spatiotemporal alignment of point cloud and image data is achieved through extrinsic parameter calibration and timestamp matching to achieve accurate fusion; multi-source point cloud data stitching is based on feature matching and optimization algorithms to achieve large-scale point cloud reconstruction; data quality enhancement is achieved through point cloud completion, denoising, enhancement and other technologies to improve data quality.

[0078] (2) The intelligent annotation stage includes generating initial annotations based on deep learning models, human annotators correcting and optimizing the results, and the system recording annotation behavior and analyzing differences. Deep learning-based 3D detection uses algorithms such as Point-based and Voxel-based to generate initial detection boxes; it uses algorithms such as PointGroup and PointNet++ to achieve instance segmentation and semantic segmentation; multimodal fusion annotation combines image information to improve the accuracy and completeness of point cloud annotation; it analyzes the difference between pre-annotated and ground truth values, transforms manual corrections into structured difference feature vectors, and identifies model weaknesses based on correction frequency and difficulty for bad case mining and key optimization.

[0079] (3) The model optimization stage includes distillation training based on differential data, updating model parameters using incremental learning, and model performance evaluation and validation. Differential-driven distillation training includes constructing a weighted distillation loss function, enhancing sampling weights and training rounds for high-difference regions, and simultaneously optimizing multiple tasks such as detection, segmentation, and classification; incremental learning of model parameters includes achieving smooth model evolution through knowledge distillation, maintaining the original capabilities of the model using techniques such as EWC and LwF, and supporting multi-version model management and fast rollback; adaptive optimization strategies include dynamic learning rate adjustment based on annotation quality, adaptive batch size setting based on task complexity, and dynamic switching of training strategies based on resource status.

[0080] (4) The deployment and update phase includes optimizing the model and packaging it into a container image, releasing it to edge nodes in a canary manner, and monitoring and collecting performance feedback.

[0081] Step 4, Deployment of Critical Services:

[0082] (1) The task scheduling service includes resource scheduling based on Kubernetes, dynamic resource allocation according to task type, and support for elastic scaling and load balancing.

[0083] (2) Model training services include distributed training framework support, automatic hyperparameter tuning, training process monitoring and management.

[0084] (3) Data management services include point cloud data version control, annotation result quality management and data lifecycle management.

[0085] (4) Monitoring and maintenance services include full-link performance monitoring, intelligent alarms and fault diagnosis and automated operation and maintenance.

[0086] Step 5: Optimize strategy implementation:

[0087] (1) Resource scheduling optimization includes resource allocation based on task priority, elastic scaling with real-time load awareness, and unified scheduling of resources across clusters.

[0088] (2) Performance optimization measures include point cloud data compression and transmission, model inference pipeline optimization, and intelligent adjustment of caching strategy.

[0089] (3) The quality assurance mechanism includes automatic evaluation of annotation quality, consistency check of model output, and detection and processing of abnormal data.

[0090] Step 6, Operation and Maintenance Support System:

[0091] (1) High availability assurance includes multi-availability zone service deployment, multi-copy data storage, automatic failover and recovery.

[0092] (2) Security protection measures include end-to-end encryption of data transmission, fine control of access permissions, and audit and traceability of operational behavior.

[0093] (3) The monitoring and alarm system includes multi-level monitoring indicator collection, intelligent anomaly detection and automated fault handling.

[0094] A self-closed-loop optimized point cloud fusion data annotation method is proposed, which constructs a unified resource pool that supports multiple computing architectures, configures differentiated hardware resources for point cloud fusion annotation tasks, and performs unified scheduling of heterogeneous computing, storage and network resources through an extended container orchestration platform. The unified resource pool includes point cloud preprocessing nodes, deep learning training nodes, inference acceleration nodes, multimodal fusion nodes and interactive annotation nodes, forming a complete point cloud processing pipeline.

[0095] Deploy an intelligent annotation system with point cloud task awareness capabilities, supporting multimodal data fusion processing, intelligent initial annotation generation, and human-machine collaborative annotation optimization. The multimodal data fusion processing includes spatiotemporal alignment of point cloud and image data, multi-source point cloud data stitching, and data quality enhancement, and improves annotation quality through feature-level fusion and decision-level fusion. The human-machine collaborative annotation optimization includes intelligent analysis of annotation data, difference quantification and feature extraction, bad case mining and key optimization, forming an effective transformation from human experience to model optimization.

[0096] A self-closed-loop model optimization mechanism is implemented, driving continuous model optimization based on labeled data analysis and differential quantization. This mechanism employs differential-driven distillation training, including a weighted distillation loss function, badcase augmentation training, and multi-task collaborative optimization, supporting incremental learning and model evolution. A task-aware elastic scheduling strategy is constructed to dynamically allocate the optimal hardware combination based on point cloud task characteristics. This strategy supports dynamic resource allocation based on point cloud task characteristics, including differentiated elastic strategies for point cloud preprocessing tasks, model training tasks, inference service tasks, and labeled interaction tasks. A cross-regional multi-active high-availability architecture is implemented, integrating a data multi-copy mechanism and intelligent fault self-healing capabilities. An intrinsic security system covering network, data, and identity is constructed. An AIOps platform integrating monitoring, logging, tracing, and automated operation and maintenance is deployed.

[0097] A self-closing-loop optimized point cloud fusion data annotation system includes a resource management module for unified resource management and intelligent scheduling; a data fusion module for multimodal data fusion processing and quality enhancement; an intelligent annotation module for initial annotation generation and human-machine collaborative optimization; a difference-driven model training and evolution model optimization module; a deployment management module for model version management and service deployment; and a monitoring and maintenance module for cluster monitoring, alarms, and automated operation and maintenance. The system supports hybrid cloud and edge collaborative deployment, and can uniformly schedule heterogeneous resources in public clouds, private clouds, and edge nodes, meeting the end-to-end requirements of point cloud data acquisition, processing, and annotation.

[0098] In summary, this invention provides a self-closing-loop optimized point cloud fusion data annotation method and system, which receives multimodal data and performs preprocessing. The multimodal data includes point cloud data, image data, and raw inertial data. The preprocessing includes format standardization, spatiotemporal alignment of multimodal data, multi-source point cloud data stitching, data augmentation, and noise filtering. Based on a pre-trained deep learning model, 3D detection is performed on the preprocessed multimodal data to obtain initial annotation results, and instance segmentation and semantic segmentation are performed on the initial annotation results. Based on preset annotation rules, the preprocessed multimodal data is analyzed to obtain true annotation results. The difference between the initial annotation results and the true annotation results is analyzed, and annotation correction operations are performed on the initial annotation structure according to the difference values. The annotation correction operations are then transformed into structured difference feature vectors. A weighted distillation loss function is constructed based on the structured difference feature vectors to train the deep learning model. The parameters are iteratively updated with the goal of minimizing the loss function until a preset number of iterations is reached to obtain the updated target deep learning model.

[0099] This invention also provides an electronic device, the electronic device comprising:

[0100] At least one processor; and,

[0101] A memory communicatively connected to the at least one processor; wherein,

[0102] The memory has a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the self-closed-loop optimized point cloud fusion data annotation method described above.

[0103] This invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the self-closed-loop optimized point cloud fusion data annotation method described above.

[0104] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0105] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A self-closed-loop optimized point cloud fusion data annotation method, characterized in that, The method includes: The system receives and preprocesses multimodal data, including point cloud data, image data, and raw inertial data. Preprocessing includes format standardization, spatiotemporal alignment of multimodal data, stitching of multi-source point cloud data, data augmentation, and noise filtering. The pre-trained deep learning model is used to perform 3D detection on the pre-processed multimodal data to obtain initial annotation results. The initial annotation results are then used for instance segmentation and semantic segmentation. Based on the preset annotation rules, the pre-processed multimodal data is analyzed to obtain the true annotation results. Analyze the differences between the initial annotation results and the actual annotation results, and perform annotation correction operations on the initial annotation structure according to the differences. Then, transform the annotation correction operations into structured difference feature vectors. The deep learning model is trained by constructing a weighted distillation loss function based on the structured difference feature vectors. The parameters are iteratively updated with the goal of minimizing the loss function until a preset number of iterations is reached to obtain the updated target deep learning model.

2. The self-closed-loop optimized point cloud fusion data annotation method according to claim 1, characterized in that, The method also includes: Multimodal data is stored in a high-performance distributed file module, initial annotation results are stored in a local cache, and parameters of the deep learning model are persisted to object storage.

3. The self-closed-loop optimized point cloud fusion data annotation method according to claim 1, characterized in that, The process of training a deep learning model by constructing a weighted distillation loss function based on the structured differential feature vectors also includes: Based on the difference value and the frequency of annotation correction operations, the multimodal data is divided into high-discrepancy regions and low-discrepancy regions. During the training of the deep learning model, a weighted distillation loss function is constructed based on the structured difference feature vectors. This increases the sampling weight and iteration rounds in high-difference regions and decreases the sampling weight and iteration rounds in low-difference regions.

4. A self-closed-loop optimized point cloud fusion data annotation system, characterized in that, The system is used to execute the self-closed-loop optimized point cloud fusion data annotation method as described in any one of claims 1 to 3, and the system includes: The unified resource management module is used to manage all task nodes in a unified manner. Task nodes include point cloud preprocessing nodes, deep learning training nodes, and interactive annotation nodes. The intelligent annotation module is used to receive multimodal data and preprocess it. Based on the pre-trained deep learning model, it performs 3D detection on the preprocessed multimodal data to obtain initial annotation results and performs instance segmentation and semantic segmentation on the initial annotation results. Based on the preset annotation rules, it analyzes the preprocessed multimodal data to obtain the real annotation results. The self-closed-loop optimization module is used to analyze the difference between the initial annotation results and the actual annotation results, and then perform annotation correction operations on the initial annotation structure according to the difference values, and then transform the annotation correction operations into structured difference feature vectors. Based on the structured difference feature vectors, a weighted distillation loss function is constructed to train the deep learning model. The parameters are iteratively updated with the goal of minimizing the loss function until the preset number of iterations is reached to obtain the updated target deep learning model.

5. The self-closed-loop optimized point cloud fusion data annotation system according to claim 4, characterized in that, The point cloud preprocessing node receives multimodal data and performs preprocessing through the central processing unit, the deep learning training node trains the deep learning model through the image processor, and the interactive annotation node receives the initial annotation results and the real annotation results through memory and analyzes the difference values.

6. The self-closed-loop optimized point cloud fusion data annotation system according to claim 4, characterized in that, The system also includes an operation and maintenance monitoring module for full-process monitoring and fault detection. When a fault occurs, it detects available task processing nodes and performs task migration, and records the data processing process and model training process as logs.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the self-closed-loop optimized point cloud fusion data annotation method as described in any one of claims 1 to 3.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the self-closed-loop optimized point cloud fusion data annotation method as described in any one of claims 1 to 3.

Images