Online labeling, training and reasoning platform and method for identifying AI model of 3D point cloud

Abstract

The invention provides an online labeling, training and reasoning platform and method for identifying an AI model of a 3D point cloud, and belongs to the technical field of artificial intelligence. The system comprises a file management module, an online labeling module, an online training module and an online reasoning module. The file management module is used for managing a point cloud file, an algorithm file, an AI model file and a classification label file; the online labeling module is used for performing category labeling on the point cloud file and storing a labeling result back to the file management module; the online training module is used for training the AI model and storing the trained AI model in the file management module; and the online reasoning module performs reasoning classification on the point cloud file by using an AI model to obtain a point cloud classification result. According to the invention, the implementation difficulty of 3D point cloud marking, training and reasoning can be reduced, and the 3D point cloud deep learning efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an online annotation, training and inference platform and method for AI models used to identify 3D point clouds. Background Technology

[0002] In related technologies, offline annotation software includes CloudCompare, ArcGIS Pro, and GeoScene Pro, while algorithm editing software includes text editors. Algorithms, annotations, and models are managed using file management systems, and models are trained and inferred using software such as Visual Studio Code and PyCharm. Configuring a 3D point cloud deep learning environment requires numerous software programs, is complex to configure, and presents a high technical barrier.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main objective of this application is to propose an online annotation, training, and inference platform and method for AI models used to identify 3D point clouds. This method can reduce the difficulty of 3D point cloud annotation, training, and inference, and also simplify the point cloud data processing process, providing greater convenience for the application of point cloud data.

[0005] To achieve the above objectives, one aspect of this application proposes an online annotation, training, and inference platform for an AI model used to identify 3D point clouds. The platform includes a file management module, an online annotation module, an online training module, and an online inference module. The file management module is used to manage point cloud files, algorithm files, AI model files, and classification label files; The online annotation module is used to classify the point cloud file and save the annotation results back to the file management module. The online training module is used to train the AI ​​model and save the trained AI model to the file management module; The online inference module uses the AI ​​model to perform inference and classification on the point cloud file, and obtains the point cloud classification result.

[0006] In some embodiments, the platform further includes an online algorithm editing module; The online algorithm editing module is used to add, delete, and modify the content of the algorithm file.

[0007] In some embodiments, the file management module includes a point cloud data receiving service and a point cloud preview file generation service; The point cloud data receiving service is used to receive point cloud data and synthesize it into point cloud files for storage. The point cloud preview file generation service is used to generate point cloud preview files based on the stored point cloud files; the point cloud preview files include a first preview file, a second preview file, and a third preview file; the first preview file describes the location of the center point of the point cloud file; the second preview file describes the hierarchical inheritance relationship of each spatial octree in each level of detail of the point cloud file; the third preview file describes the point cloud data information of each spatial octree in each level of detail of the point cloud file.

[0008] In some embodiments, the online annotation module loads and renders the point cloud file, provides a graphical interactive tool for users to select and classify the point cloud, and sets the annotation results to a custom field in the point cloud data; the annotation results include the data selected by the user and the set classification value.

[0009] In some embodiments, the online training module builds a training environment based on container technology and provides a web interface for configuring training parameters and monitoring the training process.

[0010] In some embodiments, the online training module divides the labeled point cloud file and the algorithm file into a training set, a validation set, and a test set, and trains the AI ​​model.

[0011] In some embodiments, the online inference module outputs the point cloud classification results in the following ways: The point cloud classification results are displayed in a visual form using a web page. Provide a download link for the file data containing the point cloud classification results; Provides an API interface for synchronous or asynchronous calls and returns the point cloud classification results in a user-specified format.

[0012] To achieve the above objectives, another aspect of this application proposes an online annotation, training, and inference method for an AI model used to identify 3D point clouds, implemented through the platform described above. The method includes: The file management module allows you to manage point cloud files, algorithm files, AI model files, and classification label files. The point cloud file is categorized using the online annotation module, and the annotation results are saved back to the file management module. The AI ​​model is trained using the online training module, and the trained AI model is saved to the file management module. The AI ​​model is used to perform reasoning and classification on the point cloud file through the online inference module, and the point cloud classification result is obtained.

[0013] In some embodiments, the step of classifying the point cloud file using an online annotation module and saving the annotation results back to the file management module includes: The point cloud files in the file management module are labeled and classified through the online annotation module. The point cloud to be classified and the classification value selected by the user are obtained. The outline points of the annotation space polygon are submitted to the backend classification annotation service using the X, Y, Z three-dimensional coordinate sequence and the classification value of the selected object. The classification and labeling service is used to set the classification values ​​into custom fields of point cloud data according to preset standards to obtain the labeling results. The annotation results are saved back to the file management module to obtain the annotated point cloud file, and a corresponding point cloud preview file is generated at the same time.

[0014] In some embodiments, the method further includes: The algorithm file can be edited online via the online algorithm editing module on a web page.

[0015] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0016] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0017] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method.

[0018] The embodiments of this application include at least the following beneficial effects: This application provides an online annotation, training, and inference method, platform, electronic device, storage medium, and program product for recognizing AI models of 3D point clouds. The solution includes a file management module, an online annotation module, an online training module, and an online inference module. The file management module manages point cloud files, algorithm files, AI model files, and classification label files. The online annotation module performs category annotation on the point cloud files and saves the annotation results back to the file management module. The online training module trains the AI ​​model and saves the trained AI model to the file management module. The online inference module uses the AI ​​model to perform inference and classification on the point cloud files to obtain point cloud classification results. This application can reduce the implementation difficulty of 3D point cloud annotation, training, and inference, and improve the efficiency of 3D point cloud deep learning. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of an online annotation, training, and inference platform for an AI model used to identify 3D point clouds, provided in an embodiment of this application. Figure 2 This is a flowchart illustrating the point cloud data receiving service received in an embodiment of this application. Figure 3 This is a flowchart illustrating the preview file generated by the preview file generation service provided in this application embodiment; Figure 4 This is a schematic diagram of a preview file provided in an embodiment of this application; Figure 5 This is a schematic diagram of an online preview file provided in an embodiment of this application; Figure 6 This is a schematic diagram illustrating the use of multi-point selection to outline pipe objects to be classified, as provided in an embodiment of this application. Figure 7 This is a schematic diagram of the classification results provided in the embodiments of this application; Figure 8 This is a schematic diagram of the overall classification and labeling process provided in the embodiments of this application; Figure 9 This is a schematic diagram of the algorithm editing provided in the embodiments of this application; Figure 10 This is a schematic diagram of the partitioned dataset provided in an embodiment of this application; Figure 11 This is a schematic diagram of the Docker image used for training provided in an embodiment of this application; Figure 12 This is a schematic diagram illustrating the setting of necessary training parameters provided in an embodiment of this application; Figure 13 This is a trend chart of the training and validation loss values ​​provided in the embodiments of this application; Figure 14 This is a schematic diagram of a point cloud file containing a main water supply pipeline, provided in an embodiment of this application. Figure 15 This is a schematic diagram of point cloud recognition using an AI model provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of platforms and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0023] 1) 3D, three-dimensional; 2) CloudCompare, an open-source 3D point cloud and mesh processing software; 3) ArcGIS Pro, a professional desktop geographic information system (GIS) software developed by Esri Corporation of the United States, is used for map creation, spatial analysis and visualization; 4) GeoScene Pro, a professional desktop geographic information system (GIS) software developed by Esri China. 5) Visual Studio Code, a free, open-source, cross-platform source code editor developed by Microsoft; 6) PyCharm, an integrated development environment (IDE) developed by JetBrains specifically for Python programming; 7) PAI (Platform of Artificial Intelligence), Alibaba Cloud Machine Learning Platform; 8) Kubeflow, an open-source platform built on Kubernetes, is used to simplify, deploy, and manage machine learning workflows; 9) PyTorch, a deep learning framework open-sourced by Facebook (now Meta); 10) Tensorflow, an open-source artificial intelligence and machine learning framework developed by Google; 11) IDE (Integrated Development Environment) is an application that provides a program development environment, which generally includes tools such as code editors, compilers, debuggers and graphical user interfaces; 12) LOD (Level of Detail), a computer graphics optimization technique; 13) LAS Sepecification, the standard for exchanging LiDAR data; 14) API (Application Programming Interface); 15) JSON (JavaScript Object Notation), a JavaScript object representation; 16) Minio, a high-performance, open-source object storage software compatible with Amazon S3; 17) SDK (Software Development Kit); 18) URL (Uniform Resource Locator); 19) HTTP (Hypertext Transfer Protocol); 20) PotreeConverter, a command-line tool that converts large point cloud data into Potree format; 21) Classification, in point cloud processing, refers to the process or attribute of assigning a semantic label (such as ground, vegetation, building) to each point; 22) PCL (Point Cloud Library); 23) Jupyter Notebook, an open-source web application; 24) Docker, an open-source application containerization platform.

[0024] Currently, commercial AI platforms provided by internet companies such as Alibaba Cloud AI Platform (PAI) for online users primarily use traditional file types for data annotation, training, and inference, such as images, text, videos, and audio. They lack the capability for online 3D point cloud annotation, training, and inference.

[0025] In annotation, training, and inference of 3D point cloud data, similar to traditional deep learning methods involving images, version and alignment management of source files, datasets, annotations, algorithms, and models is required. Offline annotation software includes CloudCompare, ArcGIS Pro, and GeoScene Pro; algorithm editing software uses text editors; algorithms, annotations, and models are managed using a file management system; and software such as Visual Studio Code and PyCharm is used for model training and inference. Configuring a 3D point cloud deep learning environment requires numerous software programs, is complex, and presents a high technical barrier to entry.

[0026] Kubeflow and other open-source deep learning frameworks integrate commonly used deep learning algorithms such as PyTorch and Tensorflow, but they are training and inference platforms and do not have the ability to manage files and label 3D point clouds.

[0027] In view of this, this application provides an online annotation, training and inference platform and method for AI models that identify 3D point clouds. It integrates online 3D point cloud annotation, online point cloud dataset management, online algorithm editing, online training / inference and other capabilities, which will help reduce the implementation difficulty of 3D point cloud annotation, training and inference and improve the efficiency of 3D point cloud deep learning.

[0028] The online annotation, training, and inference method for AI models used to identify 3D point clouds provided in this application relates to the field of artificial intelligence. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the online annotation, training, and inference method for AI models used to identify 3D point clouds, but is not limited to the above forms.

[0029] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0030] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0031] Figure 1 This is a schematic diagram of an online annotation, training, and inference platform for recognizing 3D point clouds provided in an embodiment of this application. The platform includes a file management module, an online annotation module, an online training module, and an online inference module. The file management module is used to manage point cloud files, algorithm files, AI model files, and classification label files; The online annotation module is used to classify point cloud files and save the annotation results back to the file management module; The online training module is used to train AI models and save the trained AI models to the file management module; The online inference module uses an AI model to perform inference and classification on point cloud files, and obtains point cloud classification results.

[0032] Optionally, the file management module of this embodiment can manage point cloud files, algorithm files, AI model files, classification label files, etc., via a web page. The point cloud files saved by the file management module can be user-uploaded point cloud files or point cloud files generated by the online annotation module and online inference module. The file management module can save both pre-trained AI model files and AI model files trained by the online training module. The online annotation module can perform category annotation on point cloud files via a web page; the online training module can train AI models via a web page; and the online inference module can use AI models to perform inference and classification on point cloud files. Users can complete 3D point cloud annotation, training, and inference operations on a unified platform, thus avoiding the hassle of switching between different software. Through this integration, users only need a web browser to complete the entire process without complex software environment configuration. This not only reduces the difficulty of 3D point cloud annotation, training, and inference but also simplifies the point cloud data processing process, providing greater convenience for the application of point cloud data. This embodiment of the invention provides visualization of classification results, allowing direct viewing of results via the web, enabling rapid demonstration and reporting of classification results.

[0033] In some embodiments, the platform also includes an online algorithm editing module; The online algorithm editing module is used to add, delete, and modify the content of algorithm files.

[0034] Optionally, the online algorithm editing module of this invention allows algorithm engineers to directly modify code, start training, and verify results on the same platform, without having to switch between local IDEs and the platform or upload and download files. After reviewing the training results, if researchers find that the model performs poorly in a certain aspect, they can open the algorithm file online, adjust the neural network structure or learning rate, and start a new round of training, which can accelerate the experimental cycle.

[0035] In some embodiments, the file management module includes a point cloud data receiving service and a point cloud preview file generation service; The point cloud data receiving service is used to receive point cloud data and synthesize it into point cloud files for storage; The point cloud preview file generation service is used to generate point cloud preview files based on stored point cloud files. The point cloud preview files include a first preview file, a second preview file, and a third preview file. The first preview file describes the location of the center point of the point cloud file. The second preview file describes the hierarchical inheritance relationship of each spatial octree in each level of detail of the point cloud file. The third preview file describes the point cloud data information of each spatial octree in each level of detail of the point cloud file.

[0036] Optionally, the file management module of this embodiment divides point cloud processing into two services: point cloud data reception and point cloud preview file generation. It generates three types of structured preview files. The beneficial effect is that, through spatial octree and LOD (Level of Detail) technology, static point cloud data that cannot be directly used on the web is transformed into dynamic intelligent data suitable for network streaming and real-time rendering, thereby overcoming the performance bottleneck of massive 3D data applications on the web. Regardless of the device or format from which the point cloud originates, the platform converts it into a preset preview format. This allows the platform's front-end renderer and annotation tools to only process the pre-set data format, greatly simplifying the complexity of the platform system.

[0037] In some embodiments, the online annotation module loads and renders point cloud files, provides graphical interactive tools for users to select and classify point clouds, and sets the annotation results to custom fields in the point cloud data; the annotation results include the data selected by the user and the set classification values.

[0038] Optionally, the online annotation module of this embodiment provides a 3D view by loading and rendering point cloud files. Graphical interactive tools (such as 3D bounding box selection and polygon drawing) transform complex annotation tasks into direct "click-drag-select" operations. Intuitive tools and smooth rendering allow annotators to quickly define the spatial extent of objects. The bounding box selection tool is particularly efficient, instantly capturing all points within a 3D area. Annotation information is directly written to custom fields in the point cloud file, achieving integrated storage of data and labels.

[0039] In some embodiments, the online training module builds a training environment based on container technology and provides a web interface for configuring training parameters and monitoring the training process.

[0040] Optionally, the container technology in this embodiment of the invention pre-packages the training environment (including frameworks and libraries) into a standardized, reproducible image. When a user starts training, the platform automatically instantiates a brand new, clean container, ensuring a consistent training environment for each session. Through the web interface, training becomes as simple as filling out a form: selecting an algorithm, selecting data, and setting the learning rate. The platform is responsible for passing these configuration parameters to the container and starting the task.

[0041] In some embodiments, the online training module divides the labeled point cloud files and algorithm files into training sets, validation sets, and test sets, and trains the AI ​​model.

[0042] Optionally, the online training module provided in this embodiment of the invention allows users to select data and algorithms simply through a web interface. The platform automatically completes a series of complex operations such as data splitting, training, validation, and model saving, lowering the technical barrier to using machine learning. Users can easily try different algorithms, different hyperparameters, and different data combinations, and quickly determine which configuration is better based on feedback from the validation set. This rapid iteration capability is key to optimizing model performance.

[0043] In some embodiments, the online inference module outputs point cloud classification results in the following ways: Display point cloud classification results in a visual format using web pages; Provide a download link for the file data containing the point cloud classification results; Provides an API interface for synchronous or asynchronous calls and returns point cloud classification results in a user-specified format.

[0044] Optionally, web page visualization can be used for project review, results presentation, and manual quality checks. File downloads meet project deliverable requirements, providing a standard point cloud file containing AI analysis results, which can be archived as part of the final project outcome and supports future offline analysis. API interfaces are the core of process automation. Synchronous / asynchronous call mechanisms allow the platform to allocate resources efficiently. Small files and requests requiring real-time responses use synchronous mode; large files and time-consuming requests use asynchronous mode, avoiding request timeouts and improving system stability and user experience.

[0045] To achieve the above objectives, another aspect of this application proposes an online annotation, training, and inference method for an AI model used to identify 3D point clouds, implemented through the platform described above. The method includes, but is not limited to, steps S101 to S104: Step S101: Manage point cloud files, algorithm files, AI model files, and classification label files through the file management module; Step S102: Use the online annotation module to annotate the point cloud file with categories, and save the annotation results back to the file management module; Step S103: Train the AI ​​model through the online training module and save the trained AI model to the file management module; Step S104: Using the online inference module, the AI ​​model is used to perform inference classification on the point cloud file to obtain the point cloud classification result.

[0046] Steps S101 to S104 as shown in the embodiments of this application improve efficiency and lower the threshold for 3D point cloud AI applications through an end-to-end closed-loop system of centralized data management, online annotation, automatic model training, and intelligent application feedback.

[0047] In some embodiments, step S102 may include, but is not limited to, steps S201 to S203: Step S201: Through the online annotation module, the point cloud files in the file management module are annotated and classified, the point cloud to be classified and the classification value selected by the user are obtained, and the outline points of the annotation space polygon are submitted to the backend classification annotation service using the X, Y, Z three-dimensional coordinate sequence and the classification value of the selected object. Step S202: Using the classification and annotation service, the classification values ​​are set into the custom fields of the point cloud data according to the preset standards to obtain the annotation results; Step S203: Save the annotation results back to the file management module to obtain the annotated point cloud file, and generate the corresponding point cloud preview file at the same time.

[0048] In step S201 of some embodiments, the user directly selects within a bounding box in 3D space. This method aligns with the user's understanding of the real world and operational habits, reducing the learning cost and operational difficulty of annotation work. This embodiment of the invention captures the complete three-dimensional spatial range of the target object, rather than a two-dimensional projection or estimation, providing high-quality, high-precision training labels for the AI ​​model. This embodiment of the invention submits the X, Y, and Z three-dimensional coordinate sequence of the contour points of the spatial polygon, which defines a precise 3D volume, ensuring the accuracy of all points within the selected area. The front-end (browser) is only responsible for interaction and data collection, leaving complex calculations and file operations to the back-end. This ensures the responsiveness of the web interface and avoids browser lag or crashes due to processing massive point clouds.

[0049] In step S202 of some embodiments, writing custom fields to the point cloud allows the annotation results to be integrated and standardized with the original data. Operating according to preset standards (such as the LAS Sepecification specification) ensures the long-term value and interoperability of the data.

[0050] In step S203 of some embodiments, a new preview file is generated after annotation, and the page can be refreshed to view the annotation effect (e.g., different objects are displayed in different colors). This instant feedback is very important for verifying annotation quality, identifying and correcting errors, and improving the efficiency of annotation work.

[0051] In some embodiments, an online annotation, training, and inference method for an AI model for recognizing 3D point clouds further includes: Step S105: The algorithm file is edited online via the online algorithm editing module on the web page.

[0052] In step S105 of some embodiments, the algorithm engineer can complete the process of writing code, starting training, viewing results, analyzing problems, and modifying the code again on the same platform and in the same browser tab. This embodiment allows for the free customization and optimization of the algorithm based on the uniqueness of one's data and the specific problem to be solved.

[0053] As an optional implementation, the online annotation, training, and inference platform for AI models used to identify 3D point clouds in this embodiment of the invention contains the following basic modules: a file management module, an annotation module, a training module, and an inference module.

[0054] File management module: Manage point cloud files, algorithm files, AI model files, classification label files, etc. via a web page.

[0055] Online annotation module: Uses a web page to annotate point cloud files with categories.

[0056] Online training module: Train AI models using web pages.

[0057] Online inference module: Uses AI models to perform inference and classification on point cloud files.

[0058] Furthermore, an additional online algorithm editing module could be added, such as: Online algorithm editing module: Used to add, delete, and modify the contents of algorithm files via a web page.

[0059] refer to Figure 1 The main workflow, with the "Online Algorithm Editing" module (highlighted by the dashed box) being optional, includes the following basic steps: Step 1: Upload point cloud files, AI algorithm files, and other files to the platform. The platform will categorize and store the files according to their type.

[0060] Step 2: Perform online annotation (i.e., classification) on the point cloud files uploaded in Step 1, and save the annotation results back to the file management module.

[0061] Step 3: Divide the point cloud files uploaded to the file management module and the labeled point cloud files into training, validation, and test sets. Use the corresponding AI algorithm files in the file management module to train the AI ​​model. After training, the trained model can be saved to the file management module.

[0062] Step 4: Using the AI ​​model in the file management module, point cloud files can be inferred online to obtain point cloud classification results.

[0063] Furthermore, in step 1, the classification tag file can be uploaded to the file management module and referenced in the online annotation module, online algorithm editing module, online training module, and online inference module.

[0064] Furthermore, in step 1, AI model files that have already been preliminarily trained offline using AI algorithm files can be uploaded to the file management module and loaded as pre-trained models in the online training module, thereby combining training sets, validation sets, test sets, and other data for retraining.

[0065] Furthermore, in step 1, the AI ​​model file that has been initially trained by the AI ​​algorithm file offline can be uploaded to the file management module and loaded as an inference model in the online inference module, thereby performing inference on the point cloud file.

[0066] Furthermore, in step 1, point cloud files uploaded to the file management module can be annotated online, classified, and the annotation results saved back to the file management module, regardless of whether they have been labeled (i.e., classified).

[0067] Furthermore, in step 1, the algorithm file uploaded to the file management module can be edited and saved online if its content needs to be modified.

[0068] Furthermore, in step 4, the point cloud file used for online inference can be a point cloud file that already exists in the file management module, or a point cloud file that can be uploaded by public users (i.e., without authentication / anonymity).

[0069] Furthermore, in step 4, the reasoning task can be performed in the form of a web page, and the reasoning result can be displayed as a web page or downloaded as file data.

[0070] Furthermore, in step 4, the inference task can be performed using an API, which can be called synchronously or asynchronously, and the result format is not limited to JSON.

[0071] Furthermore, step 4 can be used to perform inference classification on the point cloud files and then incorporate them into the file management module. Step 2 can then be used to modify the results of the inference classification, thereby achieving the purpose of AI model-assisted point cloud pre-classification.

[0072] One embodiment of the present invention is as follows: (1) The file management module contains two file-related services: point cloud data receiving service and point cloud preview file generation service. The file management module uses Minio object storage software to store uploaded point cloud files, algorithm files, model files, classification label files, and other file data. Minio can provide access interfaces to other services in the platform through APIs, and can generate file download URLs that users can directly download in their browsers through its SDK.

[0073] On the one hand, refer to Figure 2 When a user submits point cloud data via a web page, the point cloud data receiving service receives the point cloud data. After all the data has been received, the point cloud data receiving service will synthesize the received data into an original point cloud file and store the file as a whole file in a file repository such as Minio.

[0074] On the other hand, reference Figure 3 Once the point cloud data receiving service has received the uploaded point cloud data, it can notify the preview file generation service to generate a point cloud preview file via synchronous notification (e.g., HTTP request call) or asynchronous notification (e.g., message queue subscription message). The preview file generation service will retrieve the original point cloud file from the file repository and then use PotreeConverter to generate the point cloud preview file. An example of the preview file generation is shown below. Figure 4 As shown, metadata.json contains the metadata information of the preview file, describing information such as the center point location of the point cloud file; hierarchy.bin describes the hierarchical inheritance relationship of each spatial octree in each LOD (Level of Details) of the point cloud preview file; and octree.bin describes the specific stored point cloud data information of each spatial octree in each LOD of the point cloud file.

[0075] Next, the preview file generation service will save the generated preview files as folders to the file repository.

[0076] In the file management module, users can view detailed information about uploaded point cloud files, such as the overall point cloud shape, point cloud color, and reflection intensity, by opening the generated preview file for online viewing. Figure 5 As shown.

[0077] (2) In the online point cloud annotation module, users can annotate and classify point cloud files in the file management module online. Users manually select the point clouds to be classified, set classification values, and then submit the outline points of the polygon in the annotation space using the X, Y, and Z 3D coordinate sequence and the classification values ​​of the selected objects to the backend classification annotation service. The classification annotation service can set the classification values ​​in the Classification field of the point cloud data according to the LAS Sepecification standard, or use a custom field to store the classification values. The classification annotation service can use the PCL library to modify the point cloud files during storage. After modifying the point cloud files, the classification annotation service will simultaneously generate a corresponding point cloud preview file. For example... Figure 6 Load the raw point cloud file using Potree.js in a web browser, and use multi-point selection to outline the pipe objects (pipecombined) to be classified.

[0078] After the classification and annotation results are submitted, the page will automatically refresh and load a point cloud preview file containing the annotation results. The classification results are as follows: Figure 7 As shown, the white part represents the categorized Pipe Combined objects.

[0079] The overall process is as follows Figure 8 .

[0080] (3) After the point cloud data is labeled, the labeled files are also saved in the file management module. Algorithms that need to be used for training / inference can be edited and modified online to adapt to the point cloud data used for training / inference. When editing algorithms online, users can open a web page to edit the algorithm, such as... Figure 9 This is a screenshot of an algorithm being edited using a Jupyter Notebook.

[0081] (4) Point cloud training: After the point cloud files are labeled, several point cloud files can be divided into training set, validation set and test set. With the corresponding algorithm, the training parameters are set to train an AI model that can recognize 3D point clouds.

[0082] refer to Figure 10To divide a dataset into training, validation, and test sets, the interface for training tasks includes basic information and dataset settings. The basic information includes the task name and description. The dataset settings include setting the dataset proportion. In setting the dataset proportion, select the dataset to be divided and then set the scale to set the proportion of the training, validation, and test sets.

[0083] refer to Figure 11 Configure the Docker image for training and select the appropriate algorithm and startup function. The settings interface includes training settings and resource specifications. The training settings include setting the run operation, algorithm, algorithm name, and run command. The resource specifications include the type of node, computing unit, and specification.

[0084] refer to Figure 12 The necessary training parameters can be set through the parameter editing page, including the number of training epochs, batch size, Adam optimizer learning rate, scheduler gamma, and save frequency. The page also provides the options to start training and clear all settings.

[0085] After training is complete, you can view the loss value during the training process, such as... Figure 13The graph shows the trend of training and validation losses during model evaluation. The training loss reflects how well the model fits the training data, while the validation loss monitors generalization ability, prevents overfitting, and guides hyperparameter tuning. The loss values ​​gradually converge towards zero as training progresses. After training, the trained model can be saved back to the file management module for use in inference.

[0086] (5) In the point cloud inference module, the model in the file management module can be used to infer the unclassified point cloud data, thereby obtaining the classification value of the unclassified point cloud data. The operation interface of the point cloud inference module provides the point cloud raw view, inference report, and inference label. The inference report includes file name, inference model, category of inference results, and completion time. The inference label includes code, classification type, and classification style. The classification type includes background and water main. The classification style provides different color marking methods. Figure 14 This is a point cloud file containing the main water supply pipe.

[0087] like Figure 15 As shown, the blue part represents the point cloud of the main water supply pipe (WaterMain) identified using an AI model.

[0088] This invention provides an online annotation, training, and inference platform and method for AI models used to identify 3D point clouds. The online workflow for identifying 3D point clouds covers annotation, training, and inference. Users can complete all these operations on a unified platform, avoiding the hassle of switching between different software. Through this integration, users can complete the entire process using only a web browser, without the need for complex software environment configurations. This not only reduces the difficulty of 3D point cloud annotation, training, and inference but also simplifies the point cloud data processing, providing greater convenience for the application of point cloud data.

[0089] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0090] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0091] Another embodiment of the hardware structure of the electronic device, the electronic device including: The processor can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to achieve the technical solutions provided in the embodiments of this application. The memory can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and called by the processor to execute the methods described in the embodiments of this application. Input / output interfaces are used to implement information input and output; The communication interface is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). A bus is used to transfer information between various components of a device, such as processors, memory, input / output interfaces, and communication interfaces. The processor, memory, input / output interfaces, and communication interfaces communicate with each other within the device via a bus.

[0092] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0093] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0094] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0095] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0096] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0097] The online annotation, training, and inference methods, platforms, electronic devices, storage media, and program products for recognizing 3D point clouds provided in this application embodiment construct a one-stop platform integrating cloud-native architecture, web-based point cloud visualization, containerized AI workflow, and standardized data management. This transforms the originally highly specialized, complex, and costly 3D point cloud recognition technology into an easy-to-use, highly collaborative, and rapidly deployable high-productivity tool, enabling various industries to easily develop and apply their own 3D point cloud AI capabilities.

[0098] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0099] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0100] The platform embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0102] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0103] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0104] In the various embodiments of this application, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. An online annotation, training, and inference platform for AI models used to recognize 3D point clouds, characterized in that, The platform includes a file management module, an online annotation module, an online training module, and an online inference module; The file management module is used to manage point cloud files, algorithm files, AI model files, and classification label files; The online annotation module is used to classify the point cloud file and save the annotation results back to the file management module. The online training module is used to train the AI ​​model and save the trained AI model to the file management module; The online inference module uses the AI ​​model to perform inference and classification on the point cloud file, and obtains the point cloud classification result.

2. The platform according to claim 1, characterized in that, The platform also includes an online algorithm editing module; The online algorithm editing module is used to add, delete, and modify the content of the algorithm file.

3. The platform according to claim 1, characterized in that, The file management module includes point cloud data receiving service and point cloud preview file generation service; The point cloud data receiving service is used to receive point cloud data and synthesize it into point cloud files for storage. The point cloud preview file generation service is used to generate point cloud preview files based on the stored point cloud files; the point cloud preview files include a first preview file, a second preview file, and a third preview file; the first preview file describes the location of the center point of the point cloud file; the second preview file describes the hierarchical inheritance relationship of each spatial octree in each level of detail of the point cloud file; the third preview file describes the point cloud data information of each spatial octree in each level of detail of the point cloud file.

4. The platform according to claim 1, characterized in that, The online annotation module loads and renders the point cloud file, provides a graphical interactive tool for users to select and classify the point cloud, and sets the annotation results into a custom field of the point cloud data; The annotation results include the data selected by the user and the classification values ​​set.

5. The platform according to claim 1, characterized in that, The online training module builds a training environment based on container technology and provides a web interface for configuring training parameters and monitoring the training process.

6. The platform according to claim 1, characterized in that, The online training module divides the labeled point cloud file and the algorithm file into training set, validation set, and test set, and trains the AI ​​model.

7. The platform according to claim 1, characterized in that, The online inference module outputs the point cloud classification results in the following ways: The point cloud classification results are displayed in a visual form using a web page. Provide a download link for the file data containing the point cloud classification results; Provides an API interface for synchronous or asynchronous calls and returns the point cloud classification results in a user-specified format.

8. An online annotation, training, and inference method for an AI model for recognizing 3D point clouds, implemented through a platform as described in any one of claims 1 to 7, characterized in that, The method includes: The file management module allows you to manage point cloud files, algorithm files, AI model files, and classification label files. The point cloud file is categorized using the online annotation module, and the annotation results are saved back to the file management module. The AI ​​model is trained using the online training module, and the trained AI model is saved to the file management module. The AI ​​model is used to perform reasoning and classification on the point cloud file through the online inference module, and the point cloud classification result is obtained.

9. The method according to claim 8, characterized in that, The step of classifying the point cloud file using the online annotation module and saving the annotation results back to the file management module includes: The point cloud files in the file management module are labeled and classified through the online annotation module. The point cloud to be classified and the classification value selected by the user are obtained. The outline points of the annotation space polygon are submitted to the backend classification annotation service using the X, Y, Z three-dimensional coordinate sequence and the classification value of the selected object. The classification and labeling service is used to set the classification values ​​into custom fields of point cloud data according to preset standards to obtain the labeling results. The annotation results are saved back to the file management module to obtain the annotated point cloud file, and a corresponding point cloud preview file is generated at the same time.

10. The method according to claim 8, characterized in that, The method further includes: The algorithm file can be edited online via the online algorithm editing module on a web page.

Images