A 3D point cloud segmentation method, device, electronic equipment and storage medium

Abstract

The present disclosure provides a 3D point cloud segmentation method, device, electronic equipment and storage medium, the method comprising: obtaining 3D point cloud data and image data corresponding to the same sampling time and the same scene; converting the image data into a set of 2D masks, each 2D mask identifying the contour area of an object contained in the image data; converting the pixel coordinates of each 2D mask into a 3D point cloud coordinate system, thereby determining the corresponding view cone space of each 2D mask in the 3D point cloud coordinate system; for each 2D mask, segmenting the 3D point cloud data located in the view cone space corresponding to the mask in the 3D point cloud data to determine the point cloud block where the object is located. Accordingly, it is realized that the 3D point cloud data does not need to be manually labeled, and the mask generated by the image can guide the point cloud segmentation, thereby reducing the cost of manual labeling; it does not depend on specific sensor parameters, adapts to different hardware configurations, reduces the adaptation cost of different platforms, and enhances the generalization ability.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present disclosure relate to the technical field of point cloud segmentation, and in particular to a 3D point cloud segmentation method, device, electronic equipment and storage medium. BACKGROUND

[0002] In related technologies, 3D point cloud segmentation is to identify and separate different objects (such as vehicles, pedestrians, etc.) from unordered and sparse 3D point cloud data. Through high-precision 3D point cloud segmentation, the system can perceive the surrounding environment in real time and classify and track various dynamic and static objects, thereby providing reliable data support for decision-making and control.

[0003] In some embodiments, 3D point cloud segmentation technology relies on large-scale 3D data sets annotated by humans for model training to achieve identification and separation of different objects. At the same time, to adapt to specific sensor configurations (such as the number of lines of a laser radar, the internal and external parameters of a camera, etc.), the model needs to be customized for training with specific hardware parameters to ensure that the segmentation accuracy meets the actual needs.

[0004] However, this approach has the following problems. On the one hand, the cost of manual annotation is extremely high, and the annotation is difficult to cover diversified scenes and object types, resulting in limitations in model training. On the other hand, the model has too strong dependence on sensor configurations, and when the vehicle platform is changed (different vehicle sensor models and installation positions differ), the model must be retrained, increasing the adaptation cost. SUMMARY

[0005] Therefore, the present disclosure provides a 3D point cloud segmentation method, which comprises:

[0006] obtaining 3D point cloud data and image data corresponding to the same sampling time and the same scene;

[0007] converting the image data into a set of 2D masks, each 2D mask identifying the contour region of an object contained in the image data;

[0008] converting the pixel coordinates of each 2D mask into a 3D point cloud coordinate system, thereby determining the corresponding frustum space of each 2D mask in the 3D point cloud coordinate system;

[0009] for each 2D mask, segmenting the 3D point cloud data located in the frustum space corresponding to the mask in the 3D point cloud data to determine the point cloud block where the object is located.

[0010] Optionally, the conversion of the image data into a set of 2D masks, each 2D mask identifying the contour region of an object contained in the image data, comprises:

[0011] adjusting a visual base model to a mask generation mode;

[0012] inputting the image data into the visual base model to obtain a set of 2D masks output by the visual base model;

[0013] wherein each 2D mask corresponds to a binary image, and the binary image identifies a contour region of an object contained in the image data.

[0014] Optionally, the converting of the pixel coordinates of each 2D mask into the 3D point cloud coordinate system to determine a corresponding frustum space of each 2D mask in the 3D point cloud coordinate system comprises:

[0015] extracting a set of boundary pixel coordinates of each 2D mask;

[0016] converting the set of boundary pixel coordinates into a ray direction vector in the image sensor coordinate system according to an intrinsic matrix of the image sensor;

[0017] converting the ray direction vector into the 3D point cloud coordinate system according to an extrinsic matrix of the image sensor to a point cloud sensor, and connecting the converted ray direction vector with the optical center of the image sensor as a vertex to obtain a corresponding frustum space of each 2D mask in the 3D point cloud coordinate system.

[0018] Optionally, the method is applied to a system that maintains a voxel database, and the method further comprises:

[0019] finding, in the voxel database, point cloud blocks in which historical objects exist in an overlapping part with the point cloud block in which each segmented object exists according to the voxel coordinates of the point cloud block;

[0020] calculating, for each segmented object, a correlation degree between the point cloud block in which the object exists and each point cloud block in which a historical object exists in an overlapping part, and updating the point cloud block in which the object exists to the voxel database.

[0021] Optionally, the calculating, for each segmented object, a correlation degree between the point cloud block in which the object exists and each point cloud block in which a historical object exists in an overlapping part, and updating the point cloud block in which the object exists to the voxel database comprises:

[0022] calculating, for each segmented object, a correlation degree between the point cloud block in which the object exists and each point cloud block in which a historical object exists in an overlapping part;

[0023] if the association degree between the point cloud block where the object is located and the point cloud block where each target historical object located meets a preset condition, the point cloud block where the object is located is merged with the point cloud block where the target historical object is located in the voxel database;

[0024] if the association degree between the point cloud block where the object is located and the point cloud block where each target historical object located does not meet the preset condition, the point cloud block where the object is located is newly added in the voxel database.

[0025] Optionally, each 2D mask has a predicted intersection over union score when generated, and the method further comprises:

[0026] any object located in the point cloud block is considered as an active object in the voxel database if the association degree between the point cloud block where the object is located and the point cloud block where the historical object is located meets a preset condition;

[0027] the intersection over union scores of the 2D masks of the segmented objects corresponding to each active object are summed up, and the summed intersection over union scores are divided by the total number of active objects to obtain a 3D point cloud segmentation result confidence.

[0028] Optionally, the system also maintains a 3D segmentation model, which is used to segment the point cloud block where the object is located in the 3D point cloud data according to the 2D mask, and the method further comprises:

[0029] if the 3D point cloud segmentation result confidence is lower than a first preset threshold, the active object associated with the point cloud block where the object corresponding to the 2D mask whose intersection over union score exceeds a second preset threshold is located and the point cloud block where the active object is located are taken as pseudo labels;

[0030] Based on the pseudo labels, the lightweight trainable sub-module of the 3D segmentation model is fine-tuned online.

[0031] The present disclosure also provides a 3D point cloud segmentation device, which comprises:

[0032] an acquisition unit configured to acquire 3D point cloud data and image data corresponding to the same sampling time and the same scene;

[0033] a first conversion unit configured to convert the image data into a set of 2D masks, each 2D mask identifying the contour area of an object contained in the image data;

[0034] a second conversion unit configured to convert the pixel coordinates of each 2D mask into a 3D point cloud coordinate system, thereby determining the corresponding frustum space of each 2D mask in the 3D point cloud coordinate system;

[0035] A segmentation unit is configured to, for each 2D mask, segment 3D point cloud data located in a view cone space corresponding to the mask from the 3D point cloud data to determine a point cloud block where an object is located.

[0036] The present disclosure also provides an electronic device comprising a communication interface, a processor, a memory and a bus, the communication interface, the processor and the memory being connected to each other through the bus;

[0037] The memory stores machine readable instructions, and the processor executes the above method by invoking the machine readable instructions.

[0038] The present disclosure also provides a machine readable storage medium storing machine readable instructions, which, when invoked and executed by a processor, implement the above method.

[0039] Therefore, in the technical solution of the present disclosure, by obtaining 3D point cloud data and image data corresponding to the same sampling time and the same scene, and by means of the mapping relationship between the 2D mask obtained from the image data and the 3D view cone space, the object point cloud is filtered through the constraint of the corresponding view cone space. The core is to constrain the segmentation range of 3D point cloud by using the contour information (2D mask) of the object in the image, avoiding the dependence on large-scale labeled data and sensor configuration in pure point cloud segmentation.

[0040] Through the above method, the present disclosure realizes that 3D point cloud data does not need to be manually labeled, and only the mask generated by the image can guide the point cloud segmentation, reducing the cost of manual labeling. Moreover, the method does not depend on specific sensor parameters, is suitable for different hardware configurations, reduces the adaptation cost of the 3D point cloud segmentation model on different platforms, and enhances the generalization ability of the technology. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 FIG. 1 is a flowchart of a 3D point cloud segmentation method according to an exemplary embodiment;

[0043] Figure 2 FIG. 2 is a flowchart of another 3D point cloud segmentation method according to an exemplary embodiment;

[0044] Figure 3 FIG. 3 is a schematic diagram of online adjustment of a 3D segmentation model according to an exemplary embodiment;

[0045] Figure 4 This is an exemplary embodiment illustrating the hardware structure of an electronic device;

[0046] Figure 5 This is a block diagram illustrating a 3D point cloud segmentation device as an exemplary embodiment. Detailed Implementation

[0047] To enable those skilled in the art to better understand the technical solutions in this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0048] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this disclosure in other embodiments. In some other embodiments, the methods may include more or fewer steps than those described in this disclosure. Furthermore, a single step described in this disclosure may be broken down into multiple steps in other embodiments; and multiple steps described in this disclosure may be combined into a single step in other embodiments.

[0049] In related technologies, 3D point cloud segmentation is the process of identifying and separating different objects (such as vehicles and pedestrians) from disordered and sparse 3D point cloud data. Through high-precision 3D point cloud segmentation, the system can perceive the surrounding environment in real time and classify and track various dynamic and static objects, thereby providing reliable data support for decision-making and control.

[0050] In some embodiments, 3D point cloud segmentation technology relies on manually labeled large-scale 3D datasets for model training to achieve the recognition and separation of different objects. Meanwhile, to adapt to specific sensor configurations (such as the line count of a LiDAR, the intrinsic and extrinsic parameters of a camera), the model needs to be customized for specific hardware parameters to ensure that the segmentation accuracy meets actual requirements.

[0051] However, this approach has several drawbacks. First, manual annotation is extremely costly and cannot cover diverse scenarios and object types, resulting in limitations in model training. Second, the model is overly dependent on sensor configuration, and when the vehicle platform is changed (different vehicle sensor models and installation locations vary), the model must be retrained, increasing adaptation costs.

[0052] In view of this, the present disclosure aims to propose a technical solution for converting a 2D mask generated from an image into a 3D point cloud frustum space, and then segmenting the corresponding point cloud blocks in the frustum space, thereby achieving 3D point cloud segmentation without the need for manual annotation of training data.

[0053] The technical solution first acquires 3D point cloud data and image data corresponding to the same sampling time and the same scene; then, the image data is converted into a set of 2D masks, each 2D mask identifying the outline region of an object contained in the image data; further, the pixel coordinates of each 2D mask are transformed into a 3D point cloud coordinate system, thereby determining the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system; finally, for each 2D mask, the 3D point cloud data located in the view frustum space corresponding to the mask is segmented from the 3D point cloud data to determine the point cloud block where the object is located.

[0054] For example, an autonomous vehicle is traveling along a road filled with pedestrians and various vehicles. To ensure safety and make correct driving decisions, the vehicle needs to accurately identify all objects in its surrounding environment and their positions. The vehicle's LiDAR and front-facing camera simultaneously acquire road data at a given moment. The LiDAR outputs 3D point cloud data containing the 3D coordinates (x, y, z) of objects such as vehicles, pedestrians, and guardrails on the road, while the camera outputs 2D visual contours of these objects. The vehicle system (hereinafter referred to as the "system") then inputs the camera images into a pre-trained visual baseline model, such as the Segment Anything Model (SAM), to generate three 2D masks, corresponding to a car, a pedestrian, and a section of guardrail, respectively. Each mask marks the object's contour in binary image form (pixel values ​​within the mask are 1, and the background is 0). Further, the system uses camera intrinsic and extrinsic parameters to transform the three 2D masks from a pixel coordinate system to a 3D point cloud coordinate system, obtaining the view frustum space corresponding to each mask. Finally, for the 2D mask of the car, the system segments the 3D point cloud data corresponding to the 2D mask of the car into the view frustum space that encloses the car (such as the point cloud in the range of x in 5-15 meters, y in -2 to 2 meters, and z in 0-1.5 meters) in the 3D point cloud data, and obtains the point cloud block corresponding to the car; similarly, through the 2D mask and view frustum space of pedestrians and guardrails, the corresponding pedestrian point cloud blocks and guardrail point cloud blocks can be segmented, thus completing the 3D point cloud segmentation in this scene.

[0055] Therefore, the technical solution disclosed herein acquires 3D point cloud data and image data corresponding to the same sampling time and the same scene. It then uses the mapping relationship between the 2D mask obtained from the image data and the 3D view frustum space to filter the object point cloud through constraints of the corresponding view frustum space. The core of this solution lies in using the contour information (2D mask) of objects in the image to constrain the segmentation range of the 3D point cloud, thus avoiding the dependence on large-scale labeled data and sensor configuration found in simple point cloud segmentation.

[0056] Through the above methods, this disclosure achieves point cloud segmentation without the need for manual annotation of 3D point cloud data, using only image-generated masks to guide point cloud segmentation, thus reducing the cost of manual annotation. Moreover, this method does not rely on specific sensor parameters, adapts to different hardware configurations, reduces the adaptation cost of 3D point cloud segmentation models on different platforms, and enhances the generalization ability of the technology.

[0057] The present disclosure will now be described through specific embodiments and in conjunction with specific application scenarios.

[0058] Please see Figure 1 , Figure 1 This is a flowchart illustrating a 3D point cloud segmentation method as an exemplary embodiment. The method may perform the following steps:

[0059] Step 102: Obtain 3D point cloud data and image data corresponding to the same sampling time and the same scene.

[0060] For example, the LiDAR and front-facing camera on the vehicle simultaneously collect road data at a certain moment. The 3D point cloud data output by the LiDAR includes the 3D coordinates (x, y, z) of objects such as vehicles, pedestrians, and guardrails on the road, while the image data output by the camera presents the 2D visual outlines of these objects.

[0061] The 3D point cloud data and image data corresponding to the same sampling time can be acquired at the same time or acquired at different times but transformed to the same time. The core feature of 3D point clouds is their 3D spatial distribution, used to describe the three-dimensional structure of the real world. Each point contains 3D spatial coordinates (x, y, z), and some points may also contain additional information such as color and reflection intensity. These points are acquired by 3D sensors such as LiDAR and depth cameras to reflect the 3D shape of objects in the scene. Image data is 2D visual information acquired by a camera, containing the color, texture, and 2D outline of objects.

[0062] Step 104: Convert the image data into a set of 2D masks, each 2D mask identifying the outline region of an object contained in the image data.

[0063] For example, the system inputs camera images into a pre-trained visual base model, such as Segment AnythingModel (SAM), to generate three 2D masks, which correspond to a car, a pedestrian, and a section of guardrail, respectively. Each mask marks the outline of the object in the form of a binary image (the pixel value inside the mask is 1, and the background is 0).

[0064] In this context, the number of objects in the image equals the number of 2D masks. Each 2D mask is a binary image representing the outline of a single object in the image. Objects (1) and background (0) are distinguished by pixel values. Converting image data into 2D masks can be achieved using a pre-trained visual basic model, which supports automatic recognition of all objects in the image and generation of 2D masks without manual annotation, adapting to diverse scenarios. In addition to using a visual basic model to generate 2D masks, edge detection algorithms can be applied to identify edges in the image, and then contour tracking technology can be used to determine the boundaries of objects, or superpixel segmentation and other methods can be used. This disclosure does not impose any limitations on these methods.

[0065] Step 106: Transform the pixel coordinates of each 2D mask into the 3D point cloud coordinate system to determine the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system.

[0066] For example, the system uses the camera's internal parameters (including focal length, principal point position, etc.) and external parameters (describing the camera's position and orientation relative to other sensors or the world coordinate system) to transform three 2D masks corresponding to a car, a pedestrian, and a guardrail from the pixel coordinate system to the 3D point cloud coordinate system, thus obtaining the view frustum space corresponding to each mask.

[0067] The view frustum space refers to a three-dimensional region defined by the camera position, mask boundary points, and depth information. It represents the spatial extent that an object corresponding to the 2D mask might occupy in 3D space. To accurately define this region, in addition to relying on camera intrinsic and extrinsic parameters, depth estimation techniques can be incorporated. For example, learning-based methods, such as convolutional neural networks, can be used to predict the depth value of each pixel in the image, or hardware devices such as structured light and time-of-flight can be used to directly measure the depth. Once the depth information of each pixel in the 2D mask is obtained, the 2D mask can be converted into a set of points in 3D space using the inverse perspective projection transformation formula, and the 3D point cloud data located within this view frustum space can be further filtered out.

[0068] Step 108: For each 2D mask, segment the 3D point cloud data located in the view frustum space corresponding to the mask in the 3D point cloud data to determine the point cloud block where the object is located.

[0069] For example, the system defines a view cone space encompassing a car based on its 2D mask. The x-axis range is 5 to 15 meters (representing distance along the road), the y-axis range is -2 to 2 meters (representing lateral offset), and the z-axis range is 0 to 1.5 meters (representing height). The system then segments the 3D point cloud data within this view cone space. All 3D point cloud data falling within this range are categorized as point cloud blocks corresponding to the car. Similarly, using 2D masks of pedestrians and guardrails and the view cone space, the system can segment the corresponding pedestrian and guardrail point cloud blocks, thus completing the 3D point cloud segmentation for that scene.

[0070] By filtering 3D point cloud data that falls within the view frustum space, the system can accurately extract 3D point cloud blocks corresponding to 2D masks. Furthermore, efficient indexing structures, such as KD trees or voxel hashing, can be used to accelerate the point cloud query process. These methods enable rapid searching of points within specific regions from large amounts of point cloud data, significantly improving processing efficiency.

[0071] In one embodiment shown, converting the image data into a set of 2D masks, each 2D mask identifying the outline region of an object contained in the image data, includes: adjusting the visual base model to a mask generation mode; inputting the image data into the visual base model to obtain a set of 2D masks output by the visual base model; wherein each 2D mask corresponds to a binary image, the binary image identifying the outline region of an object contained in the image data.

[0072] For example, a vehicle's front-facing camera captures data about the road ahead. The camera outputs an image showing the 2D visual outlines of objects on the road, such as vehicles, pedestrians, and guardrails. This image is then input into a pre-trained visual baseline model (such as the Segment Anything Model, SAM), which is configured for mask generation. The SAM model analyzes the input image through its internal mechanisms and automatically identifies and segments different objects, generating a set of 2D masks. Each 2D mask is a binary image that precisely marks the outline region of a specific object in the image. For instance, for a car, the SAM model generates a corresponding 2D mask, which is a binary image where areas with a pixel value of 1 represent the car's location, while areas with a pixel value of 0 represent the background or other non-target objects. The same process applies to other objects, such as pedestrians and guardrails, thus forming an independent 2D mask for each detected object.

[0073] The visual foundation model (SAM) refers to a class of deep learning models pre-trained on large-scale, diverse datasets. These models possess strong generalization capabilities and can be used for various visual tasks without requiring extensive additional training. The SAM model incorporates a mask encoder and a cue decoder. The former extracts image features, while the latter generates corresponding segmentation masks based on user-provided cues (such as points and boxes). This design enables the SAM model to accurately segment various objects in an image even without explicit annotations.

[0074] In one embodiment shown, the step of transforming the pixel coordinates of each 2D mask to a 3D point cloud coordinate system to determine the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system includes: for each 2D mask, extracting its boundary pixel coordinate set; transforming the boundary pixel coordinate set to a ray direction vector in the image sensor coordinate system according to the intrinsic parameter matrix of the image sensor; transforming the ray direction vector to the 3D point cloud coordinate system according to the extrinsic parameter matrix from the image sensor to the point cloud sensor, and connecting the transformed ray direction vector with the optical center of the image sensor as the vertex to obtain the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system.

[0075] For example, the system extracts the boundary pixels (such as the coordinates of the top left corner (100, 200) and bottom right corner (300, 400) of the contour edge) from a 2D mask of a car. Then, using the camera intrinsic parameter matrix, these pixel coordinates are converted into ray direction vectors in the camera coordinate system. These rays represent straight lines originating from the camera's optical center and pointing to the various boundaries of the car's contour. Specifically, each pixel corresponds to a ray originating from the camera's optical center, and the direction of this ray is calculated by the intrinsic parameter matrix. Next, using the extrinsic parameter matrix between the LiDAR and the camera (rotation and translation parameters describing their relative position and orientation), these ray direction vectors are transformed into the LiDAR's 3D point cloud coordinate system. This process considers the position and orientation of the camera (image sensor) relative to the LiDAR (point cloud sensor), ensuring that the ray direction vectors can be accurately located in the point cloud coordinate system. Finally, using the camera's optical center as the vertex, all the transformed ray direction vectors are connected to form a view frustum that encloses the car. This view frustum defines the region in the 3D point cloud data that corresponds to the 2D mask.

[0076] The image sensor includes, but is not limited to, cameras and depth cameras, while the point cloud sensor includes, but is not limited to, LiDAR and 3D scanners. The intrinsic parameter matrix describes the camera's internal geometry, primarily including focal length and principal point coordinates, and is used to convert pixels in a 2D image into direction vectors in 3D space. Using the intrinsic parameter matrix, the system can calculate the direction of each ray, i.e., the direction emanating from the camera's optical center and passing through a specific pixel on the image plane. The extrinsic parameter matrix describes the position and orientation of the image sensor relative to the point cloud sensor. Using the extrinsic parameter matrix, the system can accurately map objects in the image sensor to their corresponding positions in the 3D point cloud coordinate system.

[0077] In one embodiment shown, the method is applied to a system that maintains a voxel database. The method further includes: searching the voxel database for point cloud blocks containing historical objects that overlap with each segmented object, based on the voxel coordinates of the point cloud block containing each segmented object; for each segmented object, calculating the correlation between the point cloud block containing the object and each point cloud block containing historical objects that overlap with it, and then updating the point cloud block containing the object to the voxel database.

[0078] For example, the system uses a 3D segmentation model to identify and segment multiple objects in the current scene, such as cars, pedestrians, and guardrails, and determines the corresponding point cloud blocks for each object. Then, based on the voxel coordinates of these point cloud blocks, the system searches a pre-maintained voxel database for historical objects that overlap with them. Specifically, if a car is detected in the current frame, the system searches the voxel database for objects in previous frames that overlap with the car's location. For each found historical object, the system calculates the correlation between the current car point cloud block and the historical object point cloud block. The correlation can be comprehensively evaluated using multiple factors, including spatial overlap rate, semantic feature similarity, and geometric feature similarity. Finally, based on whether the current car has a high correlation with a historical object, the system determines whether the car is an observation of the same historical object at different times, and updates the information of the current car point cloud block to the corresponding historical object record in the voxel database. This process ensures that the system can continuously track and update the state of various objects in the environment.

[0079] A voxel database is a data structure used to store information about objects in 3D space. It divides the space into several small cubes (i.e., voxels) and manages and queries the data within each voxel through an indexing mechanism. Voxel databases can be implemented based on hash tables or octrees to optimize storage and retrieval efficiency. Spatial overlap rate refers to the degree of overlap between two point cloud blocks in voxel space, usually measured by the intersection-union ratio (IU). Semantic feature similarity is typically achieved by extracting feature vectors from two point cloud blocks and measuring the closeness of the two feature vectors using cosine similarity, helping to distinguish different types of objects (even if there is spatial overlap). Geometric feature similarity assesses the similarity of point cloud blocks by analyzing their local geometric structure. A common approach is as follows: For two point cloud blocks to be compared, calculate the FPFH (Fast Point Feature Histograms) descriptor for each point within the block; then, aggregate the FPFH descriptors for all points within each block (e.g., calculate the average of all descriptor vectors to obtain a global feature vector representing the point cloud block); finally, measure the geometric similarity between the two point cloud blocks by comparing the similarity between the two global feature vectors.

[0080] To facilitate a better understanding of this solution by those skilled in the art, the following is combined with... Figure 2 This paper introduces the 3D point cloud segmentation and object management method disclosed herein. Please refer to [link / reference]. Figure 2 , Figure 2 This is a flowchart illustrating another 3D point cloud segmentation method as an exemplary embodiment. (As shown...) Figure 2 As shown, the system first receives 3D point cloud data and associated 2D image data from sensors (such as LiDAR and cameras). Next, the system processes the 2D image data using a semantic segmentation network or a visual baseline model to generate a 2D mask for each object in the image, representing the position and boundaries of each object. Then, based on the generated 2D mask, the system calculates the position and extent of each object in 3D space, i.e., the view frustum space. This process is achieved by back-projecting 2D pixel coordinates into 3D space, providing spatial constraints for subsequent 3D point cloud segmentation. Further, the system uses the view frustum space to segment objects in the 3D point cloud, obtaining individual object point cloud blocks. The system can then perform historical object matching in a voxel database based on these object point cloud blocks and dynamically update the maintained voxel database. The voxel database stores and manages the historical state information of all known objects in the scene; by continuously updating the voxel database, the system can maintain real-time tracking of object states in the scene. Finally, the system outputs the segmentation results of each object point cloud block in the current frame, which can be used for subsequent advanced tasks such as path planning and behavior prediction.

[0081] In this embodiment, the introduction of a voxel database enables continuous maintenance and efficient querying of object states in a 3D scene, improving the coherence of multi-frame data tracking and effectively mitigating target switching and loss issues caused by occlusion, temporary disappearance, or changes in viewpoint. By calculating correlation, the accuracy and robustness of object matching are enhanced, enabling the system to more reliably distinguish between newly appearing objects and historical objects, and achieving accurate data updates to the voxel database.

[0082] In one embodiment, the step of calculating the correlation between the point cloud block containing the object and the point cloud blocks containing each historical object that overlaps with it, and then adding the point cloud block containing the object to the voxel database for each segmented object, includes: for each segmented object, calculating the correlation between the point cloud block containing the object and the point cloud blocks containing each historical object that overlaps with it; if the correlation between the point cloud block containing the object and the point cloud block containing a target historical object that overlaps with it meets a preset condition, then the point cloud block containing the object and the point cloud block containing the target historical object are merged in the voxel database; if the correlation between the point cloud block containing the object and the point cloud blocks containing each target historical object that overlaps with it does not meet the preset condition, then the point cloud block containing the object is added to the voxel database.

[0083] For example, the system segments a point cloud block of a car in the current frame and searches for a point cloud block of a historical car in the voxel database based on its voxel coordinates. The two blocks partially overlap spatially. The system calculates the correlation between the current car and the historical car, which involves spatial overlap rate, semantic feature similarity, and geometric feature similarity. If the calculated spatial overlap rate exceeds 0.5 and the geometric feature similarity exceeds 0.8, satisfying a preset condition, the system determines that the current car is an observation of a historical car in the new frame. Therefore, it merges the current car's point cloud block with the historical car's record, updating the car's position, point cloud density, and timestamp. Conversely, if the system detects a completely new pedestrian, and the correlation between the pedestrian's point cloud block and all historical objects in the database is less than 0.8, it determines that the pedestrian is a newly appearing object and performs an addition process, adding a new entry for "pedestrian" to the voxel database.

[0084] Each entry typically represents an observed object in the voxel database. Each entry includes, but is not limited to, an entry ID, current / historical point cloud data, a timestamp, motion state (position, velocity), and semantic tags. The voxel database can use a hash table or a KD-tree as its underlying data structure to enable fast lookup of specific spatial regions (i.e., voxel coordinates), thereby efficiently finding historical objects that spatially overlap with the current point cloud block. In this embodiment, the purpose of the addition processing is to initialize a newly detected object. The system can create a new entry in the voxel database, assign it a unique ID, and store information such as the current frame's point cloud block, initial timestamp, and semantic tags. This newly detected object will be used as a historical object for association and matching in subsequent frames. The purpose of the merging processing is to update and enrich long-term observations of the same object. A simple approach is to directly stitch the point clouds together; advanced approaches can perform voxel filtering (removing duplicate points), weighted averaging (smoothing points in overlapping areas), or surface reconstruction to improve point cloud quality. Simultaneously, the object's latest position, timestamp, and other parameters are updated.

[0085] In one embodiment shown, each 2D mask has a predicted intersection-union score during generation. The method further includes: regarding historical objects whose correlation with the point cloud block containing any segmented object meets a preset condition as active objects in the voxel database; summing the intersection-union scores of the 2D masks of the segmented objects corresponding to each active object; and dividing the summed intersection-union score by the total number of active objects to obtain the confidence score of the 3D point cloud segmentation result.

[0086] For example, an autonomous vehicle is passing through a city intersection. LiDAR and cameras work synchronously, acquiring 3D point cloud data and image data respectively. In the current frame, the visual baseline model identifies five objects from the image data: two pedestrians, a bicycle, and two cars. It generates a 2D mask for each object and a corresponding predicted intersection-union (IU) score. The IU scores for the five objects are 0.85, 0.90, 0.75, 0.80, and 0.92. Next, the system back-projects the pixel coordinates of these 2D masks onto the 3D point cloud coordinate system, determines the view frustum of each 2D mask, and then filters out the point cloud blocks located within the corresponding view frustums, obtaining the point cloud blocks corresponding to each of the five objects. Assuming that the voxel database has accumulated information about these objects in the past few frames, the system can use this historical data to determine whether the point cloud blocks of potential objects in the current frame are associated with objects in the historical records of the voxel database, and mark historical objects with association scores meeting preset conditions as active objects. Assuming there are 6 historical objects in the voxel database, 5 of which are active objects (two pedestrians, one bicycle, and two cars), the system calculates the sum of the intersection-union scores (IU / R) of the 2D masks of the 5 active objects identified in the image data: 0.85 + 0.90 + 0.75 + 0.80 + 0.92 = 4.22. Dividing this by the number of active objects (5), the final 3D point cloud segmentation result has a confidence score of 0.844. This indicates that the 3D point cloud segmentation of active objects in this image frame is generally effective.

[0087] The 2D mask prediction intersection-over-union score refers to the prediction intersection-over-union score output by the visual base model when generating each 2D mask. It measures the degree of overlap between the mask and the outline of the real object (range 0-1), with a higher score indicating higher mask accuracy. Active objects refer to historical objects in the current image data frame that have been successfully matched in the voxel database. The 3D point cloud segmentation result confidence score is a metric used to evaluate the reliability of the 3D point cloud segmentation result based on the current image data frame.

[0088] In one embodiment shown, the system also maintains a 3D segmentation model, which is used to segment point cloud blocks containing objects in the 3D point cloud data based on the 2D mask. The method further includes: if the segmentation quality score is lower than a first preset threshold, then the active objects associated with the point cloud blocks containing objects corresponding to the 2D masks whose intersection-over-union scores exceed a second threshold, and the point cloud blocks containing the associated active objects, are used as pseudo-labels; based on the pseudo-labels, the lightweight trainable sub-module of the 3D segmentation model is fine-tuned online.

[0089] For example, in an autonomous driving scenario, suppose the system sets a first preset threshold of 0.75 for the confidence level of the 3D point cloud segmentation result and a second threshold of 0.9 for the intersection-union ratio (IU) score of the 2D masks. At a certain moment, when the vehicle passes through a tunnel, the image quality deteriorates due to a sudden change in illumination. The system calculates the confidence level of the current 3D point cloud segmentation result to be 0.72 (below 0.75), triggering an online adjustment mechanism. At this time, among the five 2D masks generated by the visual base model, the IU scores are 0.88, 0.93, 0.79, 0.91, and 0.85, respectively. Among them, the scores exceeding the second threshold (0.9) are 0.93 (corresponding to the car in front) and 0.91 (corresponding to the pedestrian on the right). The system determines that the point cloud blocks corresponding to these two 2D masks are associated with active objects "Car A" and "Pedestrian B" in the voxel database, respectively. Therefore, "Car A" and its point cloud blocks, and "Pedestrian B" and its point cloud blocks are used as pseudo-labels. These pseudo-labels are then used to refine the point cloud features of the lightweight PointNet++ submodule (containing trainable parameters) in the 3D segmentation model. Online fine-tuning was performed: the loss function was used to calculate the loss, the learning rate was set to 5e-5, and the parameters were updated after 3 iterations. After fine-tuning, the segmentation quality score of the next frame improved to 0.81, and the segmentation accuracy of cars and pedestrians was significantly improved.

[0090]

[0091] in, These are the updated and current trainable parameters. For online fine-tuning of the learning rate, . The loss function is calculated based on pseudo-labels. The gradient of the loss function calculated based on pseudo-labels. The 3D object segmentation result for the current image data frame is a collection of all active objects in the voxel database. This is the 3D point cloud collected at the current moment. This is a pseudo-tag.

[0092] In this embodiment, the loss function can actually be either Dice Loss or Lovasz-Softmax Loss. The core logic of online fine-tuning of the 3D segmentation model is to update the trainable parameters so that the output of the 3D segmentation model is closer to the pseudo-label. Fine-tuning lightweight trainable sub-modules can help the 3D segmentation model quickly adapt to changing environmental conditions without retraining the entire model. This reduces the demand for computing resources and time consumption, which is especially important for applications requiring real-time response, such as autonomous driving. Furthermore, it enables the 3D segmentation model to run efficiently on resource-constrained devices, demonstrating strong adaptability.

[0093] To facilitate a better understanding of this solution by those skilled in the art, the following is combined with... Figure 3 The online adjustment process for the 3D segmentation model disclosed herein is described. Please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram illustrating an online adjustment of a 3D segmentation model, as an exemplary embodiment. (As shown...) Figure 3 As shown, after the 3D segmentation model in the vehicle system completes 3D point cloud segmentation, it evaluates the confidence level of the 3D point cloud segmentation result for the current image frame. Next, the system determines whether the confidence level is lower than a first threshold. If the confidence level is not lower than the first threshold, the current 3D segmentation model is used to continue processing subsequent frames; otherwise, if the confidence level is lower than the first threshold, the 3D segmentation model needs to be fine-tuned online to optimize its performance. After fine-tuning, an updated 3D segmentation model is obtained, and the updated 3D segmentation model is used to continue processing data for subsequent frames. This process ensures that the system can adjust in a timely manner when faced with low-confidence segmentation results from the 3D segmentation model, thereby improving the system's adaptability and segmentation accuracy.

[0094] Corresponding to the embodiments of the above-described 3D point cloud segmentation method, this disclosure also provides an embodiment of a 3D point cloud segmentation device.

[0095] Please see Figure 4 , Figure 4 This is an exemplary embodiment illustrating the hardware structure of an electronic device. At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, memory 408, and non-volatile memory 410, and may also include other necessary hardware. One or more embodiments of this disclosure can be implemented in software, for example, the processor 402 reads the corresponding computer program from the non-volatile memory 410 into memory 408 and then runs it. Of course, besides software implementation, one or more embodiments of this disclosure do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0096] Please see Figure 5 , Figure 5 This is a block diagram illustrating an exemplary embodiment of a 3D point cloud segmentation apparatus 500. This 3D point cloud segmentation apparatus 500 can be applied to, for example... Figure 4 The illustrated electronic device is used to implement the technical solution of this disclosure. The device includes:

[0097] The acquisition unit 502 is used to acquire 3D point cloud data and image data corresponding to the same sampling time and the same scene;

[0098] The first conversion unit 504 is used to convert the image data into a set of 2D masks, each 2D mask identifying the outline region of an object contained in the image data;

[0099] The second transformation unit 506 is used to transform the pixel coordinates of each 2D mask to a 3D point cloud coordinate system, thereby determining the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system.

[0100] The segmentation unit 508 is used to segment the 3D point cloud data located in the view frustum space corresponding to the mask in the 3D point cloud data for each 2D mask, so as to determine the point cloud block where the object is located.

[0101] In some embodiments, converting the image data into a set of 2D masks, each 2D mask identifying the outline region of an object contained in the image data, includes:

[0102] Adjust the visual base model to mask generation mode;

[0103] The image data is input into the visual base model to obtain a set of 2D masks output by the visual base model;

[0104] Each 2D mask corresponds to a binary image, which identifies the outline region of an object contained in the image data.

[0105] In some embodiments, the step of transforming the pixel coordinates of each 2D mask to a 3D point cloud coordinate system, thereby determining the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system, includes:

[0106] For each 2D mask, extract its boundary pixel coordinate set;

[0107] Based on the intrinsic parameter matrix of the image sensor, the boundary pixel coordinate set is transformed into a ray direction vector in the image sensor coordinate system;

[0108] Based on the extrinsic parameter matrix from the image sensor to the point cloud sensor, the ray direction vector is transformed into a 3D point cloud coordinate system, and the transformed ray direction vector is connected with the optical center of the image sensor as the vertex to obtain the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system.

[0109] In some embodiments, the method is applied to a system that maintains a voxel database, and the apparatus further includes:

[0110] The lookup unit 510 is used to search for the point cloud block where the historical object with which it overlaps is located in the voxel database based on the voxel coordinates of the point cloud block where each object is located.

[0111] The first calculation unit 512 is used to calculate the correlation between the point cloud block where the object is located and the point cloud block where each historical object with overlapping parts is located for each segmented object, and then update the point cloud block where the object is located to the voxel database.

[0112] In some embodiments, for each segmented object, calculating the correlation between the point cloud block containing the object and the point cloud blocks containing each historical object that overlaps with it, and then adding the point cloud block containing the object to the voxel database, includes:

[0113] For each segmented object, calculate the correlation between the point cloud block containing the object and the point cloud block containing each historical object that overlaps with it.

[0114] If the correlation between the point cloud block containing the object and the point cloud block containing a target historical object that overlaps with it meets a preset condition, then the point cloud block containing the object and the point cloud block containing the target historical object are merged in the voxel database.

[0115] If the correlation between the point cloud block containing the object and the point cloud blocks containing each target historical object that overlaps with it does not meet the preset conditions, then the point cloud block containing the object will be added to the voxel database.

[0116] In some embodiments, each 2D mask has a predicted intersection-union score at the time of generation, and the apparatus further includes:

[0117] The determining unit 514 is used to regard the historical object whose correlation between the point cloud block where any segmented object is located and the point cloud block where the historical object is located meets a preset condition as an active object in the voxel database.

[0118] The second calculation unit 516 is used to sum the intersection-union scores of the 2D masks of the segmented objects corresponding to each active object, and divide the summed intersection-union scores by the total number of active objects to obtain the confidence score of the 3D point cloud segmentation result.

[0119] In some embodiments, the system further maintains a 3D segmentation model, which is used to segment the point cloud block containing the object in the 3D point cloud data according to the 2D mask. The apparatus further includes:

[0120] The tag unit 518 is used to associate the active object of the point cloud block where the 2D mask corresponding to the object with the intersection-union ratio score exceeding the second preset threshold, and the point cloud block where the active object is located, as pseudo-tags if the confidence of the 3D point cloud segmentation result is lower than the first preset threshold.

[0121] The fine-tuning unit 520 is used to perform online fine-tuning of the lightweight trainable sub-modules of the 3D segmentation model based on the pseudo-labels.

[0122] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0123] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0124] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0125] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0126] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0127] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0128] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points shall be provided for users to choose to authorize or refuse.

[0129] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0130] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0131] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0132] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this disclosure, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0133] The above description is merely a preferred embodiment of one or more embodiments of this disclosure and is not intended to limit the scope of one or more embodiments of this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this disclosure should be included within the scope of protection of one or more embodiments of this disclosure.

Claims

1. A 3D point cloud segmentation method, characterized in that, The method is applied to a system that maintains a voxel database storing historical objects and a 3D segmentation model. The 3D segmentation model is used to segment the point cloud block containing the object from the 3D point cloud data based on a 2D mask. The method includes: Acquire 3D point cloud data and image data corresponding to the same sampling time and the same scene; The image data is converted into a set of 2D masks, each 2D mask identifying the outline region of an object contained in the image data, and having a predicted intersection-union score at the time of generation; The pixel coordinates of each 2D mask are transformed into the 3D point cloud coordinate system, thereby determining the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system; For each 2D mask, the 3D point cloud data located in the view frustum space corresponding to the mask is segmented from the 3D point cloud data to determine the point cloud block where the object is located and the confidence level of the 3D point cloud segmentation result. If the confidence level of the 3D point cloud segmentation result is lower than the first preset threshold, then the historical object associated with the point cloud block where the 2D mask corresponding to the object with the intersection-union ratio score exceeds the second preset threshold, and the point cloud block where the historical object is located, are used as pseudo-labels. Based on the pseudo-labels, the lightweight trainable submodules of the 3D segmentation model are fine-tuned online.

2. The method according to claim 1, characterized in that, The step of converting the image data into a set of 2D masks, each 2D mask identifying the outline region of an object contained in the image data, includes: Adjust the visual base model to mask generation mode; The image data is input into the visual base model to obtain a set of 2D masks output by the visual base model; Each 2D mask corresponds to a binary image, which identifies the outline region of an object contained in the image data.

3. The method according to claim 1, characterized in that, The step of transforming the pixel coordinates of each 2D mask to a 3D point cloud coordinate system, thereby determining the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system, includes: For each 2D mask, extract its boundary pixel coordinate set; Based on the intrinsic parameter matrix of the image sensor, the boundary pixel coordinate set is transformed into a ray direction vector in the image sensor coordinate system; Based on the extrinsic parameter matrix from the image sensor to the point cloud sensor, the ray direction vector is transformed into a 3D point cloud coordinate system, and the transformed ray direction vector is connected with the optical center of the image sensor as the vertex to obtain the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system.

4. The method according to claim 1, characterized in that, The method further includes: Based on the voxel coordinates of the point cloud block where each segmented object is located, search the voxel database for the point cloud block where the historical object with overlapping parts is located. For each segmented object, the correlation between the point cloud block containing the object and the point cloud block containing each historical object that overlaps with it is calculated, and then the point cloud block containing the object is updated to the voxel database.

5. The method according to claim 4, characterized in that, For each segmented object, the correlation between the point cloud block containing the object and the point cloud blocks containing each historical object that overlaps with it is calculated, and then the point cloud block containing the object is added to the voxel database, including: For each segmented object, calculate the correlation between the point cloud block containing the object and the point cloud block containing each historical object that overlaps with it. If the correlation between the point cloud block containing the object and the point cloud block containing the target historical object that overlaps with it meets the preset conditions, then the point cloud block containing the object and the point cloud block containing the target historical object are merged in the voxel database. If the correlation between the point cloud block containing the object and the point cloud blocks containing each target historical object that overlaps with it does not meet the preset conditions, then the point cloud block containing the object will be added to the voxel database.

6. The method according to claim 4, characterized in that, The determination of the confidence level of the 3D point cloud segmentation result includes: The historical object whose correlation with the point cloud block containing any segmented object meets the preset condition is regarded as an active object in the voxel database. The intersection-union scores of the 2D masks of the segmented objects corresponding to each active object are summed, and the summed intersection-union scores are divided by the total number of active objects to obtain the confidence score of the 3D point cloud segmentation result.

7. A 3D point cloud segmentation device, characterized in that, The device maintains a voxel database storing historical objects and a 3D segmentation model. The 3D segmentation model is used to segment the point cloud block containing the object from the 3D point cloud data based on a 2D mask. The device includes: The acquisition unit is used to acquire 3D point cloud data and image data corresponding to the same sampling time and the same scene; The first conversion unit is used to convert the image data into a set of 2D masks, each 2D mask identifying the outline region of an object contained in the image data, and having a predicted intersection-union score at the time of generation; The second transformation unit is used to transform the pixel coordinates of each 2D mask to the 3D point cloud coordinate system, thereby determining the view frustum space corresponding to each 2D mask in the 3D point cloud coordinate system. A segmentation unit is used to segment 3D point cloud data located in the view frustum space corresponding to each 2D mask from the 3D point cloud data, so as to determine the point cloud block where the object is located and the confidence of the 3D point cloud segmentation result. The fine-tuning unit is used to associate the historical object of the point cloud block where the 2D mask corresponding to the object with the intersection-union ratio score exceeding the second preset threshold, and the point cloud block where the historical object is located, with the historical object as pseudo-labels if the confidence of the 3D point cloud segmentation result is lower than the first preset threshold. The fine-tuning unit is also used to perform online fine-tuning of the lightweight trainable sub-modules of the 3D segmentation model based on the pseudo-labels.

8. An electronic device, characterized in that, It includes a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus; The memory stores machine-readable instructions, and the processor executes the method according to any one of claims 1 to 6 by invoking the machine-readable instructions.

9. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-readable instructions, which, when invoked and executed by a processor, implement the method described in any one of claims 1 to 6.

Images