3D point cloud semantic segmentation method and system based on adaptive spatiotemporal aggregation enhancement

Abstract

The present invention discloses a 3D point cloud semantic segmentation method and system based on adaptive spatiotemporal aggregation enhancement, which belongs to the field of autonomous driving and environmental perception technology. The method includes: performing spatiotemporal mixing enhancement on multi-frame point clouds based on the laser radar beam structure, and improving the feature expression of sparse areas by odd-even beam combination; designing a category-adaptive frame fusion mechanism, dynamically adjusting the historical frame fusion length according to the target recognition difficulty, and enhancing the small target segmentation capability; combining the point cloud density-aware distillation strategy, using fine-grained feature alignment in high-density areas and global semantic compensation in low-density areas to optimize the generalization of the model; generating pseudo-voxel features through multi-frame images, and using trilinear interpolation and spatiotemporal alignment to achieve efficient fusion of image-point clouds. This method significantly improves the accuracy and robustness of semantic segmentation in dynamic scenes, and can be widely used in autonomous driving, robot navigation and other fields, with high practical value and industrial implementation potential.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving and environmental perception technology, and specifically relates to a 3D point cloud semantic segmentation method and system based on adaptive spatiotemporal aggregation enhancement. Background Art

[0002] As a core technology in the field of computer vision and environmental perception, 3D point cloud semantic segmentation plays an important role in scenarios such as autonomous driving and robot navigation. Compared with two-dimensional images, 3D point cloud data is sparse, unstructured, and has complex spatial distribution. Especially in open outdoor scenes, problems such as blurred target boundaries, sparse point clouds at long distances, and target motion in dynamic environments further increase the difficulty of semantic segmentation. Traditional methods rely on regular projection or voxel processing, which makes it difficult to effectively capture complex geometric structures, and the accuracy is significantly reduced in occluded scenes and long-distance target recognition. Although deep learning models (such as PointNet and SparseConv) have promoted the improvement of single-frame point cloud segmentation performance, their insufficient utilization of multi-frame spatiotemporal information has limited generalization capabilities in dynamic scenes.

[0003] In existing technologies, multimodal fusion has become a key direction for improving segmentation accuracy, but practical applications still face many challenges. For example, due to the hardware frequency difference between lidar and cameras (10Hz for lidar vs. 30Hz / 60Hz for cameras), timestamp alignment is difficult, and direct fusion is prone to introducing spatial deviations. In addition, existing fusion methods often use simple splicing or weighting strategies, failing to fully exploit the complementarity between point cloud geometry and image texture features, thus limiting the modal synergy effect. Patent CN119296079A proposes a real-time semantic segmentation solution based on a single-frame point cloud. While improving detection efficiency by improving the network structure and post-processing modules, it does not incorporate multi-frame spatiotemporal information, making it difficult to address the problems of missed target detection and incomplete boundaries in dynamic scenes. Furthermore, it lacks deep fusion of multimodal data.

[0004] On the other hand, while existing data augmentation methods (such as windowing and feature stacking) attempt to leverage temporal information, they fail to optimize for point cloud density distribution and object category differences, resulting in inefficient resource allocation. For example, small objects (such as pedestrians) are easily missed due to point cloud sparsity, while redundant information about large objects (such as vehicles) can increase the computational burden. Furthermore, knowledge distillation techniques lack differentiated strategies for application in multiple scenarios, making it difficult to balance detail preservation in high-density areas with global semantic compensation in low-density areas.

[0005] In summary, existing technologies have significant shortcomings in multi-frame spatiotemporal information fusion, multimodal alignment optimization, category adaptive enhancement, and density-aware distillation. Summary of the Invention

[0006] The purpose of the embodiments of the present invention is to provide a 3D point cloud semantic segmentation method and system based on adaptive spatiotemporal aggregation enhancement, which aims to improve the segmentation accuracy and system robustness in complex scenes by dynamically fusing multi-frame point clouds and image information and differentially allocating computing resources, thereby solving at least one technical problem involved in the background technology.

[0007] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0008] The embodiment of the present invention provides a 3D point cloud semantic segmentation method based on adaptive spatiotemporal aggregation enhancement, comprising the following steps:

[0009] Step S1, obtaining the current frame lidar point cloud and the historical frame point cloud sequence to construct a multi-frame point cloud dataset;

[0010] Step S2, performing spatiotemporal adaptive enhancement on the multi-frame point cloud dataset, specifically comprising:

[0011] Step S21, dividing each frame of point cloud according to the laser radar beam number or tilt angle, and spatially mixing the odd-numbered beams and even-numbered beams of the previous and next frame point clouds;

[0012] Step S22: setting an adaptive threshold based on the confidence of the category segmentation, allocating more historical frames to the low-confidence category for temporal enhancement, and generating an enhanced point cloud;

[0013] Step S23: Statistically enhance the local voxel density of the point cloud, and select fine-grained distillation or global distillation strategy to optimize feature expression according to the density threshold;

[0014] Step S3, synchronously acquiring multiple frames of image data, extracting image semantic features and projecting them into 3D space to generate pseudo voxel features;

[0015] Step S4, spatially aligning and fusing the enhanced point cloud features with the pseudo voxel image features, and outputting the fused features;

[0016] In step S5, the semantic label of each point is output through the segmentation network based on the fused features to complete the 3D semantic segmentation task.

[0017] Optionally, in step S21, the specific method of the spatial mixing is: alternately combining the odd-numbered line bundles and the even-numbered line bundles of the point clouds of the previous and next frames by region, so that the point cloud data of any adjacent regions come from different frames.

[0018] Optionally, in step S22, the category segmentation confidence is calculated by intersection over union (IoU), low confidence categories include pedestrians or cone-shaped obstacles, and high confidence categories include cars or trucks.

[0019] Optionally, in step S23, the fine-grained distillation extracts local structural features through the teacher network and aligns them with the student network features; the global distillation optimizes the overall features of the low-density area through semantic compensation.

[0020] Optionally, in step S3, the specific steps of generating pseudo voxel features include:

[0021] Extract semantic feature maps of multiple frames through the image backbone network;

[0022] The image feature map is projected into the point cloud coordinate system using the calibrated extrinsic parameters to generate 3D pseudo voxel features;

[0023] The pseudo voxel features are temporally fused to form a multi-frame enhanced pseudo voxel feature network.

[0024] Optionally, in step S4, the feature fusion uses trilinear interpolation to align the point cloud and pseudo voxel features, and realizes multimodal fusion through an attention mechanism or feature splicing.

[0025] Optionally, in step S5, the loss function of the segmentation network is a weighted sum of cross entropy loss and knowledge distillation loss, which is used for the joint training model.

[0026] The present invention also provides a 3D point cloud semantic segmentation system based on adaptive spatiotemporal aggregation enhancement for implementing the method, comprising:

[0027] Perception hardware module, used to synchronously collect multi-frame point cloud and image data;

[0028] The computing module is used to complete data preprocessing, feature extraction, data enhancement, feature fusion and predictive reasoning;

[0029] System control module, used to coordinate data synchronization, task scheduling and exception monitoring.

[0030] Optionally, the perception hardware module includes:

[0031] LiDAR unit, used to collect multi-frame 3D point cloud data in real time;

[0032] The camera unit synchronously acquires multiple frames of image data.

[0033] Optionally, the computing module includes a multi-frame point cloud enhancement submodule, a density-aware distillation submodule, an image pseudo-voxel feature generation submodule, a multimodal feature fusion submodule, and a semantic segmentation reasoning submodule, wherein:

[0034] The multi-frame point cloud enhancement submodule acquires continuous frame point clouds based on the time sequence, combines the beams in an orderly manner according to the physical structure of the laser beam, performs category-adaptive frame fusion according to the difficulty of category recognition, improves the feature expression ability of difficult-to-classify targets, and uses the point cloud backbone network to extract the enhanced point cloud features;

[0035] The density-aware distillation submodule analyzes the local density of the point cloud, uses fine-grained feature distillation for high-density areas, and uses global feature distillation for low-density areas;

[0036] The image pseudo voxel feature generation submodule uses the image backbone network to extract image features, generates pseudo voxel features based on multiple frames of images, and applies a spatiotemporal enhancement strategy to the pseudo features;

[0037] The multimodal feature fusion submodule establishes spatial correspondence based on the calibration relationship between the point cloud and the image, and aligns and fuses the point cloud features with the image pseudo-voxel features;

[0038] The semantic segmentation reasoning prediction submodule receives the fused features and outputs point semantic labels through the segmentation head to complete the 3D semantic segmentation task.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. Through the spatiotemporal fusion of multi-frame point clouds and orderly mixing at the laser beam level, the redundant information and temporal complementarity between consecutive frames are effectively utilized to enhance the structural integrity and feature expression capabilities of sparse areas (such as distant or occluded targets), and significantly improve the problems of missed detection of small targets and blurred boundaries in dynamic scenes.

[0041] 2. Combined with the category-adaptive frame fusion mechanism, the number of historical frames fused is dynamically adjusted according to the difficulty of identifying the target category (such as pedestrians, small obstacles such as cones), specifically enhancing the feature learning ability of difficult-to-classify targets. Without increasing redundant calculations, the category specificity and semantic expression accuracy of the segmentation results are improved.

[0042] 3. A density-aware knowledge distillation mechanism is proposed. Based on the local voxel density differences, fine-grained feature distillation is used for high-density areas (to preserve structural details), and global semantic distillation is used for low-density areas (to compensate for information loss). This achieves differentiated supervision, effectively balances the semantic segmentation accuracy of different density areas, and enhances the model's generalization ability in complex scenarios.

[0043] 4. Generate pseudo-voxel features by projecting multiple frames of images, utilize the complementarity of image texture information and point cloud geometric information, combine temporal alignment and trilinear interpolation technology, and accurately map image features to point cloud space, significantly improving the semantic segmentation accuracy of edge details and texture-sensitive targets.

[0044] 5. Adopt attention mechanism or splicing fusion strategy to avoid the alignment bias problem in traditional multimodal fusion and give full play to the synergistic advantages of heterogeneous sensor data.

[0045] 6. The modular system architecture design clearly divides the perception hardware, computing and control units, supports the rapid deployment of multiple heterogeneous platforms (such as autonomous vehicles and intelligent robots), and meets the real-time and stability requirements.

[0046] 7. Through spatiotemporal enhancement and distillation optimization, the model's dependence on hardware resources is significantly reduced, improving engineering implementation capabilities. It is suitable for complex scenarios such as heavy-load vehicle perception in mines and urban autonomous driving.

[0047] 8. Experiments show that the segmentation accuracy of this invention on public datasets (such as SemanticKITTI) is improved by about 5%-8%, especially in the detection of dynamic targets and small obstacles.

[0048] 9. The technical solution is highly scalable and can be seamlessly integrated into existing autonomous driving perception systems, providing reliable support for high-precision environmental understanding, path planning, and decision-making control, and has broad prospects for industrial application.

[0049] In summary, the present invention comprehensively solves the core problems of existing technologies such as weak adaptability to dynamic scenes, multimodal alignment deviation, and uneven density distribution through innovative spatiotemporal aggregation enhancement, density-aware distillation, and multimodal fusion strategies. It significantly improves the accuracy, robustness, and practicality of 3D point cloud semantic segmentation, and provides efficient and reliable technical support for practical applications in fields such as autonomous driving and intelligent robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0051] Figure 1 A flowchart of a 3D point cloud semantic segmentation method based on adaptive spatiotemporal aggregation enhancement provided by an embodiment of the present invention;

[0052] Figure 2 A structural block diagram of a 3D point cloud semantic segmentation system based on adaptive spatiotemporal aggregation enhancement provided by an embodiment of the present invention;

[0053] Figure 3 This is one of the hardware structure diagrams of the electronic device provided by the embodiment of the present invention;

[0054] Figure 4This is the second schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0056] The terms "first," "second," and the like in the specification and claims of the present invention are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects. For example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.

[0057] See Figure 1 As shown, an embodiment of the present invention provides a 3D point cloud semantic segmentation method based on adaptive spatiotemporal aggregation enhancement, comprising the following steps:

[0058] Step S1, obtain the current frame lidar point cloud and historical frame point cloud sequence , build a multi-frame point cloud dataset ;

[0059] Step S2, performing spatiotemporal adaptive enhancement on the multi-frame point cloud dataset, specifically comprising:

[0060] Step S21, dividing each frame of point cloud according to the laser radar beam number or tilt angle, and spatially mixing the odd-numbered beams and even-numbered beams of the previous and next frame point clouds;

[0061] Step S22: Setting an adaptive threshold based on the category segmentation confidence IoU , assign more historical frames to low confidence categories for temporal enhancement and generate enhanced point clouds ;

[0062] Step S23: Count the local voxel density of the enhanced point cloud , according to the density threshold Choose fine-grained distillation or global distillation strategy to optimize feature expression;

[0063] Step S3, synchronously acquiring multiple frames of image data, extracting image semantic features and projecting them into 3D space to generate pseudo voxel features;

[0064] Step S4, spatially aligning and fusing the enhanced point cloud features with the pseudo voxel image features, and outputting the fused features;

[0065] In step S5, the semantic label of each point is output through the segmentation network based on the fused features to complete the 3D semantic segmentation task.

[0066] In step S21, the specific method of the spatial mixing is: the odd-numbered line bundles and the even-numbered line bundles of the previous and next frame point clouds are alternately combined by region, so that the point cloud data of any adjacent region come from different frames.

[0067] In step S22, the category segmentation confidence is calculated by intersection over union (IoU), where low confidence categories include pedestrians or cone-shaped obstacles, and high confidence categories include cars or trucks.

[0068] In step S23, the fine-grained distillation extracts local structural features through the teacher network and aligns them with the student network features; the global distillation optimizes the overall features of the low-density area through semantic compensation.

[0069] It should be noted that if , it belongs to the high-density area and uses fine-grained distillation; if , then it is a low-density area, using global distillation, where represents the voxel area, Indicates the point cloud density of the voxel area.

[0070] In step S3, the specific steps of generating pseudo voxel features include:

[0071] Extract semantic feature maps of multiple frames through image backbone network ;

[0072] Use the calibration extrinsic parameters to project the image feature map into the point cloud coordinate system to generate 3D pseudo voxel features ;

[0073] Temporal fusion of pseudo voxel features to form a multi-frame enhanced pseudo voxel feature network .

[0074] In step S4, the feature fusion uses trilinear interpolation to align the point cloud and pseudo voxel features, and realizes multimodal fusion through attention mechanism or feature splicing, which specifically includes:

[0075] Step S41, establishing a spatial mapping relationship between the point cloud and the pseudo voxel image features through calibration information;

[0076] Step S42, searching for the corresponding image pseudo voxel for each point cloud feature voxel and performing trilinear interpolation;

[0077] Step S43: perform splicing or attention fusion on the point cloud features and image features: .

[0078] In step S5, the loss function of the segmentation network is a weighted sum of the cross entropy loss and the knowledge distillation loss, which is used for the joint training model.

[0079] Step S5 specifically includes:

[0080] Step S51, inputting the fused features into a semantic segmentation network head, such as MLP;

[0081] Step S52: Output the semantic category prediction for each point: ;

[0082] Step S53: The loss function uses cross entropy loss + distillation loss to jointly train the model: ;

[0083] Step S54: Finally output the semantic label of each point , complete the 3D semantic segmentation task.

[0084] Recombination Figure 2 As shown, the present invention also provides a 3D point cloud semantic segmentation system based on adaptive spatiotemporal aggregation enhancement for implementing the method described, including a perception hardware module, a computing module and a system control module.

[0085] The perception hardware module is used to synchronously collect multi-frame point cloud and image data, including:

[0086] LiDAR unit: used to collect multi-frame 3D point cloud data in real time;

[0087] Camera unit: synchronously acquires multiple frames of image data.

[0088] The computing module is used to complete data preprocessing, feature extraction, data enhancement, feature fusion and predictive reasoning; it includes a multi-frame point cloud enhancement submodule, a density-aware distillation submodule, an image pseudo-voxel feature generation submodule, a multimodal feature fusion submodule and a semantic segmentation reasoning submodule;

[0089] Among them, the multi-frame point cloud enhancement submodule obtains continuous frame point clouds based on time sequence, combines the line beams in an orderly manner according to the physical structure of the laser beam, performs category-adaptive frame fusion according to the difficulty of category recognition, improves the feature expression ability of difficult-to-classify targets, and uses the point cloud backbone network to extract the enhanced point cloud features.

[0090] The density-aware distillation submodule analyzes the local density of the point cloud and uses fine-grained feature distillation for high-density areas and global feature distillation for low-density areas.

[0091] The image pseudo voxel feature generation submodule uses the image backbone network to extract image features, generates pseudo voxel features based on multiple frames of images, and applies a spatiotemporal enhancement strategy to the pseudo features.

[0092] The multimodal feature fusion submodule establishes spatial correspondence based on the calibration relationship between the point cloud and the image, and aligns and fuses the point cloud features with the image pseudo-voxel features.

[0093] The semantic segmentation reasoning prediction submodule receives the fused features and outputs point semantic labels through the segmentation head to complete the 3D semantic segmentation task.

[0094] The system control module is used to coordinate data synchronization, task scheduling, and exception monitoring. It includes:

[0095] Communication interface: used for data transmission and synchronization between the sensing hardware module and the computing module;

[0096] Control unit: coordinates data synchronization, task scheduling, process control, and fault monitoring among various modules to support the stability and real-time requirements of the autonomous driving system;

[0097] Data storage unit: used for storing raw data, intermediate processing results, system operation logs, etc., for recording, model training and testing, system iterative optimization, troubleshooting, etc.

[0098] The perception hardware module synchronizes multi-frame point clouds with image data through a time synchronization mechanism and sends them to the computing module through a communication interface; the computing module processes the data according to the algorithm step flow and ultimately outputs results such as semantic labels; the system control module uniformly schedules raw data acquisition, feature processing and inference prediction, data recording, system health monitoring, etc. to ensure stable and efficient operation of the system.

[0099] The 3D point cloud semantic segmentation system based on adaptive spatiotemporal aggregation enhancement provided by the present invention is suitable for various 3D environment perception and understanding scenarios such as autonomous driving, intelligent robots, and automated safety assurance of engineering equipment, and has high-precision and high-robustness semantic segmentation capabilities.

[0100] See also Figure 3As shown, an embodiment of the present invention further provides an electronic device 600, which includes a processor 601, a memory 602, and a program or instruction stored in the memory 602 and executable on the processor 601. When the program or instruction is executed by the processor 601, each process of the above-mentioned embodiment of the 3D point cloud semantic segmentation method based on adaptive spatiotemporal aggregation enhancement is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0101] It should be noted that the electronic devices in the embodiments of the present invention include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0102] Figure 4 The present invention is a hardware structure diagram of an electronic device.

[0103] The electronic device 700 includes but is not limited to components such as a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710.

[0104] Those skilled in the art will understand that the electronic device 700 may also include a power supply (such as a battery) to power each component. The power supply may be logically connected to the processor 710 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 4 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0105] It should be understood that in this embodiment of the present invention, the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042. The GPU 7041 processes image data of still images or videos captured by an image capture system (e.g., a camera) in video capture mode or image capture mode. The display unit 706 may include a display panel 7061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. The user input unit 707 includes a touch panel 7071 and other input devices 7072. The touch panel 7071, also known as a touch screen, may include a touch detection system and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons and power buttons), a trackball, a mouse, and a joystick, which are not described in detail here. The memory 709 may be used to store software programs and various data, including, but not limited to, applications and an operating system. The processor 710 may integrate an application processor and a modem processor. The application processor primarily processes the operating system, user interface, and applications, while the modem processor primarily handles wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 710.

[0106] An embodiment of the present invention also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned embodiment of the 3D point cloud semantic segmentation method based on adaptive spatiotemporal aggregation enhancement are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0107] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0108] An embodiment of the present invention further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned embodiment of the 3D point cloud semantic segmentation method based on adaptive spatiotemporal aggregation enhancement, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0109] It should be understood that the chip mentioned in the embodiment of the present invention can also be called a system-on-chip, a system-on-chip, a chip system, or a system-on-chip chip, etc.

[0110] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0111] Furthermore, it should be noted that the scope of the methods and systems of the present invention is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in reverse order, depending on the functions involved. For example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Furthermore, features described with reference to certain examples may be combined in other examples.

[0112] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A 3D point cloud semantic segmentation method based on adaptive spatiotemporal aggregation enhancement, characterized in that: The steps include: Step S1, obtaining the current frame lidar point cloud and the historical frame point cloud sequence to construct a multi-frame point cloud dataset; Step S2, performing spatiotemporal adaptive enhancement on the multi-frame point cloud dataset, specifically comprising: Step S21, dividing each frame of point cloud according to the laser radar beam number or tilt angle, and spatially mixing the odd-numbered beams and even-numbered beams of the previous and next frame point clouds; Step S22: setting an adaptive threshold based on the confidence of the category segmentation, allocating more historical frames to the low-confidence category for temporal enhancement, and generating an enhanced point cloud; Step S23: Statistically enhance the local voxel density of the point cloud and select fine-grained distillation or global distillation strategy to optimize feature expression based on the density threshold. The fine-grained distillation extracts local structural features through the teacher network and aligns them with the student network features; the global distillation optimizes the overall features of low-density areas through semantic compensation. Step S3, synchronously acquiring multiple frames of image data, extracting image semantic features and projecting them into 3D space to generate pseudo voxel features; Step S4, spatially aligning and fusing the enhanced point cloud features with the pseudo voxel image features, and outputting the fused features; In step S5, the semantic label of each point is output through the segmentation network based on the fused features to complete the 3D semantic segmentation task.

2. The method according to claim 1, characterized in that In step S21, the specific method of the spatial mixing is: the odd-numbered line bundles and the even-numbered line bundles of the previous and next frame point clouds are alternately combined by region, so that the point cloud data of any adjacent region come from different frames.

3. The method according to claim 1, characterized in that In step S22, the category segmentation confidence is calculated by intersection over union (IoU), where low confidence categories include pedestrians or cone-shaped obstacles, and high confidence categories include cars or trucks.

4. The method according to claim 1, wherein In step S3, the specific steps of generating pseudo voxel features include: Extract semantic feature maps of multiple frames through the image backbone network; The image feature map is projected into the point cloud coordinate system using the calibrated extrinsic parameters to generate 3D pseudo voxel features; The pseudo voxel features are temporally fused to form a multi-frame enhanced pseudo voxel feature network.

5. The method according to claim 1, wherein In step S4, feature fusion uses trilinear interpolation to align point cloud and pseudo-voxel features, and realizes multimodal fusion through attention mechanism or feature splicing.

6. The method according to claim 1, characterized in that In step S5, the loss function of the segmentation network is a weighted sum of the cross entropy loss and the knowledge distillation loss, which is used for the joint training model.

7. A 3D point cloud semantic segmentation system based on adaptive spatiotemporal aggregation enhancement for implementing the method according to any one of claims 1 to 6, characterized in that: include: Perception hardware module, used to synchronously collect multi-frame point cloud and image data; The computing module is used to complete data preprocessing, feature extraction, data enhancement, feature fusion and predictive reasoning; System control module, used to coordinate data synchronization, task scheduling and exception monitoring.

8. The system according to claim 7, characterized in that The perception hardware module includes: LiDAR unit, used to collect multi-frame 3D point cloud data in real time; The camera unit synchronously acquires multiple frames of image data.

9. The system according to claim 7, wherein: The computational module includes a multi-frame point cloud enhancement submodule, a density-aware distillation submodule, an image pseudo-voxel feature generation submodule, a multimodal feature fusion submodule, and a semantic segmentation reasoning submodule, among which: The multi-frame point cloud enhancement submodule acquires continuous frame point clouds based on the time sequence, combines the beams in an orderly manner according to the physical structure of the laser beam, performs category-adaptive frame fusion according to the difficulty of category recognition, improves the feature expression ability of difficult-to-classify targets, and uses the point cloud backbone network to extract the enhanced point cloud features; The density-aware distillation submodule analyzes the local density of the point cloud, uses fine-grained feature distillation for high-density areas, and uses global feature distillation for low-density areas; The image pseudo voxel feature generation submodule uses the image backbone network to extract image features, generates pseudo voxel features based on multiple frames of images, and applies a spatiotemporal enhancement strategy to the pseudo features; The multimodal feature fusion submodule establishes spatial correspondence based on the calibration relationship between the point cloud and the image, and aligns and fuses the point cloud features with the image pseudo-voxel features; The semantic segmentation reasoning prediction submodule receives the fused features and outputs point semantic labels through the segmentation head to complete the 3D semantic segmentation task.

Images