A harness guided sample point cloud segmentation time sequence fusion method and device

Abstract

The application discloses a point cloud segmentation time sequence fusion method and device for wire harness guiding sampling, and relates to the technical field of intelligent driving and environment perception. The method comprises the following steps: inputting laser radar point cloud data; performing adaptive point cloud sampling based on a laser radar wire harness structure to obtain adaptive sampling point cloud; performing initial segmentation on the adaptive sampling point cloud to generate an initial segmentation result; performing time sequence perception fusion based on an inertial measurement unit pose driving on the initial segmentation result to obtain a time sequence optimization segmentation result; and outputting the time sequence optimization segmentation result as a final segmentation result. The application realizes structure lossless compression and dynamic enhancement of point cloud through adaptive sampling, and avoids time sequence network overhead through pose driving fusion, so that the bandwidth / storage constraint, geometric structure maintenance, sparse scene adaptation and real-time requirement can be met at the same time without modifying an existing perception network.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving and environmental perception technology, and in particular to a method and apparatus for point cloud segmentation and temporal fusion using wire harness-guided sampling. Background Technology

[0002] With the large-scale application of mechanical rotating LiDAR (such as Velodyne HDL-64E and Hesai Pandar series) in autonomous driving, mobile robotics, and high-precision map construction, the amount of point cloud data per frame is growing exponentially (approximately 50,000–80,000 points per frame for 32-line LiDAR and over 200,000 points for 128-line LiDAR). Under engineering constraints such as limited vehicle communication bandwidth (e.g., typical 5G-V2X bandwidth <100Mbps), limited computing power of edge computing devices, and the requirement for real-time perception systems to meet <100ms latency, point cloud sampling has become an indispensable preprocessing step in the LiDAR data processing pipeline.

[0003] Current mainstream sampling methods in the industry mainly include: Voxel Downsampling: dividing the space into a regular 3D grid, retaining the center point or random points in each cell (standard implementation in the Open3D library); Farthest Point Sampling (FPS): iteratively selecting the point farthest from the selected point set to ensure uniform spatial distribution (core preprocessing step in PointNet / PointNet++); and random downsampling and geometric feature sampling: weighted sampling based on local geometric properties such as curvature and normal vectors (e.g., UniformSampling and NormalSpaceSampling in the PCL library). These methods are highly effective in reducing data volume and have been widely integrated into the preprocessing modules of mainstream 3D perception frameworks (such as OpenPCDet and MMDetection3D).

[0004] Existing sampling techniques suffer from the following key defects: 1) Ignoring the physical beamline structure of lidar: The scanning of mechanical lidar is essentially a 360° rotational scan of discrete beamlines. Taking a 32-line Velodyne lidar as an example, its 32 lasers are arranged at fixed vertical angles (e.g., -30.67° to +10.67°) to form 32 physical scan lines. Voxel downsampling or FPS (farthest point sampling) methods treat the point cloud as an unordered set, resulting in non-uniform removal of points on the same scanline, severely disrupting the continuity of the point cloud along the scanning direction—especially at object edges and in slender structures (e.g., utility poles, guardrails), producing "jagged breaks." The continuity of the point cloud is disrupted, significantly reducing the ability of subsequent detection / segmentation modules to recognize boundary features; 2) Poor compatibility with data without ring fields: To save storage space, mainstream public datasets typically only retain the (x,y,z,i) fields in the original point cloud, omitting the beamline number (ring). Therefore, existing methods struggle to balance “reducing data volume” and “preserving geometric structure”, failing to achieve the dual goals of precise on-demand control points and lossless structural compression.

[0005] At the temporal perception level, multi-frame point cloud fusion is a key path to improve perception robustness. However, existing methods generally rely on complex temporal modeling networks (such as LSTM and Transformer), which have high computational overhead and poor real-time performance. For example, industrial-grade autonomous driving systems need to complete single-frame perception within 100ms, but the additional computation of temporal networks (usually increasing inference latency by 30% to 50%) makes it difficult for the system to meet real-time requirements. At the same time, errors in temporal estimation (such as IMU pose drift) are easily amplified, which reduces perception stability and increases the false detection rate (measured data: the false detection rate of traditional temporal methods increases by 15% to 20%).

[0006] Current mainstream time-aware solutions (such as TTNet and T-PointPillars) integrate multi-frame information by designing dedicated time-series networks, but they have the following drawbacks: 1) High computational overhead: The time-series network introduces an additional computational layer (such as LSTM), which increases inference latency by 40-60% (measured from 20ms to 35ms), failing to meet the real-time requirement of <30ms for in-vehicle systems; 2) Insufficient robustness of time-series estimation: The time-series features predicted by the network (such as target motion trajectories) are easily affected by noise, leading to "misfusion" (such as misclassifying background points as target instances); 3) Accumulated computational load: Without a window mechanism, the accumulation of historical frames causes the computational load to increase linearly over time (after 30 frames, the computational load reaches 30 times that of a single frame), making it impossible for the system to operate stably for a long time.

[0007] Therefore, existing technologies mainly rely on complex networks, which makes it impossible to achieve both timing accuracy (depending on the network) and real-time performance (computational overhead), thus failing to meet the real-time requirements of edge devices. Summary of the Invention

[0008] To address the aforementioned issues, the present invention aims to provide a point cloud segmentation and temporal fusion method and apparatus based on wire harness-guided sampling. Using LiDAR and a high-precision inertial measurement unit (IMU) as hardware foundations, it offers a lightweight solution that integrates wire harness structure perception with IMU pose-driven fusion. Through adaptive sampling, it achieves lossless compression and dynamic enhancement of the point cloud structure, and through pose-driven fusion, it avoids temporal network overhead. Ultimately, without modifying the existing perception network, it simultaneously meets four major engineering objectives: bandwidth / storage constraints, geometric structure preservation, sparse scene adaptation, and real-time performance requirements.

[0009] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On the one hand, a point cloud segmentation temporal fusion method guided by wire harness sampling is provided, the method comprising the following steps: S1. Input LiDAR point cloud data; S2. Adaptive point cloud sampling is performed based on the lidar beam structure to obtain the adaptive sampled point cloud; S3. Perform initial segmentation on the adaptive sampling point cloud and generate initial segmentation results; S4. Perform time-aware fusion based on inertial measurement unit pose-driven process on the initial segmentation results to obtain time-optimized segmentation results; S5. Output the time-optimized segmentation result as the final segmentation result.

[0010] Optionally, the adaptive point cloud sampling based on the lidar beam structure specifically includes: Determine whether the input point cloud contains wire harness numbers; When the input point cloud contains a beam number, it is directly grouped by beam number; for the input point cloud without beam number, the beam grouping of each point is estimated in reverse by querying the preset vertical elevation angle of the lidar. After obtaining the bundle grouping information of the point cloud, density detection is performed on each frame of the point cloud. Based on the density detection results, when the point cloud density is greater than or equal to the threshold, structured adaptive downsampling is performed on this frame of point cloud; when the point cloud density is less than the threshold, wire bundle interpolation upsampling is performed on this frame of point cloud. Output adaptive sampling point cloud.

[0011] Optionally, the step of estimating the beam grouping of each point by querying the preset vertical elevation angle of the lidar specifically includes: For any point P=(x,y,z), the formula for calculating its vertical elevation angle is:

[0012] Preset vertical elevation angle array for lidar beams A=[a0,a1,...,a N-1], N is the total number of wire harnesses, calculate the boundary angle array B=[b0,b1,...,b N ]:

[0013] Harness ownership determination:

[0014] Point P is then assigned to the kth harness group.

[0015] Optionally, the density detection for each frame of point cloud specifically includes: Real-time detection of point cloud density Defined as the number of points per unit harness, the calculation formula is:

[0016] in This represents the total number of point clouds in a single frame. This represents the total number of wire harnesses. Set a threshold for judging the sparsity of point clouds. ; when At that time, structured adaptive downsampling processing is performed on this frame of point cloud: If the algorithm input has requirements for the line bundles of the LiDAR point cloud data, the number of points can be precisely controlled by truncating uniform line bundles; if the algorithm input needs to compress the amount of data while preserving structural information, then each line bundle is sampled at intervals to ensure that the point cloud can still completely represent the object boundary and spatial continuity after sampling. when At that time, perform line bundle interpolation upsampling processing on this frame of point cloud: Spatial interpolation is performed on the point clouds of adjacent line bundles k and k+1 to generate a new point cloud line bundle k. mid The point cloud coordinates of the new harness are calculated using a linear interpolation formula:

[0017] in and These are the coordinates of points in adjacent line bundles; , , This represents the elevation angle of the current point and the elevation angle of the adjacent line bundle.

[0018] Optionally, the 2DPASS segmentation algorithm is used to perform initial segmentation on the adaptively sampled point cloud to generate initial segmentation results. It includes point coordinates P, label category c, and confidence level. ;in The confidence score of point P, as output by the segmentation algorithm, belonging to category c; The step of performing time-aware fusion of the initial segmentation results based on inertial measurement unit pose-driven method specifically includes: Global coordinate projection: utilizing the pose transformation matrix provided by the inertial measurement unit (IMU). T imu Project the initial segmentation result of the current frame onto the global coordinate system. ;

[0019] Sliding window management: Maintain historical frame data in a fixed window. When a new frame arrives, if the window is not full, it is added directly to the window; if the window is full, the current window result is frozen into a new set of single-frame fused data, and the new frame is included in the new window. Cross-frame clustering: In the global coordinate system, the DBSCAN clustering algorithm is used to merge the segmentation results of historical frames and the current frame; Perform category-weighted voting and position-average optimization for each cluster C: Category-weighted voting: Calculate the category distribution of points within a cluster, and select the category with the highest number of votes as the final category. ;

[0020] Location averaging optimization: Calculates the optimal location of points within a cluster. ;

[0021] P∈C means traversing every point P in cluster C, where |C| represents the number of points in cluster C; The final segmentation result in the global coordinate system is as follows ; Single-frame result mapping: mapping the segmentation results By projecting back into the current frame coordinate system, the temporally optimized single-frame segmentation result is obtained. : .

[0022] Optionally, the fixed window W is set to be no less than 30 frames; When a new frame arrives, if the number of frames is less than W, it is added directly to the window; if the number of frames is equal to W, the current window result is frozen into a new set of single-frame fused data, and the new frame is included in the new window.

[0023] Optionally, the DBSCAN clustering algorithm is used to merge the segmentation results of historical frames and the current frame, and the clustering is based on 3D Euclidean distance. :

[0024] Where (x1, y1, z1) and (x2, y2, z2) are the coordinates of the points participating in the clustering, and the distance threshold is set to... Meters, minimum number of points set to .

[0025] On the other hand, a point cloud segmentation and temporal fusion apparatus for wire harness-guided sampling is provided for implementing the method described in any of the above embodiments, the apparatus comprising: The input module is used to input LiDAR point cloud data; The point cloud sampling module is used to perform adaptive point cloud sampling based on the lidar beam structure to obtain an adaptive sampled point cloud. The initial segmentation module is used to perform initial segmentation on the adaptive sampled point cloud and generate initial segmentation results; The temporal-aware fusion module is used to perform temporal-aware fusion based on the pose of the inertial measurement unit on the initial segmentation results to obtain temporally optimized segmentation results; The output module is used to output the time-optimized segmentation result as the final segmentation result.

[0026] On the other hand, an electronic device is provided, the electronic device comprising: processor; The memory stores computer-readable instructions, which, when loaded and executed by the processor, implement the steps of the point cloud segmentation temporal fusion method described above for wire harness-guided sampling.

[0027] On the other hand, a computer-readable storage medium is provided, wherein program code is stored in the computer-readable storage medium, and the program code can be called by a processor to execute the steps of the point cloud segmentation temporal fusion method for wire harness guided sampling as described above.

[0028] The beneficial effects of the technical solution provided by this invention include at least the following: This invention achieves a systematic breakthrough in the field of lidar point cloud processing through deep collaboration between the point cloud sampling module and the temporal perception fusion module. The specific technical effects can be quantitatively described from five dimensions: geometric structure preservation, data compatibility, sparse scene adaptation, real-time performance assurance, and improved perception accuracy. All effects are verified by actual testing on the hardware platform of Hesai XT32 lidar (32-line mechanical lidar) and Supercore Electronics CH110 IMU (high-precision inertial measurement unit).

[0029] (1) Geometric structure preservation: boundary continuity achieves zero damage; Traditional point cloud sampling methods (such as voxel downsampling) treat point clouds as an unordered set, resulting in the random removal of point clouds along the laser scanning line, leading to jagged edges on objects (such as utility poles and guardrails). This invention, by using bundle grouping to achieve structured adaptive sampling, can more completely preserve the structured information of the original point cloud, ensuring boundary continuity.

[0030] (2) Data compatibility: 100% compatible with public datasets and industrial scenarios; Existing technologies require additional preprocessing for data without ring fields (such as KITTI and nuScenes), resulting in high engineering costs. This invention achieves seamless compatibility through an elevation angle reverse calculation algorithm.

[0031] (3) Sparse scene adaptive enhancement: dynamically increase point cloud density; In scenarios with long distances, the sparse point cloud density at distant locations leads to a decrease in perception capability. This invention innovatively introduces a wire harness interpolation upsampling strategy. In the sparse test scenario of XT32, the mIOU of the 2DPASS segmentation algorithm after upsampling is improved from 58.7% to 76.5%, effectively solving the perception failure problem in sparse scenarios.

[0032] (4) Real-time guarantee: Inference latency is controlled within 21ms; Traditional time-series fusion solutions (such as TTNet) rely on LSTM / Transformer networks, resulting in inference latency of up to 35ms, exceeding the <30ms requirement of automotive systems. This invention completely eliminates time-series network overhead through IMU pose-driven fusion. On the XT32+CH110 hardware platform, the inference latency is controlled within 21ms, far below the 30ms automotive real-time threshold, meeting the deployment requirements of edge devices.

[0033] (5) Perception accuracy and robustness: reduce false negative rate and improve recall rate; The temporal perception fusion module eliminates temporal network noise through clustering-voting optimization, significantly improving perception stability. In complex urban scenarios (including dense vehicles and pedestrians), the false negative rate is reduced from 18.2% to 0.2%, the recall rate is increased from 72.1% to 87.1%, and the perception reliability is improved by 35%. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1This is a flowchart of the point cloud segmentation and temporal fusion method for wire harness-guided sampling provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the adaptive sampling process provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the time-aware fusion process provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the point cloud segmentation and temporal fusion device for wire harness-guided sampling provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0037] This invention provides a point cloud segmentation and temporal fusion method using wire harness-guided sampling. This method can be implemented by an electronic device, which can be a terminal or a server. Figure 1 As shown, the processing flow of this method may include the following steps: S1. Input LiDAR point cloud data.

[0038] In this embodiment of the invention, the Hesai XT32 lidar (a 32-line mechanical lidar that supports the ring field) is used as an example to acquire three-dimensional point cloud data.

[0039] S2. Adaptive point cloud sampling is performed based on the lidar beam structure to obtain the adaptive sampled point cloud.

[0040] This invention is based on the physical characteristics of the Hesai XT32 LiDAR. By deeply exploring the geometric rules of its beam structure, it achieves structured compression and dynamic enhancement of point clouds. The adaptive sampling strategy process is as follows: Figure 2 As shown, it specifically includes: Determine whether the input point cloud contains a harness number (such as the ring field of the XT32 raw data); When the input point cloud contains wire bundle numbers, it is directly grouped according to the wire bundle numbers; for input point clouds without wire bundle numbers (such as public datasets like KITTI), the wire bundle grouping of each point is estimated in reverse by querying the preset vertical elevation angle of the lidar, achieving 100% compatible wire bundle grouping, thus seamlessly adapting to various point cloud data sources, while ensuring the structural consistency and computational efficiency of the sampling process.

[0041] The specific process of back-estimating the bundle grouping for each point is as follows: For any point P=(x,y,z), the formula for calculating its vertical elevation angle is: (Unit: degree) Preset vertical elevation angle array for lidar beams A=[a0,a1,...,a N-1 ], N is the total number of wire harnesses, calculate the boundary angle array B=[b0,b1,...,b N ]:

[0042] Harness ownership determination:

[0043] Point P is then assigned to the kth harness group.

[0044] After obtaining the point cloud's bundle grouping information, density detection needs to be performed on each frame of the point cloud to determine whether structured downsampling or upsampling should be performed. Based on the density detection results, when the point cloud density is ≥ the threshold, structured adaptive downsampling is performed on this frame of the point cloud; when the point cloud density is < the threshold, bundle interpolation upsampling is performed on this frame of the point cloud.

[0045] Specifically, real-time detection of point cloud density Defined as the number of points per unit harness, the calculation formula is:

[0046] in This represents the total number of point clouds in a single frame. This represents the total number of wire harnesses.

[0047] At the same time, a threshold for judging the sparsity of point clouds is set according to the actual needs of the algorithm. .

[0048] when At that time, structured adaptive downsampling processing is performed on this frame of point cloud: If the algorithm input has requirements for the line bundle of the LiDAR point cloud data (e.g., 16 lines are required but the LiDAR used is 32 lines), then the point count can be precisely controlled by truncating a uniform line bundle (e.g., 32 lines → taking 16 lines at intervals). This solution reduces the number of points from 60,000 points per frame of a 32-line LiDAR to 30,000 points. This solution is more compatible with the algorithm. If the algorithm input needs to compress the data volume while preserving structural information, then interval sampling (e.g., taking points at intervals between each line bundle) is performed to ensure that the point cloud after sampling can still completely represent the object boundary and spatial continuity. This approach can preserve the original data structure information to the greatest extent, reduce information loss, and thus improve the algorithm's perception accuracy.

[0049] when If the point cloud data in this frame is considered sparse, it may affect the accuracy of the algorithm's perception. Therefore, it is necessary to perform line bundle interpolation upsampling processing on this frame of point cloud. Spatial interpolation is performed on the point clouds of adjacent line bundles k and k+1 to generate a new point cloud line bundle k. mid The point cloud coordinates of the new harness are calculated using a linear interpolation formula:

[0050] in and These are the coordinates of points in adjacent line bundles; , , This represents the elevation angle of the current point and the elevation angle of the adjacent line bundle.

[0051] The upsampling strategy enhances the point cloud density in sparse regions while maintaining the continuity of scan lines. In actual tests on sparse scenes acquired by the XT32, upsampling improved the continuity of object edges by 32% and the mIOU of the 2DPASS segmentation algorithm by 18%.

[0052] S3. Perform initial segmentation on the adaptive sampling point cloud to generate initial segmentation results.

[0053] The 2DPASS segmentation algorithm (an efficient segmentation model based on 2D-3D feature fusion) is used to perform initial segmentation on the adaptively sampled point cloud, generating initial segmentation results. It includes point coordinates P, label category c, and confidence level. ;in This represents the confidence score of point P, output by the segmentation algorithm, belonging to category c. The segmentation result here only retains the target label points (such as vehicles or people) that need to be segmented, not all points, in order to reduce computational load.

[0054] S4. Perform time-aware fusion based on inertial measurement unit pose-driven process on the initial segmentation results to obtain time-optimized segmentation results.

[0055] In this embodiment of the invention, high-precision pose data from the CH110 IMU (Supercore Electronics) is used as the core, and lightweight multi-frame fusion is achieved through spatiotemporal registration, completely avoiding the computational overhead of temporal networks. The workflow is as follows: Figure 3 As shown ( Figure 3 The "segmentation / detection" approach applies to both segmentation and detection (the following explanation uses segmentation as an example), specifically including: Global coordinate projection: utilizing the pose transformation matrix provided by the inertial measurement unit (IMU). T imu (Obtained through LIO-SAM optimization of the SLAM algorithm), the initial segmentation result of the current frame is projected onto the global coordinate system. ; .

[0056] Sliding window management: Maintain historical frame data in a fixed window. When a new frame arrives, if the window is not full, it is added directly to the window; if the window is full, the current window result is frozen into a new set of single-frame fused data, and the new frame is included in the new window.

[0057] The fixed window W is set according to the device's real-time computing capability, and is usually set to no less than 30 frames. When a new frame arrives, if the number of frames is less than W, it is directly added to the window; if the number of frames is equal to W, the current window result is frozen into a new set of single-frame fused data, and the new frame is included in the new window.

[0058] For example, W can be set to 30 frames. When a new frame arrives, if the window is not full (frame count < 30), it is added directly to the window; if the window is full (frame count = 30), the current window result is frozen into a new set of single-frame fused data (only one subsequent clustering and dual optimization are performed), and the new frame is included in the new window. This mechanism ensures a constant computational load (always processing 30 frames) and avoids the linear increase in computational load caused by the accumulation of historical frames.

[0059] The 30 frames per second is not set arbitrarily, but is determined based on the computing power of the hardware device. 30 frames refers to the minimum number of frames required for the device to perform real-time calculations. If the device can process 30 frames of segmentation results per second, the fixed window can be set to 30. If the device can process 60 frames of segmentation results per second, the fixed window can be set to 60.

[0060] Cross-frame clustering: In the global coordinate system, the DBSCAN clustering algorithm is used to merge the segmentation results of historical frames and the current frame. Clustering is based on 3D Euclidean distance. :

[0061] Where (x1, y1, z1) and (x2, y2, z2) are the coordinates of the points participating in the clustering, and the distance threshold is set to... Meters (covering typical target size), minimum number of points set to .

[0062] Perform category-weighted voting and position-average optimization for each cluster C: Category-weighted voting: Calculate the category distribution of points within a cluster, and select the category with the highest number of votes as the final category. ;

[0063] Location averaging optimization: Calculates the optimal location of points within a cluster. ;

[0064] P∈C means traversing every point P in cluster C, where |C| represents the number of points in cluster C; The final segmentation result in the global coordinate system is as follows .

[0065] Single-frame result mapping: mapping the segmentation results By projecting back into the current frame coordinate system, the temporally optimized single-frame segmentation result is obtained. : .

[0066] In the aforementioned temporal perception fusion scheme, pose-driven fusion replaces the temporal network, significantly reducing inference latency; the sliding window mechanism ensures constant computational load (30-frame window); and the clustering-voting strategy eliminates temporal network noise, reducing the false negative rate by 18% and improving the recall rate by 15%. On the hardware platforms of Hesai XT32 and Supercore Electronics CH110, the system can achieve structured compression and temporal robust optimization of point cloud data without modifying the existing perception network architecture (such as 2DPASS), meeting the real-time requirement of <30ms for automotive systems.

[0067] S5. Output the time-optimized segmentation result as the final segmentation result.

[0068] Compared with the prior art, the advantages of the present invention are: In-depth utilization of the physical beamline structure of lidar: This invention uses the physical scanning beamline (ring) of a mechanical lidar (such as the Hesai XT32) as the basic structural unit for point cloud sampling, rather than treating it as a disordered set of points. By strictly constructing a beamline boundary partitioning model based on the laser's preset vertical elevation angle array, the point cloud is structurally grouped according to the scanning beamline. This feature is fundamental to solving the problems of point cloud boundary breakage and geometric structure destruction. Without this feature, the sampling process would inevitably disrupt the continuity of the point cloud along the scanning direction, resulting in jagged object edges and making lossless structural compression impossible.

[0069] Elevation Angle Reverse Calculation Algorithm and Harness Interpolation Upsampling Strategy: Addressing the industry pain point of missing ring fields in public datasets (such as KITTI), this invention proposes an elevation angle reverse calculation algorithm to dynamically reconstruct harness numbers; simultaneously, it innovatively introduces a harness interpolation upsampling strategy to achieve dynamic point cloud enhancement in sparse scenes. This feature is crucial for solving the problems of insufficient data compatibility and sparse scene perception capabilities. Without this feature, the system cannot adapt to datasets without rings, and its perception robustness cannot be improved in low-light / long-distance scenes.

[0070] A lightweight temporal-aware fusion mechanism driven by IMU pose: This invention abandons traditional temporal networks (such as LSTM) and instead utilizes the high-precision pose information of the IMU (obtained through SLAM optimization) to construct a pose-driven spatiotemporal registration process. This feature is the core to achieving low latency and high robustness. Without it, temporal fusion would inevitably rely on computationally intensive networks, leading to excessive latency (>35ms) and amplified temporal noise, failing to meet the real-time requirements of automotive applications.

[0071] Sliding Window and Clustering-Voting Optimization Framework: This invention designs a sliding window mechanism with a fixed window width, combining DBSCAN clustering and weighted voting strategies to achieve constant computational load and reduced false negative rate. This feature is key to solving the problems of linear growth in computational load and perceptual stability caused by the accumulation of historical frames. Without this feature, the system will fail due to the exponential increase in computational load over a long period of time, and temporal network noise interference cannot be eliminated.

[0072] Therefore, this invention satisfies four major engineering objectives—bandwidth / storage constraints, geometric structure preservation, sparse scene adaptation, and real-time requirements—without modifying existing sensing networks.

[0073] Accordingly, embodiments of the present invention also provide a point cloud segmentation temporal fusion apparatus for wire harness-guided sampling, such as... Figure 4 As shown, the device includes: The input module is used to input LiDAR point cloud data; The point cloud sampling module is used to perform adaptive point cloud sampling based on the lidar beam structure to obtain an adaptive sampled point cloud. The initial segmentation module is used to perform initial segmentation on the adaptive sampled point cloud and generate initial segmentation results; The temporal-aware fusion module is used to perform temporal-aware fusion based on the pose of the inertial measurement unit on the initial segmentation results to obtain temporally optimized segmentation results; The output module is used to output the time-optimized segmentation result as the final segmentation result.

[0074] Among them, the point cloud sampling module and the time-series awareness fusion module are the core of the collaborative technical solution of this invention: Point cloud sampling module: Adaptive point cloud sampling based on lidar beam structure to solve the point cloud compression problem; Timing-aware fusion module: Lightweight timing awareness based on IMU pose-driven approach to solve timing fusion problem; The two modules are seamlessly integrated: the structured adaptive sampled point cloud output by the point cloud sampling module provides high-quality input to the temporal perception fusion module, and the temporal optimization results of the temporal perception fusion module in turn improve the perception reliability of the point cloud sampling module.

[0075] Specifically, the point cloud sampling module compresses or enhances the point cloud through beam grouping, preserving the continuity of object boundaries and improving the initial accuracy of 2DPASS segmentation by 25%. The temporal perception fusion module utilizes IMU pose-driven technology to fuse the current frame's point cloud segmentation results with historical frame information, further enhancing perception robustness. In complex scene tests with the XT32 radar, this collaborative scheme improved the mIOU of target detection by 22%, and the system can run stably on edge devices for over 100 hours without any computational accumulation issues.

[0076] For ease of explanation, Figure 4 Only the main components of the device are shown. The device of this embodiment can be used to perform... Figure 1 The technical solutions of the method embodiments shown are similar in principle and in effect, and will not be described again here.

[0077] In an exemplary embodiment, the present invention also provides an electronic device, the electronic device comprising: processor; The memory stores computer-readable instructions, which, when loaded and executed by the processor, implement the steps of the point cloud segmentation temporal fusion method described above for wire harness-guided sampling.

[0078] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 5As shown, the electronic device 300 may include a first processor 3001 and a memory 3002. Optionally, the electronic device 300 may also include a transceiver 3003. The first processor 3001, the memory 3002, and the transceiver 3003 may be connected via a communication bus. The memory 3002 stores computer-readable instructions, which, when executed by the first processor 3001, implement the steps of the point cloud segmentation temporal fusion method described above for wire harness-guided sampling.

[0079] In a specific implementation, as one example, the first processor 3001 may include one or more CPUs, for example... Figure 5 CPU0 and CPU1 are shown in the diagram.

[0080] In a specific implementation, as one example, the electronic device 300 may also include multiple processors, for example... Figure 5 The first processor 3001 and the second processor 3004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0081] The memory 3002 is used to store the software program that executes the present invention, and is controlled by the first processor 3001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0082] The transceiver 3003 is used to communicate with network devices or with terminal devices.

[0083] Optionally, the transceiver 3003 may include a receiver and a transmitter. The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0084] Optionally, the transceiver 3003 can be integrated with the first processor 3001 or exist independently and be coupled to the first processor 3001 through the interface circuit of the electronic device 300. This embodiment of the invention does not specifically limit this.

[0085] It should be noted that, Figure 5 The structure of the electronic device 300 shown is not intended to limit the electronic device. Actual electronic devices may include more or fewer components than shown, or combine certain components, or have different component arrangements. Furthermore, the technical effects of the electronic device 300 can be understood by referring to the technical effects of the above-described method embodiments, and will not be repeated here.

[0086] In an exemplary embodiment, the present invention also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the steps of the point cloud segmentation temporal fusion method for wire-guided sampling as described above. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0087] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0088] The use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0089] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0090] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0091] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0092] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0093] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A point cloud segmentation and temporal fusion method using wire-guided sampling, characterized in that, Includes the following steps: S1. Input LiDAR point cloud data; S2. Adaptive point cloud sampling is performed based on the lidar beam structure to obtain the adaptive sampled point cloud; S3. Perform initial segmentation on the adaptive sampling point cloud and generate initial segmentation results; S4. Perform time-aware fusion based on inertial measurement unit pose-driven process on the initial segmentation results to obtain time-optimized segmentation results; S5. Output the time-optimized segmentation result as the final segmentation result.

2. The point cloud segmentation and temporal fusion method for wire harness-guided sampling according to claim 1, characterized in that, The adaptive point cloud sampling based on the lidar beam structure specifically includes: Determine whether the input point cloud contains wire harness numbers; When the input point cloud contains a beam number, it is directly grouped by beam number; for the input point cloud without beam number, the beam grouping of each point is estimated in reverse by querying the preset vertical elevation angle of the lidar. After obtaining the bundle grouping information of the point cloud, density detection is performed on each frame of the point cloud. Based on the density detection results, when the point cloud density is greater than or equal to the threshold, structured adaptive downsampling is performed on this frame of point cloud; when the point cloud density is less than the threshold, wire bundle interpolation upsampling is performed on this frame of point cloud. Output adaptive sampling point cloud.

3. The point cloud segmentation and temporal fusion method for wire harness-guided sampling according to claim 2, characterized in that, The step of estimating the beam grouping of each point by querying the preset vertical elevation angle of the lidar specifically includes: For any point P=(x,y,z), the formula for calculating its vertical elevation angle is: Preset vertical elevation angle array for lidar beams A=[a0,a1,...,a N-1 ], N is the total number of wire harnesses, calculate the boundary angle array B=[b0,b1,...,b N ]: Harness ownership determination: Point P is then assigned to the kth harness group.

4. The point cloud segmentation and temporal fusion method for wire harness-guided sampling according to claim 2, characterized in that, The density detection of each frame of point cloud specifically includes: Real-time detection of point cloud density Defined as the number of points per unit harness, the calculation formula is: in This represents the total number of point clouds in a single frame. This represents the total number of wire harnesses. Set a threshold for judging the sparsity of point clouds. ; when At that time, structured adaptive downsampling processing is performed on this frame of point cloud: If the algorithm input has requirements for the line bundles of the LiDAR point cloud data, the number of points can be precisely controlled by truncating uniform line bundles; if the algorithm input needs to compress the amount of data while preserving structural information, then each line bundle is sampled at intervals to ensure that the point cloud can still completely represent the object boundary and spatial continuity after sampling. when At that time, perform line bundle interpolation upsampling processing on this frame of point cloud: Spatial interpolation is performed on the point clouds of adjacent line bundles k and k+1 to generate a new point cloud line bundle k. mid The point cloud coordinates of the new harness are calculated using a linear interpolation formula: in and These are the coordinates of points in adjacent line bundles; , , This represents the elevation angle of the current point and the elevation angle of the adjacent line bundle.

5. The point cloud segmentation and temporal fusion method for wire harness-guided sampling according to claim 1, characterized in that, The 2DPASS segmentation algorithm is used to perform initial segmentation on the adaptively sampled point cloud, generating initial segmentation results. It includes point coordinates P, label category c, and confidence level. ;in The confidence score of point P, as output by the segmentation algorithm, belonging to category c; The step of performing time-aware fusion of the initial segmentation results based on inertial measurement unit pose-driven method specifically includes: Global coordinate projection: utilizing the pose transformation matrix provided by the inertial measurement unit (IMU). T imu Project the initial segmentation result of the current frame onto the global coordinate system. ; Sliding window management: Maintain historical frame data in a fixed window. When a new frame arrives, if the window is not full, it is added directly to the window; if the window is full, the current window result is frozen into a new set of single-frame fused data, and the new frame is included in the new window. Cross-frame clustering: In the global coordinate system, the DBSCAN clustering algorithm is used to merge the segmentation results of historical frames and the current frame; Perform category-weighted voting and position-average optimization for each cluster C: Category-weighted voting: Calculate the category distribution of points within a cluster, and select the category with the highest number of votes as the final category. ; Location averaging optimization: Calculates the optimal location of points within a cluster. ; P∈C means traversing every point P in cluster C, where |C| represents the number of points in cluster C; The final segmentation result in the global coordinate system is as follows ; Single-frame result mapping: mapping the segmentation results By projecting back into the current frame coordinate system, the temporally optimized single-frame segmentation result is obtained. : 。 6. The point cloud segmentation and temporal fusion method for wire harness-guided sampling according to claim 5, characterized in that, The fixed window W is set to be no less than 30 frames; When a new frame arrives, if the number of frames is less than W, it is added directly to the window; if the number of frames is equal to W, the current window result is frozen into a new set of single-frame fused data, and the new frame is included in the new window.

7. The point cloud segmentation and temporal fusion method for wire harness-guided sampling according to claim 5, characterized in that, The DBSCAN clustering algorithm is used to merge the segmentation results of historical frames and the current frame. The clustering is based on 3D Euclidean distance. : Where (x1, y1, z1) and (x2, y2, z2) are the coordinates of the points participating in the clustering, and the distance threshold is set to... Meters, minimum number of points set to .

8. A point cloud segmentation temporal fusion apparatus for wire harness-guided sampling, the apparatus being used to implement the method as described in any one of claims 1 to 7, characterized in that, The device includes: The input module is used to input LiDAR point cloud data; The point cloud sampling module is used to perform adaptive point cloud sampling based on the lidar beam structure to obtain an adaptive sampled point cloud. The initial segmentation module is used to perform initial segmentation on the adaptive sampled point cloud and generate initial segmentation results; The temporal-aware fusion module is used to perform temporal-aware fusion based on the pose of the inertial measurement unit on the initial segmentation results to obtain temporally optimized segmentation results; The output module is used to output the time-optimized segmentation results as the final segmentation result.

9. An electronic device, characterized in that, The electronic device includes: processor; A memory storing computer-readable instructions that, when loaded and executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images