A 3D obstacle segmentation attribute pre-labeling method based on point cloud and image

Abstract

The present invention discloses a 3D obstacle segmentation attribute pre-labeling method based on point cloud and image, which relates to the attribute pre-labeling technical field, including S1, 3D multi-target tracking; S2, 3D tracking result projection to image; S3, ROI area extraction; S4, retrieval of segmentation prediction model according to large category; S5, ROI area splicing and feature extraction; S6, multi-frame result fusion and output of final result. The 3D obstacle segmentation attribute pre-labeling method based on point cloud and image realizes 3D obstacle segmentation attribute pre-labeling based on point cloud and image data. Since the method fully mines the semantic information of the image through the ROI area association method and the deep learning network, the obstacle segmentation attribute pre-labeling is realized, and the accuracy is also guaranteed, which reduces the workload of subsequent manual labeling.

Description

Technical Field

[0001] The present invention relates to the technical field of attribute pre-labeling, and in particular to a 3D obstacle subdivision attribute pre-labeling method based on point cloud and image. Background Art

[0002] The 3D obstacle pre-labeling algorithm technology is a key research direction in the field of computer vision and artificial intelligence, and is widely used in the fields of autonomous driving, robot navigation, etc. The purpose of this technology is to use a large model to automatically identify and locate obstacles in 3D space and predict their attribute information to reduce the workload of subsequent manual labeling, thereby accelerating the training and implementation of deep learning models.

[0003] Most current obstacle 3D pre-labeling algorithms use point cloud data to achieve the positioning and tracking of targets in 3D space. The rapid development of vehicle-side AI models has put forward higher requirements for the labeling of training data. Labeling not only needs to provide the spatial location information of obstacles, but also needs to provide segmented category attribute information. Traditional 3D pre-labeling methods using point cloud information can only achieve target positioning and tracking, predict rough categories, and lack the ability to predict segmented attributes, resulting in high costs and low efficiency for subsequent manual labeling.

[0004] Therefore, it is urgent to improve this shortcoming. The present invention studies and improves the existing technology and its shortcomings, and provides a 3D obstacle subdivision attribute pre-labeling method based on point cloud and image. Summary of the invention

[0005] The purpose of the present invention is to provide a 3D obstacle subdivision attribute pre-labeling method based on point cloud and image to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a 3D obstacle subdivision attribute pre-labeling method based on point cloud and image, comprising the following steps:

[0007] S1. 3D multi-target tracking:

[0008] Assume that a frame of data contains point cloud data provided by a lidar sensor and multi-view images provided by multiple cameras. For this frame of data, CenterPoint is first used as the point cloud 3D target detection algorithm to obtain the 3D detection frame of each obstacle, and the data association and tracking of the detected target are realized through the Hungarian matching algorithm and Kalman filter. Finally, the offset, scale, rotation angle, large category attributes, and unique identity of each obstacle in the XYZ axis direction are obtained. Each obstacle is represented by the following parameters:

[0009] (x,y,z,sizex,sizey,sizez.roll,pitch,yaw,class,trackid);

[0010] S2, 3D tracking results are projected onto the image:

[0011] Calculate 8 vertices through the 3D box of the obstacle, and use the internal and external parameters of the camera and lidar to project the 8 vertices of the 3D box onto the image of each perspective through formula 1;

[0012]

[0013] S3. ROI area extraction:

[0014] For the positions of the eight vertices of each obstacle on each image, find the upper, lower, left, and right boundaries of the 2D polygon X1min, X2max, Y1min, and Y2max, and intercept the rectangular area as the ROI (Region of Interest) of the obstacle on the image;

[0015] S4. Retrieve segmentation prediction model based on major categories:

[0016] According to the large category attributes (cars, pedestrians, and cyclists, which can be freely expanded) output by the 3D multi-target tracking algorithm in step S1, a segmentation prediction model is selected for subsequent feature extraction and prediction of segmentation categories;

[0017] S5. ROI area stitching and feature extraction:

[0018] The ROI area images extracted from multiple perspectives are preprocessed for input data. First, each ROI area is scaled and normalized to ensure that the data input size is HxW. Then, the multi-view image data is spliced ​​in the channel dimension. Taking two-view images as an example, the dimension of the input data is (BatchSize, 2x3, H, W). Then, it is passed into the ResNet backbone network for feature extraction. Finally, the target is classified to obtain the subdivision category result of a single frame.

[0019] S6. Fusion of multiple frames and output of final result:

[0020] All frames containing the target in the time series are retrieved through the unique identity of the obstacle in the data set, and steps S1 to S5 are repeated for each frame of data to obtain the subdivided category prediction result of each frame, and the category with the highest predicted category in the total number of frames is selected as the final predicted category of the obstacle.

[0021] Furthermore, in step S1, the process of obtaining the 3D detection frame of the obstacle is as follows:

[0022] Data preparation and preprocessing: Load point cloud data (obtain raw point cloud data from the lidar sensor, which usually contains the 3D coordinates of each point and other possible attributes), and filter the point cloud using voxel grid filtering, statistical filtering and other methods to remove noise and unnecessary points, and use random downsampling, uniform grid sampling and other methods to downsample to reduce the amount of calculation and remove noise, and then divide the processed point cloud data into a series of 3D voxels;

[0023] Feature extraction and encoding: Encode the features of the points in each voxel and extract the feature vector that can represent the voxel (these features include the number, density, average height, average intensity and other statistical information of the points in the voxel). Use a 3D feature encoder (such as VoxelNet or PointPillars) to convert the voxel features into a two-dimensional feature map (i.e., a bird's-eye view or BEV map). This step reduces the dimensionality of the three-dimensional point cloud data to a two-dimensional space, which facilitates subsequent convolution operations.

[0024] CenterPoint model reasoning: The generated feature map is used as the input of the CenterPoint model. The model performs forward propagation and generates a center point heat map (indicating the center point confidence of different categories of targets, and finding the peak point as the detected target center through non-maximum suppression (NMS)) and regression targets (including coordinate regression, size regression, height regression, rotation regression, etc., which are used to regress the complete 3D bounding box of the target from the center point feature) through a series of convolutional layers, batch normalization (BN) layers, and activation function (such as ReLU) layers.

[0025] Generate 3D detection box: At the center position of each detected target, extract the coordinates, size, rotation and other attributes from the regression head result to form a 3D detection box;

[0026] Multi-sensor fusion: The information contained in the multi-view images provided by the camera is fused with the point cloud detection results to further improve the accuracy and robustness of the detection.

[0027] Furthermore, in step S1, the Kalman filter prediction step: at each new time step (frame), the state of the obstacle is predicted using the Kalman filter, that is, based on the state estimation of the previous time step and the dynamic model of the system (such as uniform motion, uniformly accelerated motion, etc.), and the prediction result will give the estimated value of the obstacle state at the current time step and its uncertainty.

[0028] Furthermore, in step S1, the Hungarian matching algorithm association step:

[0029] Construct the cost matrix: Calculate the cost matrix between all detected obstacles at the current time step and the obstacles predicted by the Kalman filter at the previous time step (the elements of the cost matrix are based on some distance metric between the detection box and the prediction box, such as IOU, Mahalanobis distance, or appearance feature distance);

[0030] Hungarian matching: The Hungarian matching algorithm is used to optimally match the cost matrix to associate the detection box of the current time step with the prediction box of the previous time step. The matching result will give the corresponding relationship between the detection box and the obstacle, and allow the obstacle status to be updated.

[0031] Kalman filter update: Based on the results of the Hungarian matching algorithm, the detection box information of the current time step is used to update the state estimate of the Kalman filter, which specifically includes adjusting the value of the state vector and updating the covariance matrix to reflect the impact of the new observation data on the obstacle state.

[0032] Furthermore, in step S2, when performing projection at different viewing angles, if there is no target obstacle on the projected image at a certain viewing angle, the image at that viewing angle is skipped.

[0033] Furthermore, in step S2, the internal and external parameters of the camera and the laser radar are as follows:

[0034] Camera internal and external parameters: The camera's internal parameters include focal length, optical center and other internal parameters, which describe the geometric and optical characteristics of camera imaging. The external parameters describe the position and posture of the camera in the world coordinate system.

[0035] LiDAR internal and external parameters: The internal parameters of LiDAR include the emission angle and angle of the laser beam, which determine the accuracy and range of the LiDAR scanning. The external parameters describe the position and posture of the LiDAR in the world coordinate system.

[0036] Furthermore, in step S6, the preprocessing of the ROI area specifically includes:

[0037] Cropping: Cropping the ROI area from the original image;

[0038] Scaling: Adjust the size of the ROI area to a fixed width (W) and height (H). Since the ROI areas in the original image have different sizes, scaling can adapt to the requirements of the model input;

[0039] Image normalization: Adjust the numerical range of image data to a specific interval (usually [0, 1] or [-1, 1]) to eliminate the impact of different data scales on model training. Image normalization helps to speed up the convergence of the model and improve the performance of the model.

[0040] Furthermore, the scale scaling method includes:

[0041] Proportional scaling: Keep the aspect ratio of the ROI area unchanged and scale it to the specified size according to the shortest or longest side. This method may cause the ROI area to not completely fill or exceed the target size after scaling, so additional processing (such as cropping or padding) is required;

[0042] Direct scaling: does not maintain the aspect ratio, but directly scales the ROI area to the specified size;

[0043] Fill scaling: On the basis of maintaining the aspect ratio, the ROI area is filled to a specified size. The filling methods include using background color, mean filling, and mirror filling.

[0044] Furthermore, the image normalization method includes:

[0045] Linear normalization: linearly map the pixel values ​​of an image to a specified interval. For example, for an 8-bit image (pixel value range is [0,255]), it can be normalized to the [0,1] interval using the formula (pixel value - minimum value) / (maximum value - minimum value);

[0046] Standardization: On the basis of normalization, the distribution of data is further adjusted to a standard normal distribution (mean 0, variance 1), which usually involves subtracting the mean and dividing by the standard deviation.

[0047] Furthermore, in step S6, according to the consistency principle of the tracked target, all frames of the obstacle in the time series are assigned the final subdivision prediction category.

[0048] The present invention provides a 3D obstacle subdivision attribute pre-labeling method based on point cloud and image, which has the following beneficial effects:

[0049] The present invention realizes the pre-labeling of 3D obstacle subdivision attributes based on point cloud and image data. Since the method fully mines the semantic information of the image through the ROI region association method and the deep learning network, the pre-labeling of obstacle subdivision attributes is realized, the accuracy is also guaranteed, and the workload of subsequent manual labeling is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a logic block diagram of a 3D obstacle subdivision attribute pre-labeling method based on point cloud and image of the present invention;

[0051] Figure 2 A multi-view projection schematic diagram of a 3D obstacle subdivision attribute pre-labeling method based on point cloud and image according to the present invention;

[0052] Figure 3A schematic diagram of finding vertex boundaries of a 3D obstacle subdivision attribute pre-labeling method based on point cloud and image according to the present invention;

[0053] Figure 4 A schematic diagram of a segmentation prediction model of a 3D obstacle segmentation attribute pre-labeling method based on point cloud and image according to the present invention;

[0054] Figure 5 The present invention is a schematic diagram of the ROI area stitching and feature extraction operation flow of a 3D obstacle subdivision attribute pre-labeling method based on point cloud and image. DETAILED DESCRIPTION

[0055] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0056] like Figure 1-Figure 5 As shown, a 3D obstacle segmentation attribute pre-labeling method based on point cloud and image includes the following steps:

[0057] S1. 3D multi-target tracking:

[0058] Assume that a frame of data contains point cloud data provided by a lidar sensor and multi-view images provided by multiple cameras. For this frame of data, CenterPoint is first used as the point cloud 3D target detection algorithm to obtain the 3D detection frame of each obstacle, and the data association and tracking of the detected target are realized through the Hungarian matching algorithm and Kalman filter. Finally, the offset, scale, rotation angle, large category attributes, and unique identity of each obstacle in the XYZ axis direction are obtained. Each obstacle is represented by the following parameters:

[0059] (x,y,z,sizex,sizey,sizez.roll,pitch,yaw,class,trackid);

[0060] In this embodiment, the process of obtaining the 3D detection frame of the obstacle is as follows:

[0061] Data preparation and preprocessing: Load point cloud data (obtain raw point cloud data from the lidar sensor, which usually contains the 3D coordinates of each point and other possible attributes), and filter the point cloud using voxel grid filtering, statistical filtering and other methods to remove noise and unnecessary points, and use random downsampling, uniform grid sampling and other methods to downsample to reduce the amount of calculation and remove noise, and then divide the processed point cloud data into a series of 3D voxels;

[0062] Feature extraction and encoding: Encode the features of the points in each voxel and extract the feature vector that can represent the voxel (these features include the number, density, average height, average intensity and other statistical information of the points in the voxel). Use a 3D feature encoder (such as VoxelNet or PointPillars) to convert the voxel features into a two-dimensional feature map (i.e., a bird's-eye view or BEV map). This step reduces the dimensionality of the three-dimensional point cloud data to a two-dimensional space, which facilitates subsequent convolution operations.

[0063] CenterPoint model reasoning: The generated feature map is used as the input of the CenterPoint model. The model performs forward propagation and generates a center point heat map (indicating the center point confidence of different categories of targets, and finding the peak point as the detected target center through non-maximum suppression (NMS)) and regression targets (including coordinate regression, size regression, height regression, rotation regression, etc., which are used to regress the complete 3D bounding box of the target from the center point feature) through a series of convolutional layers, batch normalization (BN) layers, and activation function (such as ReLU) layers.

[0064] Generate 3D detection box: At the center position of each detected target, extract the coordinates, size, rotation and other attributes from the regression head result to form a 3D detection box;

[0065] Multi-sensor fusion: Fusion of the information contained in the multi-view images provided by the camera with the point cloud detection results to further improve the accuracy and robustness of detection;

[0066] In this embodiment, the Kalman filter prediction step: at each new time step (frame), the Kalman filter is used to predict the state of the obstacle, that is, based on the state estimation of the previous time step and the dynamic model of the system (such as uniform motion, uniform accelerated motion, etc.), and the prediction result will give the estimated value of the obstacle state at the current time step and its uncertainty;

[0067] In this embodiment, the Hungarian matching algorithm associates the following steps:

[0068] Construct the cost matrix: Calculate the cost matrix between all detected obstacles at the current time step and the obstacles predicted by the Kalman filter at the previous time step (the elements of the cost matrix are based on some distance metric between the detection box and the prediction box, such as IOU, Mahalanobis distance, or appearance feature distance);

[0069] Hungarian matching: The Hungarian matching algorithm is used to optimally match the cost matrix to associate the detection box of the current time step with the prediction box of the previous time step. The matching result will give the corresponding relationship between the detection box and the obstacle, and allow the obstacle status to be updated.

[0070] Kalman filter update: Based on the results of the Hungarian matching algorithm, the detection box information of the current time step is used to update the state estimate of the Kalman filter, which includes adjusting the value of the state vector and updating the covariance matrix to reflect the impact of the new observation data on the obstacle state;

[0071] S2, 3D tracking results are projected onto the image:

[0072] The 8 vertices of the 3D box of the obstacle are calculated, and the internal and external parameters of the camera and lidar are used to project the 8 vertices of the 3D box onto the image of each perspective through formula 1, as shown in Figure 2 As shown, if there is no target obstacle on the projection image of a certain perspective, the image of this perspective is skipped;

[0073]

[0074] In this embodiment, the internal and external parameters of the camera and the laser radar are as follows:

[0075] Camera internal and external parameters: The camera's internal parameters include focal length, optical center and other internal parameters, which describe the geometric and optical characteristics of camera imaging. The external parameters describe the position and posture of the camera in the world coordinate system.

[0076] LiDAR internal and external parameters: The internal parameters of LiDAR include the emission angle and angle of the laser beam, which determine the accuracy and range of LiDAR scanning. The external parameters describe the position and posture of LiDAR in the world coordinate system.

[0077] S3. ROI area extraction:

[0078] like Figure 3 As shown, for the positions of the 8 vertices of each obstacle on each image, find the upper, lower, left and right boundaries of the 2D polygon X1min, X2max, Y1min, Y2max, and intercept the rectangular area as the ROI area (Region of Interest) of the obstacle on the image;

[0079] S4. Retrieve segmentation prediction model based on major categories:

[0080] According to the large category attributes (cars, pedestrians, and cyclists, which can be freely expanded and are not specifically limited in this embodiment) output by the 3D multi-target tracking algorithm in step S1, a segmented prediction model is selected. Figure 4 , used for subsequent feature extraction and prediction of segmented categories;

[0081] S5. ROI area stitching and feature extraction:

[0082] The ROI area images extracted from multiple perspectives are preprocessed as input data, such as Figure 5 As shown in the figure, firstly, each ROI area is scaled, image normalized and other preprocessing operations are performed to ensure that the data input size is HxW, and then the multi-view image data is spliced ​​in the channel dimension. Taking two-view images as an example, the dimension of the input data is (BatchSize, 2x3, H, W), and then it is passed to the ResNet backbone network for feature extraction. Finally, the target is classified to obtain the subdivision category result of a single frame;

[0083] In this embodiment, the preprocessing of the ROI area specifically includes:

[0084] Cropping: Cropping the ROI area from the original image;

[0085] Scaling: Adjust the size of the ROI area to a fixed width (W) and height (H). Since the ROI areas in the original image have different sizes, scaling can adapt to the requirements of the model input. The scaling methods include:

[0086] Proportional scaling (keep the aspect ratio of the ROI area unchanged, and scale it to the specified size according to the shortest or longest side. This method may cause the ROI area to not be completely filled or exceed the target size after scaling, so additional processing is required, such as cropping or padding), direct scaling (do not maintain the aspect ratio, directly scale the ROI area to the specified size), filling scaling (on the basis of maintaining the aspect ratio, fill the ROI area to the specified size, and the filling methods include using background color, mean filling, and mirror filling);

[0087] Image normalization: adjust the numerical range of image data to a specific interval (usually [0,1] or [-1,1]) to eliminate the impact of different data scales on model training. Image normalization helps to speed up the convergence of the model and improve the performance of the model. The methods of image normalization include: linear normalization (linearly mapping the pixel values ​​of the image to a specified interval. For example, for an 8-bit image (pixel value range is [0,255]), it can be normalized to the interval [0,1] by the formula (pixel value - minimum value) / (maximum value - minimum value)), standardization (on the basis of normalization, further adjust the distribution of data to a standard normal distribution (mean is 0, variance is 1), which specifically involves the operations of subtracting the mean and dividing by the standard deviation);

[0088] S6. Fusion of multiple frames and output of final result:

[0089] All frames containing the target in the time series are retrieved through the unique identity of the obstacle in the data set, and steps S1 to S5 are repeated for each frame of data to obtain the subdivided category prediction result of each frame, and the category with the highest predicted category in the total number of frames is selected as the final predicted category of the obstacle; and according to the consistency principle of the tracked target, all frames of the obstacle in the time series will be assigned the final subdivided prediction category.

[0090] The embodiments of the present invention are given for the purpose of illustration and description, and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A 3D obstacle subdivision attribute pre-labeling method based on point cloud and image, characterized in that: The following steps are involved: S1. 3D multi-target tracking: Assume that a frame of data contains point cloud data provided by a lidar sensor and multi-view images provided by multiple cameras. For this frame of data, CenterPoint is first used as the point cloud 3D target detection algorithm to obtain the 3D detection frame of each obstacle, and the data association and tracking of the detected target are realized through the Hungarian matching algorithm and Kalman filter. Finally, the offset, scale, rotation angle, large category attributes, and unique identity of each obstacle in the XYZ axis direction are obtained. Each obstacle is represented by the following parameters: (x,y,z,sizex,sizey,sizez.roll,pitch,yaw,class,trackid); S2, 3D tracking results are projected onto the image: Calculate 8 vertices through the 3D box of the obstacle, and use the internal and external parameters of the camera and lidar to project the 8 vertices of the 3D box onto the image of each perspective through formula 1; S3. ROI area extraction: For the positions of the eight vertices of each obstacle on each image, find the upper, lower, left, and right boundaries of the 2D polygon X1min, X2max, Y1min, and Y2max, and intercept the rectangular area as the ROI area of ​​the obstacle on the image; S4. Retrieve segmentation prediction model based on major categories: According to the large category attributes output by the 3D multi-target tracking algorithm in step S1, a subdivision prediction model is selected for subsequent feature extraction and prediction of subdivision categories; S5. ROI area stitching and feature extraction: The ROI area images extracted from multiple perspectives are preprocessed for input data. First, each ROI area is preprocessed to ensure that the data input size is HxW. Then, the multi-view image data is spliced ​​in the channel dimension and then passed to the ResNet backbone network for feature extraction. Finally, the target is classified to obtain the subdivision category result of a single frame. S6. Fusion of multiple frames and output of final result: All frames containing the target in the time series are retrieved through the unique identity of the obstacle in the data set, and steps S1 to S5 are repeated for each frame of data to obtain the subdivided category prediction result of each frame, and the category with the highest predicted category in the total number of frames is selected as the final predicted category of the obstacle.

2. The method for pre-labeling 3D obstacle subdivision attributes based on point cloud and image according to claim 1, characterized in that: In step S1, the process of obtaining the 3D detection frame of the obstacle is as follows: Data preparation and preprocessing: Load point cloud data, filter the point cloud using voxel grid filtering and statistical filtering, and downsample using random downsampling and uniform grid sampling, and then divide the processed point cloud data into a series of 3D voxels; Feature extraction and encoding: Perform feature encoding on the points within each voxel, extract the feature vector that can represent the voxel, and use the 3D feature encoder to convert the voxel features into a two-dimensional feature map; CenterPoint model reasoning: The generated feature map is used as the input of the CenterPoint model. The model performs forward propagation and generates a center point heat map and regression target through a series of convolutional layers, batch normalization layers, and activation function layers. Generate 3D detection box: At the center position of each detected object, extract the coordinates, size, rotation and other attributes from the regression head result to form a 3D detection box; Multi-sensor fusion: Fusion of the information contained in the multi-view images provided by the camera with the point cloud detection results.

3. The method for pre-labeling 3D obstacle subdivision attributes based on point cloud and image according to claim 1, characterized in that: In step S1, the Kalman filter prediction step: at each new time step, the state of the obstacle is predicted using the Kalman filter, that is, based on the state estimation of the previous time step and the dynamic model of the system, and the prediction result will give the estimated value of the obstacle state at the current time step and its uncertainty.

4. The method for pre-labeling 3D obstacle subdivision attributes based on point cloud and image according to claim 1, characterized in that: In step S1, the Hungarian matching algorithm association steps are: Construct the cost matrix: Calculate the cost matrix between all obstacles detected in the current time step and the obstacles predicted by the Kalman filter in the previous time step; Hungarian matching: The Hungarian matching algorithm is used to optimally match the cost matrix to associate the detection box of the current time step with the prediction box of the previous time step. The matching result will give the corresponding relationship between the detection box and the obstacle, and allow the obstacle status to be updated. Kalman filter update: Based on the results of the Hungarian matching algorithm, the detection box information of the current time step is used to update the state estimate of the Kalman filter.

5. The method for pre-labeling 3D obstacle subdivision attributes based on point cloud and image according to claim 1, characterized in that: In the step S2, when performing projection at different viewing angles, if there is no target obstacle on the projection image at a certain viewing angle, the image at that viewing angle is skipped.

6. The method for pre-labeling 3D obstacle subdivision attributes based on point cloud and image according to claim 1, characterized in that: In step S2, the internal and external parameters of the camera and the laser radar are as follows: Camera internal and external parameters: The camera's internal parameters include focal length, optical center, and other internal parameters that describe the geometric and optical characteristics of camera imaging, while the external parameters describe the position and posture of the camera in the world coordinate system; LiDAR internal and external parameters: The internal parameters of LiDAR include the emission angle and angle of the laser beam, which determine the accuracy and range of the LiDAR scanning. The external parameters describe the position and posture of the LiDAR in the world coordinate system.

7. The method for pre-labeling 3D obstacle subdivision attributes based on point cloud and image according to claim 1, characterized in that: In step S6, the preprocessing of the ROI area specifically includes: Cropping: Cropping the ROI area from the original image; Scaling: adjust the size of the ROI area to a fixed width and height; Image normalization: Adjust the numerical range of image data to a specific interval to eliminate the impact of different data scales on model training.

8. The method for pre-labeling 3D obstacle subdivision attributes based on point cloud and image according to claim 7, characterized in that: The scaling method includes: Proportional scaling: Keep the aspect ratio of the ROI area unchanged and scale it to the specified size according to the shortest or longest side; Direct scaling: does not maintain the aspect ratio, but directly scales the ROI area to the specified size; Fill scaling: On the basis of maintaining the aspect ratio, the ROI area is filled to a specified size. The filling methods include using background color, mean filling, and mirror filling.

9. The method for pre-labeling 3D obstacle subdivision attributes based on point cloud and image according to claim 7, characterized in that: The image normalization method comprises: Linear normalization: linearly map the pixel values ​​of an image to a specified interval; Standardization: On the basis of normalization, the distribution of data is further adjusted to the standard normal distribution.

10. The method for pre-labeling 3D obstacle subdivision attributes based on point cloud and image according to claim 1, characterized in that: In step S6, according to the consistency principle of the tracked target, all frames of the obstacle in the time series are assigned the final subdivision prediction category.

Images