A BEVFusion industrial inspection robot positioning method based on multi-scale wavelet transform

Abstract

The application discloses a BEVFusion industrial inspection robot positioning method based on multi-scale wavelet transform, multi-frequency domain decomposition is carried out on the input characteristics of a multi-modal sensor through discrete wavelet transform, low-low frequency, high-low frequency, low-high frequency and high-high frequency information are separated out, and the enhanced features are realized by combining a convolutional neural network and a full convolutional neural network. Specifically, the improved CNN decomposes and enhances the camera features, and then reconstructs the camera features through inverse wavelet transform, and meanwhile, the extracted laser radar point cloud features are combined, so that the camera and point cloud features are fused, and the improved RCNN deeply fuses the camera and laser radar features. The application of the method significantly improves the target detection range and semantic segmentation capability in a dynamic environment, and provides important technical support for the application and development of the fields of autonomous navigation of an industrial robot, monitoring of a high-risk environment device and multi-sensor collaborative perception of an intelligent manufacturing system.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, robot autonomous navigation and multi-sensor fusion SLAM technology, and in particular relates to a BEVFusion industrial inspection robot localization method based on multi-scale wavelet transform. Background Technology

[0002] In the field of industrial inspection robots, BEVFusion technology has been widely applied to autonomous inspection and intelligent detection tasks. Its ability to fuse multimodal sensor data into a unified bird's-eye view significantly improves the completeness of environmental perception and the accuracy of decision-making. However, existing BEVFusion methods still face many challenges in complex industrial environments. For example, cameras are easily interfered with in bright reflective or strongly changing lighting conditions, leading to distorted feature extraction; LiDAR data often exhibits severe sparsity at long distances or in highly reflective areas, reducing the accuracy of spatial structure reconstruction; multi-sensor data fusion also relies on precise time synchronization and spatial calibration, but vibrations and temperature variations common in industrial settings can easily cause registration errors, affecting the overall fusion quality. Simultaneously, dynamic occlusion and unstructured interference (such as moving machinery or personnel) can also interfere with the perception results, further reducing the stability of the BEV view. Therefore, improving the perception robustness, fusion efficiency, and semantic understanding capabilities of BEVFusion in industrial scenarios has become a core issue that urgently needs to be addressed in current research and practical applications in this field. Summary of the Invention

[0003] Objective: This invention aims to provide a BEVFusion-based localization method for industrial inspection robots using multi-scale wavelet transform. The improved BEVFusion method optimizes the robot's autonomous localization performance in low-texture areas, highly reflective surfaces, complex lighting conditions, and dynamic occlusion environments. This addresses issues such as insufficient feature extraction accuracy and inaccurate localization caused by low texture, strong lighting, and occlusion interference in industrial inspection environments, thereby enhancing the localization stability and accuracy of inspection robots in dynamic and complex industrial scenarios, and ultimately improving the applicability and industrial deployment efficiency of BEVFusion technology.

[0004] Technical solution: The present invention provides a BEVFusion industrial inspection robot localization method based on multi-scale wavelet transform, comprising the following steps:

[0005] Step S1: Acquire camera image data and LiDAR point cloud data as multimodal data, and perform color normalization, distortion correction and scaling on the image data; at the same time, denoise the LiDAR point cloud data.

[0006] Step S2: Construct a BEVFusion model, use a CNN that integrates multi-scale wavelet transform to extract image features from the image data, and combine viewpoint transformation to project the features from the perspective view onto the bird's-eye view BEV space so that it conforms to the bird's-eye view BEV representation format, thus obtaining the camera BEV features.

[0007] Step S3: Use SCN to extract point cloud features from the LiDAR point cloud data, unfold the point cloud features into the bird's-eye view BEV space, and align them with the camera BEV features; at the same time, use interpolation methods and feature filling techniques to fill in the data gaps in the BEV grid to obtain the LiDAR BEV features.

[0008] Step S4: The camera BEV features and the lidar BEV features are fused using the RCNN improved by multi-scale wavelet transform to obtain the BEV fused features;

[0009] Step S5: Use the CenterPoint object detection head to process BEV fusion features to achieve 3D object detection; at the same time, use U-Net to predict road elements and generate a BEV semantic map, enabling the industrial inspection robot to have environmental perception capabilities.

[0010] Step S6: Train the BEVFusion model, convert the trained BEVFusion model into an INT8 quantized model and an FP16 model, and deploy it in the industrial inspection robot localization task.

[0011] Furthermore, step S1 specifically involves: firstly, using mean-standard deviation normalization to perform color normalization on the camera image data:

[0012]

[0013] Where I is the original image; I' is the normalized image; μ is the mean of the RGB channels; and σ is the standard deviation of the RGB channels.

[0014] Distortion correction of camera images involves remapping the image coordinates using the camera's intrinsic parameter matrix K and distortion parameters.

[0015]

[0016] Among them, f x and f y These are the focal lengths in pixel coordinates; c x and c y These are the coordinates of the optical center. Then, the camera image is scaled using the affine transformation matrix S:

[0017]

[0018] Among them, (W) o H o (W) represents the original image size; t H t The target size is defined as follows; simultaneously, SOR is used to denoise the LiDAR point cloud data:

[0019]

[0020] Where k represents the number of nearest neighbors, i.e., the number of neighboring points considered when calculating the average distance; p i p represents the target point in the current point cloud; j Indicates with p i The j-th point among the k nearest points; d i Point p i The average Euclidean distance to its k nearest neighbors is used to determine whether a point is an outlier; that is, if d i >μ d +λσ d If μ is an outlier, it is considered an outlier and is removed. d σ represents the average neighborhood distance of all points. d λ represents the standard deviation of the neighborhood distances of all points, and λ represents the hyperparameter of the filtering threshold.

[0021] Furthermore, step S2 specifically involves: firstly, decomposing the camera image using a Haar wavelet-based DWT:

[0022]

[0023] Where X represents the preprocessed image features; DWT(·) represents the discrete wavelet transform; X LL ,X LH ,X HL ,X HH These represent low-low frequency, low-high frequency, high-low frequency, and high-high frequency components, respectively; f LL ,f LH ,f HL ,f HH The four filters represent the Haar wavelet; subsequently, a CNN is used to extract and enhance features of low-low frequency, low-high frequency, high-low frequency, and high-high frequency components.

[0024]

[0025] Where X' LL ,X' LH ,X' HL ,X' HH This refers to the features extracted and enhanced by the CNN; CNN H (·) indicates a deep CNN; CNN L(·) indicates a shallow CNN;

[0026] Multi-scale features are reconstructed using inverse wavelet transform (IWT):

[0027] X' = ​​IWT(X′) LL ,X′ LH ,X′ HL ,X′ HH )

[0028] Where X' represents the reconstructed camera image features; IWT(·) represents the inverse wavelet transform;

[0029] Finally, the features from the perspective viewpoint are projected into the BEV space through a viewpoint transformation based on homography transformation:

[0030]

[0031] Where H represents the homography transformation matrix; K represents the camera intrinsic parameter matrix; R is the rotation matrix, representing the rotation of the camera relative to the BEV viewpoint; t is the translation vector, representing the offset of the camera relative to the BEV reference plane; n is the normal vector, representing the direction of the BEV reference plane; d represents the distance from the camera to the BEV reference plane; X C The BEV representation of camera features combines global and detailed information while maintaining correct geometric relationships.

[0032] Furthermore, step S3 specifically involves: first, mapping the point cloud data to a 3D voxel mesh, and then using max pooling to aggregate features.

[0033]

[0034] Where P represents the point cloud dataset; p i Represents a single lidar point; (x i ,y i ,z i ) represents point p i Coordinates in 3D space; r i Point p i The reflection intensity; V xyz P represents the index value in the voxel grid. min The minimum coordinate value of the point cloud data is represented by v; the resolution of the voxel grid is represented by v; and F(x,y,z) represents the 3D voxel features after max pooling.

[0035] Subsequently, the 3D voxel representation is processed using SCN through sparse convolution:

[0036]

[0037] Among them, Wsc Ω represents the sparse convolution kernel weights; (i,j,k) represents the convolution window range; (i,j,k) represents the index offset of the convolution operation window; F'(x,y,z) represents the 3D voxel features processed by SCN.

[0038] Next, the point cloud features are expanded into the BEV space by max pooling in the Z-axis direction using the maximum height projection method, so that the BEV grid cell values ​​depend on the highest point features:

[0039]

[0040] Where V(x,y) represents the point cloud features in the BEV space; F L This represents the BEV point cloud features after Z-axis pooling.

[0041] Finally, the BEV point cloud features are initially aligned with the BEV features in the image:

[0042] F=λX C +(1-λ)F L

[0043] Where F represents the initially aligned BEV features; λ represents the hyperparameter that controls the initial weights of different modal features and determines the importance of camera and radar features;

[0044] Simultaneously, bilinear interpolation is used to calculate the weighted average of neighboring points to fill in the missing data areas in the BEV grid, and K-nearest neighbor (KNN) interpolation is combined for further optimization to form the optimized preliminary aligned BEV features:

[0045]

[0046] Where F'(x,y) represents the BEV feature filled by interpolation; w ij The interpolation weights represent the contribution of neighboring points to the interpolation point; (x i ,y j ) represents the coordinates of nearby known points; F"" represents the BEV feature optimized by KNN nearest neighbor interpolation; K represents the number of points interpolated by KNN nearest neighbors; F"' represents the optimized BEV feature with initial alignment.

[0047] Furthermore, step S4 specifically involves: firstly, using Haar wavelet-based DWT to decompose the initially aligned BEV features:

[0048] [F LL ,F LH ,F HL ,F HH ] = DWT(F”')

[0049] Among them, FLL ,F LH ,F HL ,F HH These represent low-low frequency, low-high frequency, high-low frequency, and high-high frequency components, respectively. Then, R-CNN is used to extract features from these components.

[0050] F LL ',F LH ',F HL ',F HH '=Conv θ (F LL ,F LH ,F HL ,F HH )

[0051] Among them, Conv θ (·) represents the RCNN convolution operation with parameter θ; finally, the decomposed features are reconstructed using IWT:

[0052] F B =IWT(F LL ',F LH ',F HL ',F HH ')

[0053] Among them, F B This represents the BEV features after deep fusion, used for target detection and environmental perception.

[0054] Furthermore, step S5 specifically involves: first, using CenterPoint to calculate whether each pixel belongs to the target center point based on the final BEV feature through dense prediction, and then regressing the detection results.

[0055]

[0056] Where H, W, and C represent the height, width, and number of channels of the BEV feature map, respectively; Y c (x,y) represents the predicted heatmap response value of the target center point, indicating the probability that the pixel belongs to the target center point; (x,y) represents the pixel coordinates in the BEV space; (x c ,y c ) represents the true coordinates of the target's center point in the BEV space; σ represents the parameter controlling the extent of Gaussian distribution expansion; b represents the predicted 3D bounding box value of the target; x c ,y c ,z c The coordinates of the target's center point are represented by w, l, and h; the dimensions of the target are represented by w, l, and h; the direction angle of the target is represented by θ; and v represents the target's orientation angle. x ,v yRepresents the velocity vector of the target;

[0057] Using the U-Net encoder-decoder architecture for BEV semantic segmentation:

[0058]

[0059] Where S represents the BEV semantic map after semantic segmentation; U-Net(·) represents the U-Net semantic segmentation network; represents the encoder features of the l-th layer of U-Net; Conv(·) represents a 2D convolution operation used to extract semantic features; M(·) represents a max pooling operation used to reduce the resolution of the feature map; This represents the decoder features of the l-th layer of U-Net; Up(·) represents the upsampling operation; denoted as a skip connection; P(S(x,y,c)) represents the probability that pixel (x,y) belongs to category c from the BEV perspective; c represents the semantic category index; S(x,y,c) represents the prediction score of the semantic segmentation network for pixel (x,y) belonging to category c.

[0060] Furthermore, step S6 specifically involves quantizing the trained BEV model using INT8 quantization.

[0061]

[0062] Where, round(·) represents INT8 quantization; S is the scaling factor used to map FP32 data to the INT8 range; Z is the zero-point offset used to ensure zero-point alignment; F B ' represents the BEV perspective feature after INT8 quantization; Y c (x,y) represents the predicted value of the target center point heatmap after INT8 quantization; b' represents the target bounding box information after INT8 quantization; S' represents the predicted result of the quantized INT8 semantic map.

[0063] The trained BEV model is quantized using FP16 quantization:

[0064]

[0065] Where float16(·) represents FP16 quantization; F B ' represents the BEV viewpoint characteristics after FP16 quantization; Y c "(x,y) represents the predicted value of the target center point heatmap after FP16 quantization; b" represents the target bounding box information after FP16 quantization; S" represents the predicted result of the FP16 semantic map after quantization.

[0066] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the present invention.

[0067] The present invention also discloses a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the method of the present invention.

[0068] The present invention also discloses a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method of the present invention.

[0069] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: Attached Figure Description

[0070] Figure 1 A schematic diagram of the BEVFusion industrial inspection robot localization method based on multi-scale wavelet transform;

[0071] Figure 2 Improved BEV feature fusion flowchart. Detailed Implementation

[0072] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0073] like Figure 1 , 2 As shown, the BEVFusion industrial inspection robot localization method based on multi-scale wavelet transform of the present invention includes:

[0074] Step S1: Input multimodal data into the system, namely camera images and LiDAR point cloud data, and perform color normalization, distortion correction and scaling on the image data; at the same time, denoise the LiDAR point cloud data to reduce data noise and improve the accuracy of feature matching.

[0075] In step S1, mean-standard deviation normalization is first used to normalize the camera image data to reduce the impact of different camera devices on lighting variations, white balance, and exposure differences, ensuring the consistency and stability of the input data. Subsequently, distortion correction is performed on the camera images to eliminate barrel or pincushion distortion caused by wide-angle or fisheye lenses, ensuring that the inspection robot's visual information accurately reflects the spatial structure of the actual industrial environment. Next, an affine transformation matrix is ​​used to scale the camera images to match image data of different resolutions, facilitating subsequent feature matching and multi-view fusion. Simultaneously, statistical filtering (SOR) is used to denoise the LiDAR point cloud data to remove outliers and isolated points caused by dust, obstacles, and dynamic interference in the industrial environment, improving the accuracy of feature matching and the positioning precision of the inspection robot.

[0076] Step S2: Use a CNN that incorporates multi-scale wavelet transform to enhance image features, and combine viewpoint transformation to project the features from the perspective viewpoint into the BEV space so that it conforms to the BEV representation format, which facilitates fusion with LiDAR point cloud features.

[0077] In step S2, the camera image is first decomposed into low-low frequency, high-low frequency, low-high frequency, and high-high frequency components using Haar wavelet-based DWT. The low-low frequency components are used to enhance the overall scene perception capability, while the high-low frequency, low-high frequency, and high-high frequency components are used to optimize the clarity of local features. Then, CNN is used to extract and enhance the features of the low-low frequency, high-low frequency, low-high frequency, and high-high frequency components. Specifically, deep CNN extracts and enhances the global semantic information of the low-low frequency components, while shallow CNN extracts and enhances the local edges and textures of the high-low frequency, low-high frequency, and high-high frequency components. Next, IWT is used to reconstruct the multi-scale features enhanced by the CNN to ensure that high- and low-frequency information can work synergistically, improving the accuracy and stability of the inspection robot's target object recognition. Finally, perspective transformation based on homography is used to project the features from the perspective viewpoint into the BEV space to conform to the BEV representation format, enabling accurate fusion with the LiDAR point cloud features and providing rich visual input for the inspection robot's high-precision target detection and environmental understanding.

[0078] Step S3: Process the LiDAR point cloud data using a sparse convolutional neural network (SCN) to extract the spatial structure information of the 3D object. Then, unfold the point cloud features into the BEV space and perform preliminary alignment with the BEV features of the image. Simultaneously, interpolation methods and feature imputation techniques are used to compensate for data gaps in the BEV grid.

[0079] In step S3, the point cloud data is first mapped to a 3D voxel grid, and max pooling is used for feature aggregation to generate a compact 3D voxel representation. Then, SCN is used to process the 3D voxel representation through sparse convolution to extract the spatial structure information of the 3D object and generate a high-dimensional feature representation. Next, max pooling is performed along the Z-axis using the maximum height projection method to expand the point cloud features into the BEV space, making the BEV grid cell values ​​dependent on the highest point features, thus preserving obstacle height information. Finally, the BEV point cloud features are initially aligned with the BEV image features to ensure fusion at the same scale. Simultaneously, bilinear interpolation is used to calculate the weighted average of neighboring points to fill in missing data areas in the BEV grid, and K-nearest neighbor (KNN) interpolation is combined for further optimization, making the BEV grid features smoother and more continuous, forming optimized preliminary aligned BEV features that provide high-quality input data for subsequent multimodal fusion and object detection.

[0080] Step S4: The camera BEV features and the lidar BEV features are deeply fused through the improved RCNN using multi-scale wavelet transform, thereby forming a more stable BEV fusion feature.

[0081] In step S4, the initially aligned BEV features are first decomposed into low-low frequency, high-low frequency, low-high frequency, and high-high frequency components using Haar wavelet-based DWT, thus achieving more refined modal feature modeling. Subsequently, RCNN is used to extract features from these components, enabling the extraction of multi-scale, multi-directional, and multi-modal feature representations while preserving the sub-band feature space structure. Finally, the decomposed features are reconstructed using IWT to obtain the final deeply fused BEV features. This ensures that the wavelet-transformed BEV features maintain good information representation capabilities even in high-resolution environments. These features not only accurately represent target objects in complex environments but also provide richer semantic information in different task scenarios, improving the accuracy and stability of target detection, environmental perception, and autonomous navigation.

[0082] Step S5: Use the CenterPoint object detection head to process the final BEV features to achieve 3D object detection; at the same time, use U-Net to predict road elements and generate a high-precision BEV semantic map to ensure that the industrial inspection robot has accurate environmental perception capabilities.

[0083] In step S5, CenterPoint is first used to calculate whether each pixel belongs to the target center point using a dense prediction method on the final BEV features. The detection results, including center coordinates, size, orientation angle, and velocity information, are then regressed for subsequent path planning and environmental understanding by the inspection robot. Simultaneously, U-Net is used to extract multi-scale features of the final BEV through a downsampling encoder. Then, spatial resolution is gradually restored through skip connections and an upsampling decoder to predict semantic information such as roads, lanes, obstacles, and parking areas in the BEV space, generating a high-precision BEV semantic map. This provides the industrial inspection robot with comprehensive environmental perception and path planning information, ensuring its efficient and stable operation in complex industrial environments.

[0084] Step S6: Convert the trained BEVFusion model into an INT8 quantized model and an FP16 model so that it can be efficiently deployed in the localization task of industrial inspection robots.

[0085] In step S6, the trained BEV model is quantized using both INT8 and FP16 quantization methods to adapt to different hardware architectures and enable efficient deployment on industrial inspection robot platforms. INT8 quantization reduces computational overhead by converting the model's weights and activation values ​​to 8-bit integers, making it more suitable for operation on resource-constrained embedded devices or low-power AI computing platforms. FP16 quantization converts 32-bit floating-point numbers to 16-bit floating-point numbers, reducing storage requirements while retaining floating-point computational capabilities. Both quantized models can perform 3D object detection, semantic segmentation, environmental perception, and path planning tasks with lower power consumption and higher inference speed in resource-constrained computing environments, ensuring stable operation of the industrial inspection robot in complex industrial environments and improving inspection efficiency and system response speed.

Claims

1. A BEVFusion industrial inspection robot localization method based on multi-scale wavelet transform, characterized in that, Includes the following steps: Step S1: Acquire camera image data and LiDAR point cloud data as multimodal data, and perform color normalization, distortion correction and scaling on the image data; at the same time, denoise the LiDAR point cloud data. Step S2: Construct a BEVFusion model, use a CNN that integrates multi-scale wavelet transform to extract image features from the image data, and combine viewpoint transformation to project the features from the perspective view onto the bird's-eye view BEV space so that it conforms to the bird's-eye view BEV representation format, thus obtaining the camera BEV features. Step S2 specifically involves: First, using Haar wavelet-based DWT to decompose the camera image: ； in, This represents the features of the preprocessed image; Represents the discrete wavelet transform; These represent low-low frequency, low-high frequency, high-low frequency, and high-high frequency components, respectively. The four filters represent the Haar wavelet; subsequently, a CNN is used to extract and enhance features of low-low frequency, low-high frequency, high-low frequency, and high-high frequency components. ； in, This represents the features extracted and enhanced by the CNN; This represents a deep CNN; This represents a shallow CNN; Multi-scale features are reconstructed using inverse wavelet transform (IWT): ； in, Represents the features of the reconstructed camera image; Indicates inverse wavelet transform; Finally, the features from the perspective viewpoint are projected into the BEV space through a viewpoint transformation based on homography transformation: ； in, Represents the homography transformation matrix; Represents the camera intrinsic parameter matrix; Let be the rotation matrix, representing the rotation relationship of the camera relative to the BEV viewpoint; is the translation vector, representing the offset of the camera relative to the BEV reference plane; is the normal vector, representing the direction of the BEV reference plane; This represents the distance from the camera to the BEV reference plane; The BEV representation of camera features combines global and detailed information while maintaining correct geometric relationships; Step S3: Use SCN to extract point cloud features from the LiDAR point cloud data, unfold the point cloud features into the bird's-eye view BEV space, and align them with the camera BEV features; at the same time, use interpolation methods and feature filling techniques to fill in the data gaps in the BEV grid to obtain the LiDAR BEV features. Step S4: The camera BEV features and the lidar BEV features are fused using the RCNN improved by multi-scale wavelet transform to obtain the BEV fused features; Step S5: Use the CenterPoint object detection head to process BEV fusion features to achieve 3D object detection; at the same time, use U-Net to predict road elements and generate a BEV semantic map, enabling the industrial inspection robot to have environmental perception capabilities. Step S6: Train the BEVFusion model, convert the trained BEVFusion model into an INT8 quantized model and an FP16 model, and deploy it in the industrial inspection robot localization task.

2. The BEVFusion industrial inspection robot localization method based on multi-scale wavelet transform according to claim 1, characterized in that, Step S1 specifically involves: First, using mean-standard deviation normalization to perform color normalization on the camera image data. ； in, Original image; The image is after normalization; This is the average value of the RGB channels; Standard deviation of RGB channels; Distortion correction of camera images is performed using the camera's intrinsic parameter matrix. Remapping image coordinates using distortion parameters: ； in, and These are the focal lengths in pixel coordinates; and These are the coordinates of the optical center. Then, an affine transformation matrix is ​​used. Scale the camera image: ； in, Original image size; Target size; simultaneously, SOR is used to denoise the LiDAR point cloud data: ； in, This indicates the number of nearest neighbors, i.e., the number of neighboring points considered when calculating the average distance; This represents the target point in the current point cloud; Indicates and Recent The first point in the nth point One point; Point With The average Euclidean distance between nearest neighbors is used to determine whether a point is an outlier, i.e., if... If a point is found to be out of the question, it is considered an outlier and removed from the list. This represents the average neighborhood distance of all points. The standard deviation of the neighborhood distances of all points. The hyperparameter representing the filtering threshold.

3. The BEVFusion industrial inspection robot localization method based on multi-scale wavelet transform according to claim 1, characterized in that, Step S3 specifically involves: first, mapping the point cloud data to a 3D voxel mesh, and then using max pooling to aggregate features. ； in, Represents a point cloud dataset; Indicates a single lidar point; Point Coordinates in 3D space; Point The intensity of reflection; This represents the index value in the voxel grid; This represents the minimum coordinate value of the point cloud data; Indicates the resolution of the voxel grid; This represents the 3D voxel features after max pooling. Subsequently, the 3D voxel representation is processed using SCN through sparse convolution: ； in, Indicates the sparse convolution kernel weights; Indicates the range of the convolution window; Indicates the index offset of the convolution operation window; This represents 3D voxel features processed by SCN; Next, the point cloud features are expanded into the BEV space by max pooling in the Z-axis direction using the maximum height projection method, so that the BEV grid cell values ​​depend on the highest point features: ； in, This represents the point cloud features in the BEV space; This represents the BEV point cloud features after Z-axis pooling. Finally, the BEV point cloud features are initially aligned with the BEV features in the image: ； in, Indicates the initial alignment of BEV features; The hyperparameters represent the initial weights of different modal features and determine the importance of camera and radar features; Simultaneously, bilinear interpolation is used to calculate the weighted average of neighboring points to fill in the missing data areas in the BEV grid, and K-nearest neighbor (KNN) interpolation is combined for further optimization to form the optimized preliminary aligned BEV features: ； in, This represents the BEV features filled using interpolation methods; The interpolation weight represents the contribution of neighboring points to the interpolation point. Represents the coordinates of nearby known points; This represents the BEV features optimized using KNN nearest neighbor interpolation; This represents the number of points used in KNN nearest neighbor interpolation. This represents the optimized, initially aligned BEV features.

4. The BEVFusion industrial inspection robot localization method based on multi-scale wavelet transform according to claim 1, characterized in that, Step S4 specifically involves: First, using Haar wavelet-based DWT to decompose the initially aligned BEV features: ； in, These represent low-low frequency, low-high frequency, high-low frequency, and high-high frequency components, respectively. Then, R-CNN is used to extract features from these components. ； in, The parameter is The RCNN convolution operation is performed; finally, the decomposed features are reconstructed using IWT: ； in, This represents the BEV features after deep fusion, used for target detection and environmental perception.

5. The BEVFusion industrial inspection robot localization method based on multi-scale wavelet transform according to claim 1, characterized in that, Step S5 specifically involves: First, using CenterPoint to calculate whether each pixel belongs to the target center point using a dense prediction method on the final BEV features, and then regressing the detection results. ； in, These represent the height, width, and number of channels of the BEV feature map, respectively. The predicted target center point heatmap response value represents the probability that the pixel belongs to the target center point; Represents pixel coordinates in BEV space; This represents the true coordinates of the center point of the target in BEV space; A parameter that controls the extent of the Gaussian distribution expansion; Represents the predicted 3D bounding box value of the target; Indicates the coordinates of the target's center point; Indicates the size of the target; Indicates the direction angle of the target; Represents the velocity vector of the target; Using the U-Net encoder-decoder architecture for BEV semantic segmentation: ； in, This represents the semantic map of BEV after semantic segmentation. This represents the U-Net semantic segmentation network; Indicates the U-Net number Encoder features of the layer; This represents a 2D convolution operation used to extract semantic features; This indicates a max pooling operation, used to reduce the resolution of the feature map; Indicates the U-Net number Decoder features of the layer; Indicates an upsampling operation; Indicates a skip connection; Represents pixels from the BEV perspective Category The probability of; Represents a semantic category index; This indicates that the semantic segmentation network represents the pixels. Category The predicted score.

6. The BEVFusion industrial inspection robot localization method based on multi-scale wavelet transform according to claim 1, characterized in that, Step S6 specifically involves quantizing the trained BEV model using INT8 quantization. ； in, Indicates INT8 quantization; It is a scaling factor used to map FP32 data to the INT8 range; It is the zero-point offset, used to ensure zero-point alignment; This represents the BEV perspective features after INT8 quantization; This represents the predicted value of the target center point heatmap after INT8 quantization. This represents the target bounding box information after INT8 quantization; This represents the quantized INT8 semantic map prediction result; The trained BEV model is quantized using FP16 quantization: ； in, Indicates FP16 quantization; This represents the BEV viewpoint characteristics after FP16 quantization; This represents the predicted value of the target center point heatmap after FP16 quantization; This represents the target bounding box information after FP16 quantization; This represents the quantized FP16 semantic map prediction result.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 1.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

Images