Visible light camera surround view function implementation method based on surrounding low illumination

Abstract

The invention discloses a method for realizing a surround view function of a visible light camera based on surrounding low illumination. A plurality of cameras and laser radars are carried on the whole body of the quadruped robot; carrying out necessary pre-processing on the acquired image, wherein the pre-processing comprises noise reduction, contrast adjustment, color correction and the like; the method comprises the following steps: preliminarily splicing images acquired by a camera to form a preliminary 360-degree look-around image; the blind area is predicted and filled by using the space positioning information provided by the laser radar and combining the image data of the preorder time; performing further optimization processing, including edge smoothing, color consistency adjustment and the like, on the complemented look-around image; and the real-time around-view image updating is realized. The method has the beneficial effects that the blind area between all-round cameras of the quadruped robot is filled up, high-precision splicing and real-time updating of images are realized, and the environment perception capability and autonomous navigation performance of the robot are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot vision, and in particular to a method for realizing the panoramic view function of a visible light camera based on low illuminance around. Background Art

[0002] In the field of robot vision, panoramic camera systems are widely used because they can provide a panoramic view of the environment around the robot. As a highly flexible automated device, quadruped robots play an important role in search and rescue, environmental monitoring, security patrol and other scenarios. To achieve effective environmental perception and autonomous navigation, quadruped robots are usually equipped with multiple cameras, which are designed to cover the entire area around the robot.

[0003] Traditional panoramic camera systems usually consist of four cameras, which are respectively installed in the front, rear, left and right directions of the robot. Each camera is responsible for capturing the field of view in its corresponding direction, and through image stitching technology, the views in the four directions are synthesized into a complete 360° panoramic image. However, due to the limitations of the physical installation angle and field of view angle of the camera, there are inevitably blind spots between adjacent cameras. These blind spots will seriously affect the robot's comprehensive perception of the surrounding environment and limit its navigation and decision-making ability in complex environments.

[0004] The prior art has the following defects:

[0005] 1. Blind spot problem: The visual blind spots between adjacent cameras make the robot unable to obtain continuous environmental information, which is a major defect for application scenarios that require precise environmental perception.

[0006] 2. Discontinuous image stitching: Due to the existence of blind spots, simple image stitching technology cannot achieve a smooth and continuous panoramic view, resulting in obvious seams or overlaps at the image edges.

[0007] 3. Poor environmental adaptability: The panoramic camera systems in the prior art usually have certain requirements for lighting conditions, and the image quality in low illuminance environments is poor, affecting the robot's visual perception ability.

[0008] 4. Lack of real-time performance: In a dynamic environment, the prior art may not be able to achieve fast and accurate update of the panoramic image, resulting in the robot's response to environmental changes not being timely enough.

[0009] 5. High algorithm complexity: Existing image stitching and blind spot processing algorithms often have high computational complexity and are difficult to run in real time on resource-constrained robot platforms. Summary of the Invention

[0010] The present invention aims to overcome the above-mentioned deficiencies in the prior art and provides a method for realizing the panoramic view function of a visible light camera based on low illuminance around, which has good environmental adaptability, continuous image stitching and high real-time performance.

[0011] To achieve the above object, the present invention adopts the following technical solutions:

[0012] A method for realizing the panoramic view function of a visible light camera based on low illuminance around, with several cameras and lidars mounted on the body of a quadruped robot, specifically including the following steps:

[0013] (1) Collect image information of the surrounding environment through the cameras on the body of the quadruped robot, and collect three-dimensional point cloud data through the lidar. Joint calibration is performed on the camera image data and the lidar point cloud data to obtain the corresponding relationship between the image information and the point cloud information;

[0014] (2) Image preprocessing, perform noise reduction, contrast adjustment, and color correction on the image data collected by the camera;

[0015] (3) Preliminary stitching, extract image feature points and perform matching, and then stitch the images into a complete panoramic image;

[0016] (4) Lidar-assisted stitching, use the point cloud data obtained by the lidar to align with the camera image in space, perform precise registration on the point cloud data, use the image data in the previous time period to fill the blind area, and complete the blind area filling through a time series prediction model;

[0017] (5) Image completion and optimization, perform edge smoothing processing and color consistency adjustment on the image, and update the image in real time.

[0018] In the present invention, a number of cameras are mounted on the whole body of the quadruped robot to jointly cover the omnidirectional field of view around the robot; the lidar mounted on the quadruped robot is used to obtain accurate environmental distance information and the three-dimensional position information of the quadruped robot in real time, providing an accurate spatial reference for the image data; necessary preprocessing is performed on the collected images, including noise reduction, contrast adjustment, color correction, etc., to adapt to different lighting conditions and improve the image quality; the images collected by the cameras are initially stitched to form a preliminary 360° panoramic image; using the spatial positioning information provided by the lidar and combining the image data of the previous time, the blind area is predicted and filled to achieve the continuity and integrity of the panoramic image; further optimization processing is performed on the completed panoramic image, including edge smoothing, color consistency adjustment, etc., to improve the continuity and visual quality of the image; the real-time update of the panoramic image is realized to maintain the accuracy and real-time performance of the panoramic image. Through the above technical solutions, the present invention not only effectively fills the blind area between the panoramic cameras of the quadruped robot, but also realizes the high-precision stitching and real-time update of the images, significantly improving the environmental perception ability and autonomous navigation performance of the robot. This method ensures the accurate judgment of the robot on the surrounding environmental information and enables accurate self-positioning in complex environments.

[0019] Preferably, in step (2), noise reduction processing is performed on the collected images. The Gaussian filter is used to perform noise reduction processing on the collected images. It smooths the images through convolution operations to reduce noise. The mathematical formula of Gaussian filtering is as follows:

[0020]

[0021] where I raw (x, y) is the pixel value of the original image at the coordinate (x, y), σ is the standard deviation of the Gaussian kernel, which controls the smoothing degree of the filter (the larger the value, the stronger the smoothing effect), and * represents the convolution operation.

[0022] Preferably, in step (2), the histogram equalization method is used to enhance the image contrast; histogram equalization enhances the image contrast by redistributing the gray values of the image. Its formula is as follows:

[0023]

[0024] where I eq (x, y) is the gray value of the equalized image at the coordinate (x, y), L is the number of gray levels (for example, 256 levels for an 8-bit image), MN is the total number of image pixels (width × height), and h(i) is the number of pixels with the gray value of i.

[0025] Preferably, in step (2), a white balance algorithm is used to perform color correction to unify the image tone; the white balance algorithm adjusts the ratios of the three colors, red, green, and blue, of the image to achieve the effect of color balance, and its mathematical formula is as follows:

[0026]

[0027] where I wb (x,y) is the red component value of the image after white balance at the coordinate (x,y), R avg is the global average value of the red component of the image, and R(x,y) is the red component value of the original image at the coordinate (x,y).

[0028] Preferably, in step (3), the SIFT algorithm is used to extract and match the image feature points. The specific steps include:

[0029] (31) Key point detection: Use the Difference of Gaussian function to detect the scale-invariant feature points of the image;

[0030] (32) Feature description: Generate descriptors based on the gradient directions around the feature points;

[0031] (33) Feature matching: Use the Euclidean distance to perform descriptor matching.

[0032] Preferably, in step (3), through the calculation of the perspective transformation matrix, the points on one image plane are mapped to another image plane, and its mathematical formula is as follows:

[0033]

[0034] where H is a 3x3 perspective transformation matrix, (x,y) are the points on the original image plane, (x′,y′) are the points on the transformed image plane, and w' is the scaling factor of the homogeneous coordinates, which is used to normalize the coordinates (i.e., x' = x” / w', y' = y” / w').

[0035] Preferably, in step (4), the spatial alignment of the lidar point cloud data and the camera image is achieved through the external calibration matrix T C2L to convert the points in the lidar coordinate system to the camera coordinate system. The conversion formula is as follows:

[0036] P C = T C2L P L

[0037] where P C is the point in the camera coordinate system, P L is the point in the lidar coordinate system, and T C2Lis the external parameter matrix, which is a 4×4 homogeneous transformation matrix containing rotation and translation parameters, and has the following form:

[0038]

[0039] where, R 3x3 is the rotation matrix from the lidar to the camera (converted from Euler angles or quaternions), and t 3x1 is the translation vector from the lidar to the camera. Example: If the lidar is 0.5 meters to the right of the origin of the camera coordinate system, then t = [0.5, 0, 0]^T.

[0040] Preferably, in step (4), the ICP algorithm is used to accurately register the point cloud data. The specific steps include:

[0041] (41) Initial alignment: Select the initial registration points;

[0042] (42) Nearest point matching: Find the nearest point pairs for each point;

[0043] (43) Transformation estimation: Calculate the rigid body transformation matrix with the minimum mean square error;

[0044] (44) Iterative optimization: Update the transformation matrix, and repeat (42) and (43) until convergence.

[0045] Preferably, in step (5), a bilateral filter is used for edge smoothing. The bilateral filter achieves an edge-preserving smoothing effect by considering both the spatial distance and color similarity of pixels. Its mathematical formula is as follows:

[0046]

[0047] where, I smooth (x, y) is the pixel value of the smoothed image at the coordinate (x, y), W is the normalization factor to ensure that the sum of weights is 1, i, j are the neighborhood offsets relative to the central pixel (x, y) (for example, the neighborhood window is 5×5), I(x + i, y + j) is the original pixel value of the neighborhood pixel at the offset (i, j), f d (i, j) is the Gaussian function in the spatial domain, and f r (I(x, y) - I(x + i, y + j)) is the Gaussian function in the intensity domain.

[0048] Preferably, in step (5), a color transfer algorithm is used to adjust the color consistency of the stitched image to ensure the consistency of the visual effect. The color transfer algorithm makes the color distribution of the target image consistent with that of the source image. Its mathematical formula is as follows:

[0049] I color(x, y) = αI1(x, y) + (1 - α)I2(x, y)

[0050] wherein, I color (x, y) is the color value of the adjusted image at the coordinate (x, y), I1(x, y) is the pixel value of the source image (reference hue), I2(x, y) is the pixel value of the target image (to be adjusted), and α is the weight coefficient that controls the adjustment intensity (for example, α = 0.5 represents medium-intensity mixing).

[0051] The beneficial effects of the present invention are as follows: It not only effectively fills the blind spots between the omnidirectional cameras of the quadruped robot, but also realizes high-precision stitching and real-time update of images, significantly improving the robot's environmental perception ability and autonomous navigation performance; it ensures the accurate judgment of the robot on the surrounding environmental information and enables precise self-positioning in complex environments. Description of the Drawings

[0052] Figure 1 is the flowchart of the method of the present invention. Detailed Embodiments

[0053] The following further describes the present invention in conjunction with the drawings and detailed embodiments.

[0054] The present invention aims to solve the blind spot problem that occurs when a quadruped robot uses omnidirectional cameras to achieve 360° omnidirectional images, as well as the problem of discontinuity in image stitching caused by installation angle limitations. As Figure 1 described in the embodiments, the present invention proposes a method for realizing the omnidirectional function of visible light cameras based on low illuminance in the four directions. A number of cameras are mounted on the body of the quadruped robot. Specifically, the quadruped robot is equipped with four wide-angle cameras, which are respectively directed forward, backward, left, and right. Each wide-angle camera has a field of view of 130° - 150°, jointly covering the entire omnidirectional field around the quadruped robot; using the 360° lidar mounted on the quadruped robot to obtain accurate environmental distance information and the three-dimensional position information of the quadruped robot in real time, providing an accurate spatial reference for the image data; performing necessary preprocessing on the collected images, including noise reduction, contrast adjustment, and color correction, etc., to adapt to different lighting conditions and improve the image quality; initially stitching the images collected by the cameras to form a preliminary 360° omnidirectional image; using the spatial positioning information provided by the lidar, combined with the image data of the previous time, predicting and filling the blind spots to achieve the continuity and integrity of the omnidirectional image; further optimizing the filled omnidirectional image, including edge smoothing, color consistency adjustment, etc., to improve the continuity and visual quality of the image; realizing the update of the real-time omnidirectional image to maintain the accuracy and real-time nature of the omnidirectional image.

[0055] Through the above technical solutions, the present invention not only effectively fills the blind spots between the omnidirectional cameras of the quadruped robot, but also realizes high-precision stitching and real-time update of images, significantly improving the environmental perception ability and autonomous navigation performance of the quadruped robot. This method ensures the accurate judgment of the surrounding environmental information of the quadruped robot and enables accurate self-positioning in complex environments.

[0056] Specifically, it includes the following steps:

[0057] (1) Collect image information of the surrounding environment through the cameras around the quadruped robot, and collect three-dimensional point cloud data through the lidar. Joint calibration is performed on the camera image data and the lidar point cloud data to obtain the corresponding relationship between the image information and the point cloud information;

[0058] (2) Image preprocessing, performing noise reduction, contrast adjustment, and color correction on the image data collected by the camera;

[0059] In order to improve the image quality, it is necessary to perform noise reduction on the collected images. The Gaussian filter is used to perform noise reduction on the collected images. It smooths the image through convolution operations to reduce noise. The mathematical formula of Gaussian filtering is as follows:

[0060]

[0061] where I raw (x, y) is the pixel value of the original image at the coordinate (x, y), σ is the standard deviation of the Gaussian kernel, which controls the smoothness of the filter (the larger the value, the stronger the smoothness effect), and * represents the convolution operation.

[0062] To enhance the image contrast, the histogram equalization method is used to enhance the image contrast; histogram equalization enhances the image contrast by redistributing the gray values of the image. Its formula is as follows:

[0063]

[0064] where I eq (x, y) is the gray value of the equalized image at the coordinate (x, y), L is the number of gray levels (for example, 256 levels for an 8-bit image), MN is the total number of image pixels (width × height), and h(i) is the number of pixels with the gray value i.

[0065] To ensure the unity of image colors under different lighting conditions, the white balance algorithm is used for color correction to unify the image tone; the white balance algorithm adjusts the proportions of the three colors red, green, and blue in the image to achieve the effect of color balance. Its mathematical formula is as follows:

[0066]

[0067] Among them, I wb (x, y) is the red component value at the coordinate (x, y) of the white balance-adjusted image, R avg is the global average value of the red component of the image, and R(x, y) is the red component value at the coordinate (x, y) of the original image.

[0068] (3) Preliminary stitching, extracting image feature points and performing matching, and then stitching the images into a complete panoramic image;

[0069] Use the SIFT (Scale-Invariant Feature Transform) algorithm to extract image feature points and perform matching. The specific steps include:

[0070] (31) Key point detection: Use the Difference of Gaussian function (DoG) to detect scale-invariant feature points of the image;

[0071] (32) Feature description: Generate descriptors according to the gradient directions around the feature points;

[0072] (33) Feature matching: Use the Euclidean distance for descriptor matching.

[0073] Through the calculation of the perspective transformation matrix, map the points on one image plane to another image plane. The mathematical formula is as follows:

[0074]

[0075] Among them, H is a 3x3 perspective transformation matrix, (x, y) is the point on the original image plane, (x′, y′) is the point on the transformed image plane, and w' is the scaling factor of homogeneous coordinates for normalizing the coordinates (i.e., x' = x” / w', y' = y” / w').

[0076] (4) LiDAR-assisted stitching, using the point cloud data obtained by the LiDAR to perform spatial alignment with the camera image, accurately registering the point cloud data, filling the blind area with the image data of the previous time period, and completing the blind area filling through a time series prediction model (such as a Kalman filter);

[0077] The spatial alignment between the LiDAR point cloud data and the camera image is achieved through the external parameter calibration matrix T C2L and convert the points in the LiDAR coordinate system to the camera coordinate system. The conversion formula is as follows:

[0078] P C = T C2L P L

[0079] Among them, P C is the point in the camera coordinate system, PL is a point in the lidar coordinate system, T C2L is the extrinsic parameter matrix, a 4×4 homogeneous transformation matrix that contains rotation and translation parameters, and has the following form:

[0080]

[0081] where, R 3x3 is the rotation matrix from the lidar to the camera (converted from Euler angles or quaternions), t 3x1 is the translation vector from the lidar to the camera. Example: If the lidar is 0.5 meters to the right of the origin of the camera coordinate system, then t = [0.5, 0, 0]^T.

[0082] Use the ICP (Iterative Closest Point) algorithm to perform precise registration on the point cloud data. The specific steps include:

[0083] (41) Initial alignment: Select the initial registration points;

[0084] (42) Nearest point matching: Find the nearest point pairs for each point;

[0085] (43) Transformation estimation: Calculate the rigid body transformation matrix with the minimum mean square error;

[0086] (44) Iterative optimization: Update the transformation matrix, and repeat (42) and (43) until convergence.

[0087] Introduce an advanced AVM (Around View Monitoring) completion algorithm, use the image data in the previous time period to fill the blind area, and perform blind area completion through a time series prediction model (such as a Kalman filter). The basic steps of the Kalman filter include:

[0088] Prediction: Predict the current state based on the state at the previous moment;

[0089] Update: Update the predicted state using the current measurement data.

[0090] (5) Image completion and optimization, perform edge smoothing processing and color consistency adjustment on the image, and update the image in real time;

[0091] To eliminate the edge discontinuity after image stitching, use a bilateral filter for edge smoothing processing; the bilateral filter achieves an edge-preserving smoothing effect by considering both the spatial distance and color similarity of pixels, and its mathematical formula is as follows:

[0092]

[0093] where, I smooth(x, y) is the pixel value of the smoothed image at coordinate (x, y), W is the normalization factor to ensure the sum of weights is 1, i, j are the neighborhood offsets relative to the central pixel (x, y) (for example, the neighborhood window is 5×5), I(x + i, y + j) is the original pixel value of the neighborhood pixel at offset (i, j), and f d (i, j) is the Gaussian function in the spatial domain, and f r (I(x, y) - I(x + i, y + j)) is the Gaussian function in the intensity domain.

[0094] The color transfer algorithm is used to adjust the color consistency of the stitched image to ensure the consistency of the visual effect; the color transfer algorithm makes the color distribution of the target image consistent with that of the source image by adjusting it, and its mathematical formula is as follows:

[0095] I color (x, y) = αI1(x, y) + (1 - α)I2(x, y)

[0096] Among them, I color (x, y) is the color value of the adjusted image at coordinate (x, y), I1(x, y) is the pixel value of the source image (reference hue), I2(x, y) is the pixel value of the target image (to be adjusted), and α is the weight coefficient to control the adjustment intensity (for example, α = 0.5 represents medium-intensity mixing).

[0097] Through the image caching and loop update mechanism, the real-time update of the surround-view image is realized; the specific steps include:

[0098] Image caching: Maintain an image cache queue to store the recent several frames of images;

[0099] Loop update: Each time a new image is acquired, update the cache queue and synthesize the new image with the images in the cache to generate the latest surround-view image.

[0100] In the present invention, the accurate spatial reference information provided by the lidar is used for accurate stitching and blind area complementation of the camera images. The lidar obtains the three-dimensional information of the environment in real time through the point cloud data, and aligns this information with the camera images in space to ensure the accuracy of the stitched images. Specifically: Through the calibration algorithm, determine the transformation matrix T C2L between the lidar and the camera to realize the conversion of the spatial coordinate system; use the ICP (Iterative Closest Point) algorithm to align the lidar point cloud with the camera images to ensure the accurate matching of the images and the point cloud; based on the blind area positioning information of the lidar, combined with the image data in the previous time period, use the Kalman filter for blind area complementation to ensure the continuity and integrity of the surround-view image.

[0101] In the present invention, through a series of image processing techniques, including noise reduction, contrast adjustment, and color correction, the quality of images under different lighting conditions is improved. Specifically: a Gaussian filter is used to smooth the image and reduce noise; the histogram equalization method is adopted to enhance the image contrast; the red, green, and blue color ratios of the image are adjusted through a white balance algorithm.

[0102] In the present invention, the surround view image is updated in real time to achieve real-time update of the surround view image, adapt to dynamic environmental changes, and ensure the accuracy and real-time nature of visual perception. Specifically: image caching, maintaining an image cache queue to store the most recent several frames of images for use in real-time updates; cyclic update, each time a new image is acquired, the cache queue is updated, and the new image is synthesized with the images in the cache to generate the latest surround view image; real-time guarantee, through optimized algorithms, the efficiency of image synthesis and update is improved to ensure real-time update of the surround view image.

[0103] In summary, the present invention significantly improves the environmental perception ability of quadruped robots through the following innovative points:

[0104] Low illuminance adaptability: A camera designed specifically for low illuminance environments is selected, and through image enhancement technology, the visual perception of the robot under insufficient lighting conditions is improved.

[0105] AVM completion algorithm: An advanced AVM (Around View Monitoring) completion algorithm is introduced, which uses the image data of the previous time to intelligently predict and fill the blind area, achieving a seamless panoramic view.

[0106] Real-time surround view image update: Through optimized algorithms and hardware configurations, rapid update of the surround view image is achieved, ensuring the robot's real-time response to dynamic environments.

[0107] Optimization of image stitching technology: Advanced image processing techniques are adopted to achieve smoother and more continuous image stitching, eliminating the seam and overlap problems in traditional stitching methods.

[0108] Improvement of computing efficiency: Aiming at the computing resource limitations of the robot platform, the algorithm is optimized to reduce the computing complexity, ensuring the real-time nature and efficiency of the algorithm.

[0109] Through these innovative points, the present invention not only solves the blind area problem in the prior art, but also significantly improves the environmental perception ability and autonomous navigation performance of quadruped robots under various lighting conditions, having important practical application value and broad market prospects.

Claims

1. A method for realizing the panoramic view function of a visible light camera based on low illuminance in all directions, characterized in that, A quadruped robot is equipped with several cameras and lidars all around it. The specific steps are as follows: (1) Collect image information of the surrounding environment through the cameras on the quadruped robot, and collect three-dimensional point cloud data through the lidar. Joint calibration is performed on the camera image data and the lidar point cloud data to obtain the corresponding relationship between the image information and the point cloud information; (2) Image preprocessing, perform noise reduction, contrast adjustment, and color correction on the image data collected by the camera; (3) Initial stitching, extract image feature points and perform matching, and then stitch the images into a complete panoramic image; (4) Lidar-assisted stitching, use the point cloud data obtained by the lidar to align with the camera image in space, perform precise registration on the point cloud data, use the image data in the previous time period to fill the blind area, and complete the blind area through a time series prediction model; (5) Image completion and optimization, perform edge smoothing processing and color consistency adjustment on the image, and update the image in real time.

2. The method for realizing the panoramic view function of a visible light camera based on low illuminance in all directions according to claim 1, characterized in that, In step (2), when performing noise reduction on the collected image, a Gaussian filter is used to perform noise reduction on the collected image. It performs smoothing processing on the image through convolution operations to reduce noise. The mathematical formula of Gaussian filtering is as follows: where I raw (x, y) is the pixel value of the original image at the coordinate (x, y), σ is the standard deviation of the Gaussian kernel, which controls the smoothness of the filter, and * represents the convolution operation.

3. The method for realizing the panoramic view function of a visible light camera based on low illuminance around, as claimed in claim 1, is characterized in that In step (2), the histogram equalization method is used to enhance the image contrast; histogram equalization enhances the image contrast by redistributing the gray values of the image. Its formula is as follows: where I eq (x,y) is the gray value of the equalized image at the coordinate (x,y), and L is the number of gray levels, MN is the total number of image pixels, and h(i) is the number of pixels with a gray value of i.

4. A method for realizing the panoramic view function of a visible light camera based on low illuminance in all directions, characterized in that, In step (2), the white balance algorithm is used to perform color correction to unify the image tone; the white balance algorithm adjusts the proportions of the three colors red, green, and blue in the image to achieve the effect of color balance. Its mathematical formula is as follows: Among them, I wb (x, y) is the red component value of the white balance-adjusted image at the coordinate (x, y), R avg is the global average value of the red component of the image, and R(x, y) is the red component value of the original image at the coordinate (x, y).

5. A method for implementing the panoramic view function of a visible light camera based on low illuminance in all directions, characterized in that, In step (3), the SIFT algorithm is used to extract image feature points and perform matching. The specific steps include: (31) Key point detection: Use the Difference of Gaussians function to detect scale-invariant feature points of the image; (32) Feature description: Generate descriptors according to the gradient directions around the feature points; (33) Feature matching: Use the Euclidean distance to perform descriptor matching.

6. The method for realizing the panoramic view function of a visible light camera based on low illuminance in all directions according to claim 1, characterized in that In step (3), through the calculation of the perspective transformation matrix, points on one image plane are mapped to another image plane. Its mathematical formula is as follows: Where, H is a 3x3 perspective transformation matrix, (x, y) are points on the original image plane, (x′, y′) are points on the transformed image plane, and w' is the scaling factor of the homogeneous coordinates for normalizing the coordinates.

7. A method for realizing the panoramic view function of a visible light camera based on low illuminance in all directions according to claim 1, characterized in that, In step (4), the spatial alignment of the lidar point cloud data and the camera image is achieved through the extrinsic calibration matrix T C2L to transform the points in the lidar coordinate system into the camera coordinate system. The transformation formula is as follows: P C = T C2L P L where, P C is a point in the camera coordinate system, and P L is a point in the lidar coordinate system. T C2L is the extrinsic parameter matrix, which is a 4×4 homogeneous transformation matrix containing rotation and translation parameters and has the following form: Among them, R 3x3 is the rotation matrix from the lidar to the camera, and t 3x1 is the translation vector from the lidar to the camera.

8. A method for realizing the panoramic view function of a visible light camera based on low illuminance around, characterized in that, In step (4), the ICP algorithm is used to perform precise registration on the point cloud data. The specific steps include: (41) Initial alignment: Select initial registration points; (42) Nearest point matching: Find the nearest point pairs for each point; (43) Transformation estimation: Calculate the rigid body transformation matrix with the minimum mean square error; (44) Iterative optimization: Update the transformation matrix, and repeat (42) and (43) until convergence.

9. A method for realizing the panoramic view function of a visible light camera based on low illuminance in all directions, characterized in that, In step (5), a bilateral filter is used for edge smoothing processing; the bilateral filter achieves an edge-preserving smoothing effect by considering both the spatial distance and color similarity of pixels. Its mathematical formula is as follows: where I smooth (x, y) is the pixel value of the smoothed image at the coordinate (x, y), W is the normalization factor to ensure that the sum of weights is 1, i and j are the neighborhood offsets relative to the central pixel (x, y), I(x + i, y + j) is the original pixel value of the neighborhood pixel at the offset (i, j), f d (i, j) is the spatial domain Gaussian function, f r (I(x, y) - I(x + i, y + j)) is the intensity domain Gaussian function.

10. A method for implementing the panoramic view function of a visible light camera based on low illuminance in all directions, characterized in that, In step (5), the color transfer algorithm is used to adjust the color consistency of the spliced image to ensure the consistency of the visual effect; the color transfer algorithm makes the color distribution of the target image consistent with that of the source image by adjusting it, and its mathematical formula is as follows: I color (x, y) = αI1(x, y) + (1 - α)I2(x, y) Among them, I color (x, y) is the color value of the adjusted image at the coordinate (x, y), I1(x, y) is the pixel value of the source image, I2(x, y) is the pixel value of the target image, and α is the weight coefficient that controls the adjustment intensity.

Images