Robot pose estimation method based on laser point cloud and visual SLAM

Abstract

The invention discloses a robot pose estimation method based on laser point cloud and visual SLAM, and relates to the technical field of robot sensing and positioning, and the method comprises the steps: in a data collection and integration link, installing a specific camera and a radar to collect and compress data; a data preprocessing and feature strengthening stage: processing the image and the point cloud to improve the quality and extract the features; a cross-modal network is constructed through feature fusion and accurate matching, and a special matching mode is adopted; space-time calibration and coordinate unification are realized by means of a high-precision circuit; multi-modal fusion and reinforcement learning are applied to loop detection and intelligent optimization; correcting filtering and visualizing uncertainty by combining pose smoothing and intelligent output with a prediction model; according to the method, the pose estimation precision, reliability and environmental adaptability of the robot are effectively improved, resource utilization is optimized, the pose uncertainty is visually evaluated, and the overall performance and the intelligent level of the robot are improved.

Description

Technical Field

[0001] The present invention relates to the field of robot perception and positioning technology, and in particular to a robot posture estimation method based on laser point cloud and visual SLAM. Background Art

[0002] With the widespread application of robotics in many fields, such as indoor service robots and autonomous vehicles, robot pose estimation has become one of the key technologies. Traditional pose estimation methods are mainly based on laser SLAM or visual SLAM alone. Laser SLAM uses laser radar to obtain high-precision three-dimensional point cloud data, which can accurately perceive the geometric structure of the environment and has significant advantages in obstacle detection and positioning. However, laser radar is expensive, and its point cloud data lacks texture information, making it difficult to identify the category and attributes of objects, limiting the robot's deeper understanding and interaction capabilities with the environment.

[0003] On the other hand, visual SLAM uses a camera to collect image sequences and achieves pose estimation through feature extraction and matching. It performs well in texture-rich environments and can provide rich semantic information. However, visual SLAM is greatly affected by factors such as lighting changes and textureless areas. The accuracy of pose estimation is prone to fluctuations in complex environments, and monocular vision has scale ambiguity problems. Although binocular or RGB-D vision can partially solve this problem, the system complexity increases and it still faces challenges in some special scenarios. Therefore, the use of laser SLAM or visual SLAM alone is difficult to meet the needs of high-precision and high-reliability pose estimation of robots in complex and changeable environments, which has prompted the research and development of pose estimation technology based on the fusion of laser point cloud and visual SLAM.

[0004] In view of this, this application is hereby filed. Summary of the invention

[0005] The purpose of the present invention is to provide a robot pose estimation method based on laser point cloud and visual SLAM to solve the problems of poor accuracy and poor adaptability of robot pose estimation mentioned in the above background technology.

[0006] To solve the above technical problems, the present invention provides a robot pose estimation method based on laser point cloud and visual SLAM, which includes the following steps: data acquisition and integration: RGB-D cameras and laser radars that are calibrated with each other and have overlapping fields of view are installed on the robot, color images, depth images and three-dimensional point cloud data are collected at a predetermined frame rate, the sensor layout direction is planned according to the robot's operating task to obtain more critical data areas, and a hybrid compression algorithm is used to compress the collected data online to reduce the burden of data transmission and storage; data preprocessing and feature enhancement: intelligent illumination compensation and noise filtering are performed on color images to improve image quality, adaptive voxelization is performed based on image semantics and texture features to balance data volume and information retention, outlier noise is first removed from laser point cloud data, and then deep learning models are used to extract deep geometric features of point clouds and perform semantic voxelization; feature fusion and precise matching: a cross-modal feature fusion network is constructed to deeply fuse visual and laser features. In the matching stage, visual features are extracted using feature points guided by semantic segmentation and are processed through a feature point neighborhood graph The structure is matched and verified with the attention mechanism. The laser point cloud extracts feature points based on the local curvature change and matches them with the attention mechanism in combination with the color information weighting; spatiotemporal calibration and coordinate unification: with the help of high-precision hardware synchronization circuits, the time synchronization of visual and laser data is achieved, and the error is controlled in a very small range. The data of both are mapped to a unified world coordinate system through precise calibration; loop detection and intelligent optimization: using the loop detection method of multimodal fusion, the visual loop is first quickly screened through the bag-of-words model and then accurately verified by deep learning scene recognition and a multi-scale strategy is adopted. The laser loop combines the multi-scale method of global feature descriptors and local geometric matching, and uses the reinforcement learning algorithm to optimize the loop decision process according to the robot state and environmental information. The back-end adopts an optimization algorithm based on factor graphs and dynamically adjusts weights; pose smoothing and intelligent output: the Kalman filter is used to smooth the pose information, and the Kalman filter is corrected and compensated in combination with the pose prediction model based on robot kinematics and environmental perception, the uncertainty of the pose estimation is quantified, and displayed on the navigation map in a visual form such as an error ellipse.

[0007] Furthermore, in the data collection and integration steps, a hybrid compression algorithm is used to compress the laser point cloud based on the geometric structure and density, and the compression ratio is dynamically adjusted according to the local curvature and point spacing of the point cloud. The visual image is compressed using a generative adversarial network, and efficient compression and reconstruction are achieved by training specific generator and discriminator networks.

[0008] Furthermore, in the feature fusion and precise matching steps, the cross-modal feature fusion network shares some feature extraction layers, and fuses visual and laser features through specific connection layers. The attention mechanism of visual features assigns weights based on the stability of image feature points, and calculates weight values ​​by comprehensively considering factors such as the contrast and scale invariance of feature points. The attention mechanism of laser point cloud matching adjusts weights based on the local structural complexity of the point cloud, focusing on areas with complex structures.

[0009] Furthermore, in the time-space calibration and coordinate unification steps, the synchronization accuracy of the high-precision hardware synchronization circuit can reach the nanosecond level, and a high-precision clock source and synchronization signal transmission line are used. The data time consistency is ensured through precise timestamp marking and calibration algorithms. At the same time, the calibration process uses a high-precision calibration board and an automated calibration algorithm to improve the calibration accuracy.

[0010] Furthermore, in the loop detection and intelligent optimization steps, the bag-of-words model of visual loop detection adopts a construction method based on semantic features, and a visual vocabulary is constructed according to the semantic category of the image. The reward function of the reinforcement learning algorithm combines the loop accuracy and system resource consumption, giving positive rewards for correct loops and negative rewards for decisions with high resource consumption to guide the algorithm to optimize decisions.

[0011] Furthermore, in the posture smoothing and intelligent output steps, the posture prediction model takes into account the dynamic constraints of the robot and combines the robot's mass, inertia, motor performance and other parameters for modeling. The color depth of the visualized error ellipse indicates the degree of uncertainty, and the darker the color, the higher the uncertainty. At the same time, the visualization display mode and parameters can be adjusted according to user needs.

[0012] Furthermore, in the data collection and integration step, if data transmission is interrupted, an automatic reconnection and data cache continuation mechanism is provided, a heartbeat detection algorithm is used to monitor the data connection status, and a ring buffer structure is used for data cache, and key data is transmitted first when the connection is restored.

[0013] Furthermore, in the feature fusion and precise matching steps, for areas where matching fails, the feature extraction and matching strategies are automatically adjusted for a second attempt. The adjustment strategies include changing the feature extraction threshold, expanding the matching search range, replacing the feature matching algorithm, etc., and the adjustment strategy is intelligently selected based on the cause of the first matching failure.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention combines the advantages of laser and vision with a unique multimodal data collection and processing method combined with an advanced feature fusion matching mechanism, such as collecting key data according to the task planning layout, processing data with a hybrid compression algorithm, etc. The cross-modal feature fusion network and the attention mechanism match to overcome the limitations of a single sensor. In complex environment testing, compared with the traditional single SLAM method, the average error of pose estimation is greatly reduced, laying a solid foundation for the precise navigation and operation of the robot; improving the accuracy and reliability of pose estimation; 2. Deep learning is used for image illumination compensation, point cloud feature extraction and semantic voxelization during data preprocessing and feature enhancement. Multimodal fusion loop detection is used in loop detection and intelligent optimization, and reinforcement learning is used to optimize decision making. This enables the robot to estimate pose stably and accurately in different lighting, scene structures and environments with dynamic object interference, reducing the impact of environmental changes and enhancing environmental adaptability and robustness. 3. Time synchronization and coordinate calibration are achieved through high-precision hardware synchronization circuits to reduce fusion errors and computational overhead. The backend dynamically adjusts the weight algorithm based on the factor graph and allocates resources based on data reliability to avoid waste. The mechanism strategy in data collection integration and feature fusion matching improves the efficiency and stability of system operation, significantly improves resource utilization and processing speed, and optimizes system resource utilization and efficiency; 4. In the stage of posture smoothing and intelligent output, the Kalman filter is corrected and the uncertainty is quantified by combining kinematics and perception models. The error ellipse is displayed on the navigation map to facilitate operators to monitor the posture status. It provides reference for other posture information-dependent systems to help them adjust strategies based on uncertainty, improve system intelligence and scientific decision-making; and provide intuitive posture uncertainty assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a principle block diagram of the robot pose estimation method based on laser point cloud and visual SLAM. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] See also Figure 1 The present invention provides a technical solution: a robot pose estimation method based on laser point cloud and visual SLAM, comprising: 1. Robot platform and sensor configuration: We selected an indoor service robot with a main structure of a circular chassis with a diameter of about 50 cm and a height of 120 cm. An RGB-D camera and a laser radar with mutual calibration and overlapping fields of view were installed on the top of the robot. The RGB-D camera uses a domestic low-cost high-resolution model with a horizontal viewing angle of 70 degrees, a vertical viewing angle of 50 degrees, a resolution of 1280×720, and can capture color images and depth images at a rate of 30 frames per second. The laser radar uses a domestic laser radar with 16-line scanning, with a vertical viewing angle range of -15 degrees to +15 degrees and a horizontal scanning frequency of 10 Hz, which can obtain three-dimensional point cloud data of the surrounding environment. The sensor layout has been carefully designed. Considering that the robot mainly moves in a planar manner in an indoor environment and needs to fully perceive surrounding obstacles and objects such as furniture, the lidar is installed slightly forward of the center of the top of the robot so that its horizontal scanning plane is parallel to the bottom of the robot, which can better scan the environmental information in front and on both sides; the RGB-D camera is installed about 10 cm behind the lidar, slightly higher than the lidar. Its tilt angle enables the camera's field of view to cover a certain area in front of and above the robot. This layout is conducive to obtaining rich visual information, such as having an advantage in identifying desktop objects or ceiling signs.

[0018] Robot pose estimation method based on laser point cloud and visual SLAM, including: 2. Data collection and integration: After the robot starts running in the indoor environment, the RGB-D camera and the laser radar collect data synchronously according to the predetermined frame rate and scanning frequency. During the data collection process, a hybrid compression algorithm is used to compress the data online. For laser point cloud data, since the geometric structure in the indoor environment is relatively complex but the overall spatial range is limited, compression is performed based on the geometric structure and density. For example, in the open ground area, the point cloud density is low, and the compression ratio can be set to a higher value, such as 8:1; while in areas with large curvature changes and rich geometric features such as furniture edges and corners, the compression ratio is reduced to 4:1 to retain more detailed information. Visual images are compressed using generative adversarial networks. The trained generator and discriminator networks can compress the original image data volume by about 60%, while the image can still be reconstructed well during back-end processing to ensure that key visual information is not lost. The collected data is stored in the robot's local solid-state hard drive. The storage format uses an optimized binary format, which can improve the data reading and writing efficiency and reduce the time overhead of data storage and reading. During the data transmission process, if a short network interference or transmission line failure causes data transmission interruption, the system will automatically start the reconnection mechanism and try to reconnect once every 0.5 seconds. At the same time, the data cache adopts a circular buffer structure, and the cache size is set to be able to store the acquisition data of the last 10 seconds. When the connection is restored, key data in the cache is transmitted first, such as images and point cloud data within the last 2 seconds, to ensure the continuity and integrity of the data.

[0019] Robot pose estimation method based on laser point cloud and visual SLAM, including: 3. Data preprocessing and feature enhancement: Color image processing: For color images, intelligent lighting compensation is first performed. The lighting distribution is analyzed based on a deep learning model that is pre-trained on a dataset of indoor scene images containing various lighting conditions (strong direct light, weak light shadows, uniform lighting, etc.). For example, in a living room scene with strong light from a window on one side, the model can accurately identify areas with strong lighting and adaptively adjust the image pixel values ​​to make the brightness and contrast of the entire image more uniform, improve image quality, and reduce feature extraction errors caused by uneven lighting.

[0020] Next, noise filtering is performed, and median filtering is used to remove common noise interference such as salt and pepper noise. Then, adaptive voxelization is performed based on the semantics and texture features of the image. For areas that semantically belong to large flat walls, a larger voxel size, such as 10×10 pixels, is used because its texture is simple and the information is relatively single; while for areas in the image that are semantically complex objects (such as decorative ornaments, electrical equipment, etc.), the voxel size is reduced to 2×2 pixels to retain more detailed features and balance the amount of data and information retention.

[0021] Laser point cloud processing: The laser point cloud data is first filtered by radius to remove outlier noise, and the radius threshold is set to 0.2 meters, that is, points with an average distance of more than 0.2 meters from the surrounding point clouds are judged as outliers and removed. Then, a three-dimensional convolutional neural network is used to extract the deep geometric features of the point cloud. The network structure design is optimized for the spatial distribution characteristics of the point cloud, and contains multiple convolution layers and pooling layers. For example, a 3×3×3 convolution kernel is used for convolution operations, which can effectively extract complex geometric features such as corners and furniture edges. Finally, semantic voxelization is performed to divide the point cloud in the indoor environment into categories such as ground, wall, and furniture according to semantics. Different voxelization strategies are used for different categories of point clouds. For example, the voxel size can be appropriately increased when voxelizing the ground point cloud, while smaller voxels are used for furniture point clouds to retain shape details.

[0022] Robot pose estimation method based on laser point cloud and visual SLAM, including: 4. Feature fusion and precise matching: Cross-modal feature fusion network: A cross-modal feature fusion network is constructed, in which some feature extraction layers are shared. For example, the first two convolutional layers of visual feature extraction share parameters with some low-level feature extraction layers of laser point cloud feature extraction, and the visual and laser features are fused through a specific connection layer. This connection layer organically combines the color and texture information in the visual features with the spatial geometric features of the laser point cloud to form a new joint feature representation.

[0023] Visual feature matching: Visual features use semantic segmentation-guided feature point extraction. For example, in indoor scenes, feature points located in the contours of semantically key objects such as furniture contours, door and window edges, and texture-rich areas are preferentially extracted. Matching verification is performed through the feature point neighborhood graph structure and the attention mechanism. The attention mechanism of visual features assigns weights based on the stability of image feature points. Feature points with high contrast, good scale invariance, and stable appearance in multiple frames are given higher weights. For example, when identifying a black switch on a white wall indoors, the feature points at the edge of the switch have a higher weight. In an indoor environment test that includes multiple rooms such as a living room and a bedroom, after feature point extraction and matching verification, the accuracy of valid matching point pairs was improved by about 30% compared to traditional methods.

[0024] Laser point cloud matching: The laser point cloud extracts feature points based on the local curvature change, and extracts feature points in areas where the curvature change exceeds 0.5. The color information is weighted and matched with the attention mechanism. The attention mechanism of the laser point cloud matching adjusts the weights based on the local structural complexity of the point cloud, and focuses on areas with complex structures (such as areas with dense furniture placement). For example, in a room with cluttered furniture, a higher weight is given to the point cloud matching around objects such as sofas and coffee tables to improve the matching accuracy of these areas.

[0025] Robot pose estimation method based on laser point cloud and visual SLAM, including: 5. Space-time calibration and coordinate unification: High-precision hardware synchronization circuits are used to achieve time synchronization of visual and laser data, with synchronization accuracy up to nanoseconds. The circuit uses an atomic clock as a high-precision clock source, and the synchronization signal transmission line uses a shielded twisted pair cable to effectively reduce signal transmission delays and interference. Data time consistency is ensured through precise timestamp marking and calibration algorithms, and the timestamp marking is accurate to nanoseconds. During the calibration process, a high-precision calibration plate is used, which has specific geometric patterns and marking points. An automated calibration algorithm is used, and the robot scans and photographs the calibration plate at different positions and postures. The system automatically calculates the relative position and posture relationship between the RGB-D camera and the lidar, and the calibration accuracy error is controlled within 2 mm, thereby accurately mapping the data of both to a unified world coordinate system.

[0026] Robot pose estimation method based on laser point cloud and visual SLAM, including: 6. Loop detection and intelligent optimization: Visual loop detection: Visual loops are first quickly screened through a bag-of-words model, which builds a visual vocabulary based on semantic features. For example, the visual features of common indoor objects (such as tables, chairs, electrical appliances, doors and windows, etc.) are used as vocabulary elements. In an office environment test containing multiple similar rooms, the bag-of-words model can quickly screen out possible loop frames, with an average processing time of about 0.1 seconds per frame. Then, deep learning scene recognition is used for accurate verification. The deep learning model uses a convolutional neural network architecture and is trained on a large number of indoor scene image data sets. It can accurately identify scene features in different rooms. After deep learning verification, the accuracy of loop detection reaches more than 95%. In addition, a multi-scale strategy is adopted to first screen possible loop areas in low-resolution images (such as 320×180), and then finely verify them in high-resolution images, which improves detection efficiency.

[0027] Laser loopback detection: Laser loop closure combines global feature descriptors with a multi-scale approach of local geometric matching. First, potential loops are screened based on the global feature descriptors of point cloud distribution (such as the overall shape of the point cloud and the spatial distribution feature vector). In a large indoor shopping mall environment test, the loop search range can be quickly narrowed, reducing the amount of calculation by about 70%. Then, detailed geometric structure matching is confirmed in the local area. For example, in the suspected loop area, local geometric structures such as furniture and columns are accurately matched. At the same time, the reinforcement learning algorithm is used to optimize the loop decision process. The reward function of the reinforcement learning algorithm combines loop accuracy with system resource consumption, giving a reward of +10 for correct loops and a reward of -5 for decisions with high resource consumption (such as long calculation time and large data transmission volume). After training, the loop decision reduces system resource consumption by about 40% while ensuring accuracy. The backend uses an optimization algorithm based on factor graphs and dynamically adjusts weights. It dynamically allocates constraint weights according to the data reliability of vision and laser sensors in different environments (such as empty halls and narrow corridors) and motion states (such as uniform linear motion and turning). For example, in a corridor environment with dim light and unclear visual features, the constraint weight of the visual data is reduced and the weight of the laser data is increased, making the pose estimation results more accurate and reliable.

[0028] Robot pose estimation method based on laser point cloud and visual SLAM, including: 7. Posture smoothing and intelligent output: The Kalman filter is used to smooth the pose information, and the Kalman filter is corrected and compensated by combining the pose prediction model based on robot kinematics and environmental perception. The pose prediction model takes into account the dynamic constraints of the robot and is modeled based on the robot's mass of 20 kg, inertia of 5 kg / m2, and maximum torque of the motor of 10 Nm. When the robot accelerates, decelerates, or turns, the pose prediction model can predict the pose change trend in advance based on the robot's motion state and environmental information (such as the distribution of surrounding obstacles), and correct the prediction results of the Kalman filter, reducing the pose estimation error by about 45%. The uncertainty of the pose estimation is quantified and displayed on the navigation map in a visual form such as an error ellipse. The lengths of the major and minor axes of the error ellipse are calculated based on the covariance matrix of the pose estimation. The color depth indicates the degree of uncertainty, and the darker the color, the higher the uncertainty. For example, when the robot approaches a complex furniture placement area or an area with large light changes, the color of the error ellipse becomes darker, which intuitively reflects the reduced reliability of the pose estimation. The operator or other systems that rely on the robot's pose information can adjust the strategy or optimize the robot's navigation path in a timely manner based on the information of the error ellipse.

[0029] In summary, in the data collection stage, the present invention carefully plans the sensor layout direction according to the task, and uses a unique hybrid compression algorithm to perform efficient compression based on the geometric structure and density of the laser point cloud and the characteristics of the visual image. In the data preprocessing stage, a deep learning model is used to perform deep processing and semantic voxelization on the color image illumination and the spatial characteristics of the laser point cloud, breaking through the limitations of traditional fixed parameter processing. In terms of feature fusion and matching, an innovative cross-modal feature fusion network is constructed and an attention mechanism with multi-factor considerations is introduced to achieve deep fusion and precise matching of visual and laser features, which is different from the traditional simple splicing or independent matching method. Spatiotemporal calibration relies on nanosecond-level high-precision hardware synchronization circuits and precise calibration, far exceeding the accuracy level of existing technologies. Loop detection and optimization adopt a strategy of multimodal fusion and decision optimization based on reinforcement learning, and the dynamic weight adjustment algorithm of the back-end factor graph further enhances the intelligence of the system. The output and correction of the posture are combined with the dynamic model and visual uncertainty display to comprehensively improve the performance and practicality of the system. These unique technical means work together to fundamentally solve many problems in existing technologies such as limited pose estimation accuracy, insufficient environmental adaptability, and low resource utilization efficiency, greatly promoting the development of robot pose estimation technology.

[0030] The present invention has constructed a set of efficient, accurate, intelligent and adaptable posture estimation system in the field of robot posture estimation through a series of innovative technical means. From the collection and integration of sensor data, to the deep extraction and fusion of features, to spatiotemporal calibration, loop detection optimization, and smooth output and correction of posture, each link has incorporated innovative design concepts and technical implementation methods that are different from the existing technology. These innovations have not only improved the key performance indicators of robot posture estimation, but also made substantial breakthroughs and progress in multi-sensor fusion technology, the application of deep learning in the field of robot perception, and intelligent optimization algorithms. It has laid a solid technical foundation for the application of autonomous navigation, precise operation, and intelligent interaction of robots in complex environments, and is worthy of wide promotion and application in related technical fields.

Claims

1. A robot pose estimation method based on laser point cloud and visual SLAM, characterized by: The following steps are involved: Data collection and integration: Install mutually calibrated RGB-D cameras and lidars with overlapping fields of view on the robot to collect color images, depth images, and 3D point cloud data at a predetermined frame rate. Plan the sensor layout direction according to the robot's task to obtain more critical data areas, and use a hybrid compression algorithm to compress the collected data online to reduce the burden of data transmission and storage. Data preprocessing and feature enhancement: Intelligent illumination compensation and noise filtering are performed on color images to improve image quality. Adaptive voxelization is performed based on image semantics and texture features to balance data volume and information retention. Outlier point noise is first removed from laser point cloud data, and deep learning models are used to extract deep geometric features of point clouds and perform semantic voxelization. Feature fusion and precise matching: A cross-modal feature fusion network is constructed to deeply fuse visual and laser features. In the matching stage, the visual features are extracted using semantic segmentation-guided feature points and matched and verified through the feature point neighborhood graph structure and attention mechanism. The laser point cloud extracts feature points based on local curvature changes and matches them in combination with color information weighting and attention mechanism. Space-time calibration and coordinate unification: With the help of high-precision hardware synchronization circuit, the time synchronization of visual and laser data is achieved, and the error is controlled within a very small range. Through precise calibration, the data of both are mapped to a unified world coordinate system; Loop detection and intelligent optimization: Using the loop detection method of multimodal fusion, the visual loop is first quickly screened through the bag-of-words model and then accurately verified by deep learning scene recognition and a multi-scale strategy is adopted. The laser loop combines the multi-scale method of global feature description and local geometric matching, and uses the reinforcement learning algorithm to optimize the loop decision process according to the robot state and environmental information. The back-end adopts an optimization algorithm based on factor graphs and dynamically adjusts weights; Posture smoothing and intelligent output: Use Kalman filtering to smooth the posture information, and use the posture prediction model based on robot kinematics and environmental perception to correct and compensate the Kalman filter, quantify the uncertainty of posture estimation, and display it in a visual form such as error ellipse on the navigation map.

2. The robot pose estimation method based on laser point cloud and visual SLAM as claimed in claim 1, characterized in that: In the data collection and integration steps, a hybrid compression algorithm is used to compress the laser point cloud based on the geometric structure and density, and the compression ratio is dynamically adjusted according to the local curvature and point spacing of the point cloud. The visual image is compressed using a generative adversarial network, and efficient compression and reconstruction are achieved by training specific generator and discriminator networks.

3. The robot pose estimation method based on laser point cloud and visual SLAM as claimed in claim 1, characterized in that: In the feature fusion and precise matching steps, the cross-modal feature fusion network shares some feature extraction layers, and fuses visual and laser features through specific connection layers. The attention mechanism of visual features allocates weights based on the stability of image feature points, and calculates weight values ​​by comprehensively considering factors such as the contrast and scale invariance of feature points. The attention mechanism of laser point cloud matching adjusts weights based on the local structural complexity of the point cloud, and focuses on areas with complex structures.

4. The robot pose estimation method based on laser point cloud and visual SLAM as claimed in claim 1, characterized in that: In the time-space calibration and coordinate unification steps, the synchronization accuracy of the high-precision hardware synchronization circuit can reach the nanosecond level. A high-precision clock source and synchronization signal transmission line are used. Accurate timestamp marking and calibration algorithms are used to ensure data time consistency. At the same time, the calibration process uses a high-precision calibration board and an automated calibration algorithm to improve the calibration accuracy.

5. The robot pose estimation method based on laser point cloud and visual SLAM as claimed in claim 1, characterized in that: In the loop detection and intelligent optimization steps, the bag-of-words model of visual loop detection adopts a construction method based on semantic features, and a visual vocabulary is constructed according to the semantic category of the image. The reward function of the reinforcement learning algorithm combines the loop accuracy and system resource consumption, and gives positive rewards to correct loops and negative rewards to decisions with high resource consumption to guide the algorithm to optimize decisions.

6. The robot pose estimation method based on laser point cloud and visual SLAM as claimed in claim 1, characterized in that: In the posture smoothing and intelligent output steps, the posture prediction model takes into account the dynamic constraints of the robot and combines the robot's mass, inertia, motor performance and other parameters for modeling. The color depth of the visualized error ellipse indicates the degree of uncertainty, and the darker the color, the higher the uncertainty. At the same time, the visualization display mode and parameters can be adjusted according to user needs.

7. The robot pose estimation method based on laser point cloud and visual SLAM as claimed in claim 1, characterized in that: In the data collection and integration step, if data transmission is interrupted, an automatic reconnection and data cache continuation mechanism is provided, a heartbeat detection algorithm is used to monitor the data connection status, and a ring buffer structure is used for data cache, and key data is transmitted first when the connection is restored.

8. The robot pose estimation method based on laser point cloud and visual SLAM as claimed in claim 1, characterized in that: In the feature fusion and precise matching steps, for areas where matching fails, the feature extraction and matching strategies are automatically adjusted for a second attempt. The adjustment strategies include changing the feature extraction threshold, expanding the matching search range, replacing the feature matching algorithm, etc. The adjustment strategy is intelligently selected based on the cause of the first matching failure.

Images