Mobile robot vision SLAM method, system and device and storage medium

Abstract

The invention provides a mobile robot vision SLAM method and system, computer equipment and a storage medium, and belongs to the field of simultaneous localization and map construction.The method comprises the steps that an original environment image sequence of the position where a mobile robot is located is collected; dividing an original environment image sequence into a plurality of non-overlapped pixel blocks, and constructing a three-dimensional block group by searching similar blocks; sequentially performing three-dimensional transformation, hard threshold filtering and inverse transformation on the three-dimensional block group to generate an estimated pixel value of the pixel block after the first-stage denoising; forming a new block group through the estimated pixel value of the pixel block after denoising in the first stage and the original environment image sequence, and performing Wiener filtering and weighted average on the new block group in sequence to generate a denoised image; eliminating dynamic feature points in the de-noised image to obtain a final static feature point set; and optimizing the three-dimensional map points and the camera pose of the robot according to the final static feature point set, and generating a local three-dimensional point cloud map. According to the method, the visual SLAM performance in a complex environment can be comprehensively improved, and the positioning precision and the map stability are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of simultaneous localization and mapping (SLAM), specifically relating to a visual SLAM method, system, computer equipment, and storage medium for mobile robots. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is a core technology in mobile robotics, autonomous driving, and augmented reality, enabling robot localization and environmental mapping by processing image data acquired by visual sensors. In recent years, significant progress has been made in the robustness of SLAM technology in complex environments and in multi-robot collaboration. Mainstream algorithms such as ORB-SLAM achieve efficient localization and mapping through ORB feature extraction and matching, and are widely used in single-robot navigation and some multi-robot scenarios. It describes a monocular SLAM framework based on ORB features, emphasizing the robustness of feature extraction and backend optimization. Furthermore, ORB-SLAM3, proposed by C. Campos et al., further expands the applicability of visual, visual-inertial, and multi-map SLAM, becoming an important reference for current single-robot visual SLAM.

[0003] Visual SLAM utilizes cameras to capture environmental images, constructs environmental maps, and determines robot positions through feature extraction, matching, and pose optimization. It boasts advantages such as low cost and high adaptability, making it suitable for various indoor and outdoor scenarios. However, in visual SLAM, the captured environmental images often contain varying degrees of noise in complex environments. Image noise significantly impacts the accuracy of feature extraction in low-light or weak-texture environments, consequently affecting robot localization and environmental map construction. Existing techniques often employ Gaussian filtering or median filtering for preprocessing, but these methods have limited effectiveness in suppressing complex noise. For example, ORB-SLAM directly extracts ORB features from the original environmental image, and noise interference easily leads to feature point loss or matching errors. The BM3D algorithm proposed by A. Danielyan et al. achieves efficient denoising through block matching and 3D transformation, but this method cannot meet the requirements for suppressing complex noise, and its application in visual SLAM still needs optimization to meet real-time requirements.

[0004] Therefore, traditional filtering methods are insufficient in suppressing complex noise in low-light and weak-texture environments, leading to feature point loss or matching errors, which affects the stability of localization and mapping. Summary of the Invention

[0005] To address the problem that noise in images in complex environments affects the localization and mapping accuracy of SLAM systems in existing technologies, this invention provides a visual SLAM method for mobile robots.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A visual SLAM method for mobile robots, comprising: Collect raw environmental image sequences of the mobile robot's location; The original environmental image sequence is divided into multiple non-overlapping pixel blocks, denoted as reference blocks; a search window is set around each reference block, and a three-dimensional block group is formed by the reference blocks and other image blocks within the search window with similar pixel distribution to the reference blocks; The three-dimensional block group is mapped to the frequency domain through a three-dimensional transformation, and hard thresholding is applied to the three-dimensional block group in the frequency domain. The frequency domain three-dimensional block group after hard thresholding is then transformed inversely to convert the frequency domain data back into a three-dimensional block group in the spatial domain, generating the estimated pixel values ​​of the pixel blocks after the first stage of denoising. A new block group is formed by combining the estimated pixel values ​​of the pixel blocks after the first stage of denoising with the original environmental image sequence. Wiener filtering is applied to the new block group, and the block group results after Wiener filtering are weighted and averaged to generate a denoised image. Dynamic feature points in the denoised image are removed to obtain the final static feature point set; the 3D map points and the robot's camera pose are optimized based on the final static feature point set to generate a local 3D point cloud map.

[0007] Preferably, the step of setting a search window around each reference block and forming a three-dimensional block group by using the reference block and other image blocks within the search window with similar pixel distribution to the reference block includes the following steps: Set a search window around each reference block, and search for each reference block. Search for similar blocks in the search window. Calculate the reference block based on weighted Euclidean distance Similar blocks Similarity: ; In the formula, Indicates the side length of the pixel block; Similarity with a fixed threshold To make a comparison, if and only if Then, the selected similar blocks are added to the three-dimensional block group to obtain the final three-dimensional block group.

[0008] Preferably, the process involves mapping the three-dimensional block group to the frequency domain through a three-dimensional transformation, performing hard-threshold filtering on the three-dimensional block group in the frequency domain, and then performing an inverse transformation on the frequency domain three-dimensional block group after hard-threshold filtering to convert the frequency domain data back into a spatial domain three-dimensional block group. The process of generating estimated pixel values ​​for the first-stage denoised pixel blocks includes the following steps: Through three-dimensional transformation The three-dimensional block group is mapped to the frequency domain; hard thresholding is then applied to the three-dimensional block group in the frequency domain to obtain the hard-thresholded frequency domain three-dimensional block group.Y first : ; In the formula, This indicates a hard threshold operation. , Expressed as the standard deviation of noise, These are empirical coefficients used to adjust the filter strength; The frequency domain 3D block group after hard threshold filtering is inversely transformed, and the inversely transformed 3D block group is aggregated by weighted average to obtain the estimated pixel value of the pixel block after the first stage of denoising: ; ; In the formula, This represents the estimated pixel value of the pixel block after the first stage of denoising; x Indicates the pixel position in the image. Indicates the weighting coefficient. A three-dimensional block group representing a spatial domain; This represents the number of non-zero coefficients retained in the frequency domain after the three-dimensional block group has been transformed and thresholded.

[0009] Preferably, the process of removing dynamic feature points from the denoised image to obtain the final static feature point set includes the following steps: A dynamic target segmentation mask for the denoised image is generated using the YOLO-fastestV2 model. The ORB feature points in the dynamic region of the denoised image are then removed using the dynamic target segmentation mask to generate a static feature point set. The LK optical flow algorithm is used to track feature points in the static feature point set, and feature points with speeds exceeding a threshold are removed to obtain an updated point set. Calculate the frame difference between adjacent frames in the update point set, generate a binary difference map, and remove residual feature points in the dynamic region based on the binary difference map to obtain the final static feature point set.

[0010] Preferably, the formula for calculating the frame difference is: ; In the formula, This represents the frame difference between frame t and frame (t-1) at coordinates (x, y), reflecting the change in pixel brightness between the two frames; Let represent the pixel brightness at position (x,y) in the t-th frame of the image. This represents the pixel brightness at position (x,y) in the (t-1)th frame of the image.

[0011] Preferably, the step of optimizing the 3D map points and the robot's camera pose based on the final static feature point set to generate a local 3D point cloud map includes the following steps: Extract ORB feature points from the current frame and the previous keyframe from the final static feature point set, and calculate the number of feature matching pairs. Combining dynamic object depth information and tracking results, and based on the adaptively adjusted scene view and robot movement speed threshold, it is determined whether the current frame meets the keyframe insertion conditions; if multiple frames consecutively meet the insertion conditions, keyframes with an overlap rate greater than a set value are merged by comparing feature matching overlap rates. After inserting keyframes, the camera pose and 3D map points are locally bundled and adjusted by minimizing the reprojection error using features from the final static feature point set to generate a local 3D point cloud map.

[0012] Preferably, the method further includes processing image sequences of multiple robots in the same scene to generate keyframes containing ORB features, depth information, and pose estimation; quantizing and compressing the ORB feature descriptors to generate compressed keyframes; and prioritizing the transmission of keyframes with more than a preset threshold of matching pairs. The receiving robot performs ORB feature matching on the shared keyframes, estimates the relative poses between robots based on the matching results, and uses an incremental graph optimization algorithm to update the poses and map points of each robot by minimizing the global reprojection error, thereby generating a consistent global 3D point cloud map.

[0013] This invention also proposes a visual SLAM system for mobile robots, comprising: A vision sensor, which is configured on a mobile robot, is used to acquire a sequence of raw environmental images of the location of the mobile robot. The image denoising module is used to divide the original environmental image sequence into multiple non-overlapping pixel blocks, denoted as reference blocks; a search window is set around each reference block, and a three-dimensional block group is formed by the reference blocks and other image blocks within the search window with similar pixel distribution to the reference blocks; The three-dimensional block group is mapped to the frequency domain through a three-dimensional transformation, and hard thresholding is applied to the three-dimensional block group in the frequency domain. The frequency domain three-dimensional block group after hard thresholding is then transformed inversely to convert the frequency domain data back into a three-dimensional block group in the spatial domain, generating the estimated pixel values ​​of the pixel blocks after the first stage of denoising. A new block group is formed by combining the estimated pixel values ​​of the pixel blocks after the first stage of denoising with the original environmental image sequence. Wiener filtering is applied to the new block group, and the block group results after Wiener filtering are weighted and averaged to generate a denoised image. The map generation module is used to remove dynamic feature points from the denoised image to obtain the final static feature point set; based on the final static feature point set, the 3D map points and the robot's camera pose are optimized to generate a local 3D point cloud map.

[0014] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in the mobile robot visual SLAM method.

[0015] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute any of the steps in the mobile robot visual SLAM method.

[0016] The visual SLAM method for mobile robots provided by this invention has the following beneficial effects: This invention divides the original environmental image sequence into non-overlapping pixel blocks. Similar blocks are searched within a search window around each reference block to form a three-dimensional block group. By aggregating redundant information from similar blocks, the ability to distinguish noise is enhanced, providing richer feature data for subsequent frequency domain processing. The three-dimensional block group is mapped from the spatial domain to the frequency domain. Utilizing the difference in the distribution of noise and effective information in the frequency domain, noise is more effectively suppressed while preserving image edge and texture details, thereby improving the denoising effect and providing a more stable input for subsequent ORB feature extraction. In the frequency domain, a hard threshold is used to initially suppress noise components while preserving effective information, completing the first stage of noise suppression. The second stage combines the denoising results from the first stage with the original image information, and further optimizes the denoising effect with Wiener filtering and weighted aggregation. This effectively suppresses complex noise (such as Gaussian white noise) in low-light and weak-texture environments, while preserving key details such as image edges and textures. This provides clear and stable input for subsequent ORB feature extraction, significantly reduces feature point loss or matching errors caused by noise, comprehensively improves the performance of visual SLAM in complex environments, and enhances the localization accuracy and mapping stability of the mobile robot visual SLAM system. Attached Figure Description

[0017] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the mobile robot visual SLAM method according to Embodiment 1 of the present invention; Figure 2 This is a flowchart of image denoising in Embodiment 1 of the present invention; Figure 3 This is a diagram illustrating the implementation process of the visual SLAM method for mobile robots according to Embodiment 1 of the present invention. Figure 4This is a flowchart of the multi-robot collaborative mapping method in Embodiment 3 of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0020] To address the shortcomings of existing technologies, such as insufficient image denoising performance, poor robustness in dynamic target removal, low keyframe selection efficiency, and inadequate real-time performance in multi-robot collaborative mapping, this invention provides a visual SLAM method for mobile robots. This method improves positioning accuracy, robustness, and real-time performance in complex environments by optimizing image denoising, dynamic target removal, keyframe selection, and multi-robot collaborative mapping techniques. The following details the technical problems, objectives, and implementation methods of this invention, including technical features, processing flow, mathematical principles, and accompanying drawings, providing a feasible technical solution for those skilled in the art.

[0021] Example 1 This invention provides a visual SLAM method for mobile robots, specifically a visual SLAM method based on optimized BM3D denoising, optical flow tracing, and keyframe matching. It is suitable for localization and map building in low-light, weak-texture, high-dynamic environments and multi-robot collaborative scenarios, and can be applied to mobile robot navigation, autonomous driving, augmented reality, and other fields. Figure 1 and Figure 3 As shown, it includes the following steps: Step 1: Collect visual sensor data from the mobile robot.

[0022] To achieve simultaneous localization and mapping (SLAM) of mobile robots in complex environments, this invention equips the mobile robot with a vision sensor to acquire raw environmental image data of the robot's location in real time, providing a foundation for subsequent localization and map building. Specifically, the robot is equipped with a monocular or RGB-D camera to capture a sequence of raw environmental images of the mobile robot's location. In addition, the robot is equipped with an inertial measurement unit (IMU) to collect angular velocity and acceleration data, aiding in the estimation of robot pose changes. All sensor data is processed in real time by an embedded processor. This sensor data provides reliable input for image denoising, feature extraction, and pose optimization, ensuring the system's robustness and real-time performance in low-light, weak-texture, or high-dynamic environments.

[0023] Step 2: Denoise the image based on the optimized BM3D algorithm.

[0024] Image noise in low-light or weak-texture environments significantly affects the accuracy of feature extraction in visual SLAM systems, leading to localization drift and map building errors. Therefore, this invention employs an optimized BM3D algorithm to process the acquired raw environmental image sequences. Preprocessing is performed to extend the similarity of two-dimensional image patches to three-dimensional space. Transform domain filtering is then used to fully utilize redundant information in the image, suppressing complex noise and preserving image details. For example... Figure 2 As shown, in specific implementation, the system will input the image. Divide into multiple non-overlapping pixel blocks, denoted as reference blocks. ,in Represents pixels, z ( x This is the original environment image. For real images, The noise is Gaussian white noise. These pixel blocks have no overlapping areas; each pixel belongs to only one block. This partitioning method avoids information redundancy or repetitive processing between blocks, improving the algorithm's processing efficiency while ensuring denoising effectiveness, thus meeting the real-time requirements of mobile robot visual SLAM systems. (Image captured) z ( x This does not equal the real-world scenario. Instead, it was affected by some kind of noise. n ( x The result of contamination, meaning the captured image is "clean image + noise," the core task of image denoising is to remove noise from... z ( x Recover as much as possible in ) .

[0025] For each reference block Search for similar blocks in the search window. Similarity is calculated based on weighted Euclidean distance. : ; In the formula, similarity This represents the similarity distance, which is used to determine whether similar blocks should be added to a 3D block group by comparing the similarity distance with a set threshold. N Representing the side length of a pixel block, it divides an image into multiple fixed-size image blocks, usually denoted as . N * N Pixels (e.g., 8×8, 16×16). The selected similar blocks are spatially related to the reference blocks. When they are arranged in a certain order, they form a set containing multiple two-dimensional image blocks. From the data structure perspective, it presents a "three-dimensional" feature, that is, the spatial dimension (width, height) of each two-dimensional block plus the number dimension of blocks (a sequence of multiple similar blocks), hence it is called a "three-dimensional block group".

[0026] Similar blocks are filtered by setting a threshold. If and only if Only when these conditions are met will they be added to the 3D block group. By combining similar 2D blocks into a 3D structure, subsequent 3D transformations (such as 3D Fourier transform) can be used to more effectively suppress noise in the frequency domain, while preserving the edge and texture details of the image, thereby improving the denoising effect and providing a more stable input for subsequent ORB feature extraction.

[0027] Through three-dimensional transformation , group the three-dimensional blocks Mapping to the frequency domain yields the frequency domain result after the first stage of filtering. In the frequency domain, hard thresholding is applied:

[0028] ; In the formula, This represents the frequency domain result after the first stage of filtering, i.e., the three-dimensional block group in the frequency domain after hard threshold filtering; Y Represents the frequency domain coefficients after three-dimensional transformation. This indicates a hard threshold operation, typically set to... , Expressed as the standard deviation of noise, These are empirical coefficients used to adjust the filter strength. After filtering, an inverse transform is performed to generate the first-stage estimate. This estimate is then aggregated using a weighted average to obtain the estimated pixel value at pixel position x after the first-stage denoising.

[0029] ; ; In the formula, This represents the estimated pixel value at pixel position x after the first stage of denoising; x Indicates the pixel position in the image. Indicates the weighting coefficient. This represents a three-dimensional block group that has undergone inverse transformation back to the spatial domain after the first stage of filtering. This represents the number of non-zero coefficients retained in the frequency domain after the three-dimensional block group has been transformed and thresholded.

[0030] By aggregating similar blocks, a data foundation is provided for subsequent 3D transformation denoising, thereby more effectively suppressing image noise while preserving the edge and texture details of the image, creating more stable conditions for subsequent ORB feature extraction.

[0031] Second phase utilization The image is grouped into blocks with the original environment image, Wiener filtering is applied, and a weighted average is used to generate the final denoised image. ; In the formula, For denoised images, denoised images Used for ORB feature extraction, it significantly reduces feature point loss caused by noise and improves the stability of localization and mapping. Indicating the second phase, utilizing The three-dimensional block group is obtained by Wiener filtering the block group formed by the original environment image and the block group.

[0032] The second stage combines the denoising results from the first stage with the original image information, and further optimizes the denoising effect with Wiener filtering and weighted aggregation, making the final denoised image more suitable for ORB feature extraction, reducing feature point loss caused by noise, and improving the stability of SLAM system localization and mapping.

[0033] The proposed efficient image denoising method overcomes the problem of insufficient suppression of complex noise by traditional filtering methods, reduces the risk of feature point loss or matching errors in low-light and weak-texture environments, and meets the real-time requirements of mobile robot platforms.

[0034] Step 3: Remove dynamic targets from the final denoised image.

[0035] In highly dynamic environments, dynamic targets are often mistaken for static feature points, leading to unstable feature matching and consequently affecting the localization and mapping accuracy of SLAM systems. To address this, this invention combines lightweight YOLO depth segmentation, Lucas-Kanade (LK) optical flow tracing, and frame difference methods to construct a multi-layered dynamic target removal mechanism. Furthermore, the system incorporates angular velocity and acceleration information acquired by the inertial measurement unit (IMU) to help determine whether the current frame is in a phase of rapid motion and whether the possibility of dynamic interference is increasing. By predicting the robot's motion state through trends in angular velocity and acceleration, motion priors are provided for the dynamic target removal mechanism in subsequent images, improving the system's perception accuracy in fast-moving scenarios.

[0036] In practice, for denoised images A lightweight YOLO-fastestV2 model is applied to generate dynamic object segmentation masks. ,in Indicates a dynamic region. This represents a static region. Specifically, the YOLO-fastestV2 model uses an end-to-end convolutional neural network structure. First, it extracts multi-scale features from the image, then detects targets belonging to a preset dynamic category and outputs the corresponding bounding boxes and category confidence scores. The system maps the detection results to a binary mask image, marking all dynamic category target bounding box regions as 1 (dynamic region) and the rest as 0 (static region), thus generating a dynamic target segmentation mask. In highly dynamic environments, such as moving pedestrians and vehicles in video scenes, or moving mechanical parts in industrial scenes, the image regions occupied by these moving targets are called dynamic regions. (Corresponding segmentation mask) The "" part represents the area in the image that belongs to a dynamic target (an object that moves autonomously or is affected by the environment), which is different from the background and the static area where stationary objects are located.

[0037] Next, ORB feature points within the dynamic region are removed using a mask to generate a static feature point set. .

[0038] Subsequently, The feature points in the image are analyzed using the LK optical flow algorithm, based on brightness consistency constraints. ; In the formula, This represents the brightness of the pixel at image coordinates (x, y) at time t; Indicates time The brightness of the pixel shifted to (x+u,y+v) due to motion; x Represents the x-coordinate of a pixel in the image plane. y Represents the ordinate of a pixel in the image plane. t Indicates the initial time. u This represents the displacement of a pixel in the x-direction. v This represents the displacement of a pixel in the y-direction. This represents the time increment, i.e., from t to ( ( ) time interval.

[0039] Minimize the error within a local window: ; In the formula, Indicates optical flow estimation ( The error between the robot's displacement and the actual situation is used to measure the matching deviation under the assumption of brightness conservation. This means performing error calculation on a neighboring pixel centered at (x,y) in the image (e.g., a square / circular window covering multiple pixels), and solving the optical flow by accumulating the error within the window and utilizing local smoothness constraints.

[0040] The LK optical flow algorithm is used to trace the set of static feature points retained after dynamic region culling. It assists in analyzing feature changes in static scenes, improving the accuracy of target detection and localization in highly dynamic environments.

[0041] Solving for motion vectors The removal speed exceeds the threshold. Feature points are used to obtain the updated point set. Finally, the frame difference between adjacent frames in the update point set is calculated using the frame difference method:

[0042] ; In the formula, This represents the frame difference between frame t and frame (t-1) at coordinates (x, y), reflecting the change in pixel brightness between the two frames; Let represent the pixel brightness at position (x,y) in the t-th frame of the image. This represents the pixel brightness at position (x,y) in the (t-1)th frame of the image.

[0043] Based on the calculated adjacent frame difference With threshold Compare, if the frame difference is greater than If it is less than or equal to, it is determined to be a dynamic region. If the dynamic region is not a static region, it is determined to be a static region. A binary difference map is generated based on the difference between the dynamic and static regions. Residual feature points in the dynamic region are then removed based on the binary difference map to obtain the final set of static feature points. This mechanism significantly improves the robustness of feature matching and the accuracy of SLAM localization in dynamic scenes through YOLO fast localization, LK optical flow precise tracking, and frame difference method verification.

[0044] The robust dynamic target elimination method proposed above, through the combination of depth segmentation and optical flow tracing, can efficiently identify and eliminate interfering targets in highly dynamic environments, solve the problem of insufficient detection and tracking capabilities of existing technologies for fast-moving targets, and improve the stability of feature matching and the positioning accuracy of SLAM systems in dynamic environments.

[0045] Step 4: Adaptive detection and filtering of keyframes in dynamic environments.

[0046] Keyframe selection is a core component of SLAM systems. Existing technologies typically select keyframes based on fixed thresholds, but this method does not fully consider the dynamic characteristics of complex scenes, leading to keyframe redundancy or insufficiency. Keyframe redundancy increases computational burden and reduces system real-time performance; insufficient keyframes weaken pose optimization effects, affecting positioning accuracy and ground motion. Figure 1To address the impact of varying motion levels on detection algorithm performance in dynamic environments, this invention proposes a dynamic keyframe selection strategy. Based on an environmental assessment threshold, the system adaptively selects a dynamic method, balancing YOLO recall with frame difference accuracy.

[0047] Specifically, for the previous frame of a continuous frame image sequence With the current frame Define the inter-frame grayscale difference matrix The scene dynamic intensity estimated by the global mean can be expressed as: ; In the formula, The scene dynamic intensity serves as an indicator of the degree of environmental change, guiding the selection of subsequent detection methods. This is based on the dynamic intensity and a given threshold. The decision function is:

[0048] ; in The representative decision function, if the dynamic strength Exceeding the threshold Call the YOLO detection model To ensure high recall, use the frame difference thresholding method instead. Generate a binarized region indicator map to improve local accuracy. The system constructs a dynamic decision map. The system identifies whether a pixel belongs to a dynamic region, and then performs an element-by-element filtering operation. The final filtered image is generated through element-by-element dynamic region suppression.

[0049] ; in The final selected image is formed by extracting static and stable image portions through pixel-by-pixel multiplication to suppress the influence of dynamic regions. ORB feature points are then extracted from the final static feature point set, comparing the current frame with the previous keyframe, and the number of feature matching pairs is calculated. This method precisely selects keyframes, improving the robustness of feature matching and the accuracy of SLAM localization in dynamic scenes. If multiple frames consecutively meet the insertion conditions, the system compares the overlap rate of feature matching and merges keyframes with high overlap rates to reduce redundancy. After keyframe insertion, the camera pose and 3D map points are locally bundled and adjusted by minimizing reprojection error, optimizing the camera pose and 3D map points to generate a local 3D point cloud map. The core of this invention focuses on "real-time pose estimation of cameras / sensors in unknown environments," that is, dynamically calculating the current position and orientation of the device in 3D space through sensor data and simultaneously providing pose constraints for map construction. The local bundling and adjustment by minimizing reprojection error specifically optimizes the coordinates of camera pose and 3D map points within a local range, making the theoretical position of the 3D points projected onto the image coincide as much as possible with the actual observed feature point position, thereby correcting pose deviations and map point errors and improving the accuracy of the local map. This method significantly improves the robustness of pose estimation and mapping efficiency in fast-moving or dynamic scenes through dynamic threshold adjustment and overlap rate optimization.

[0050] like Figure 3 As shown, when feature matching in dynamic scenes is unstable (e.g., weak textures, or loss of feature points due to rapid motion), a "constant velocity model" is introduced as a priori motion state. This model, based on the motion speed of the previous few frames, assumes the robot maintains a constant speed for a short period and predicts the initial pose of the current frame. The prediction result serves as an initial reference for dynamic detection and feature matching, reducing pose estimation jumps caused by missing features and working in conjunction with the dynamic intensity determination logic to improve the continuity of keyframe selection. When the dynamic intensity indicates severe scene motion and insufficient feature matching, the initial pose provided by the constant velocity model can help optimize the generation accuracy of the dynamic decision map, ensuring the stability of keyframe selection.

[0051] The efficient keyframe selection strategy proposed in this invention overcomes the problem of keyframe redundancy or insufficiency caused by the fixed threshold method in complex scenes, improving pose optimization and system real-time performance. Simultaneously, the efficient multi-robot collaborative mapping method optimizes keyframe sharing and data transmission mechanisms, alleviating communication limitations in low-bandwidth environments and improving the global mapping performance of the multi-robot SLAM system. Figure 1 Consistency and real-time performance.

[0052] This invention addresses the image noise interference problem faced by mobile robots in visual SLAM under low light, weak texture, and high dynamic environments. It utilizes an optimized BM3D algorithm with block matching and 3D transformation techniques to generate high-quality denoised images, effectively suppressing noise and providing clear input for subsequent feature extraction, thus improving the robustness and stability of feature extraction. By combining a lightweight YOLO depth segmentation model, LK optical flow tracing algorithm, and frame difference method, it detects and removes dynamic target interference in real time, accurately tracks key feature points, and generates stable feature matching results, significantly improving feature matching accuracy in low light, weak texture, and high dynamic scenes. Furthermore, through an optimized keyframe matching mechanism, it dynamically selects keyframes and minimizes reprojection errors, enhancing localization and model building. Figure 1 This invention achieves high consistency and accurately reflects the location information of mobile robots and the quality of environmental map construction. By integrating BM3D denoising, YOLO, optical flow tracing, and frame difference methods for collaborative processing, as well as keyframe matching optimization, it comprehensively improves the performance of visual SLAM in complex environments, significantly enhancing positioning accuracy and map stability. It is suitable for robot navigation and autonomous driving scenarios.

[0053] Example 2 Based on the same inventive concept, this invention also proposes a mobile robot visual SLAM system. The mobile robot visual SLAM system based on BM3D optical flow tracing (hereinafter referred to as the BLO-SLAM system) includes: A vision sensor, which is configured on a mobile robot, is used to acquire a sequence of raw environmental images of the location of the mobile robot.

[0054] The image denoising module is used to divide the original environmental image sequence into multiple non-overlapping pixel blocks, referred to as reference blocks; a search window is set around each reference block, and a three-dimensional block group is formed by the reference blocks and other image blocks within the search window with similar pixel distribution to the reference blocks.

[0055] The three-dimensional block group is mapped to the frequency domain through a three-dimensional transformation, and hard thresholding is performed on the three-dimensional block group in the frequency domain. The frequency domain three-dimensional block group after hard thresholding is then transformed inversely to convert the frequency domain data back into a three-dimensional block group in the spatial domain, generating the estimated pixel value of the pixel block after the first stage of denoising.

[0056] A new block group is formed by combining the estimated pixel values ​​of the pixel blocks after the first stage of denoising with the original environmental image sequence. Wiener filtering is applied to the new block group, and the block group results after Wiener filtering are weighted and averaged to generate a denoised image.

[0057] The map generation module is used to remove dynamic feature points from the denoised image to obtain the final static feature point set; based on the final static feature point set, the 3D map points and the robot's camera pose are optimized to generate a local 3D point cloud map.

[0058] Example 3 Multi-robot SLAM builds a consistent global map through distributed data sharing, but existing technologies have limitations in data transmission efficiency and global optimization complexity. The large volume of data communication is susceptible to low-bandwidth environments, and delays exist in real-time keyframe sharing and global pose optimization, making it difficult to meet the global consistency and real-time requirements of multi-robot collaborative mapping in complex scenarios.

[0059] To solve the above problems, such as Figure 4 As shown, this embodiment proposes a method for multi-robot collaborative mapping based on the mobile robot visual SLAM method provided in Embodiment 1 and the BLO-SLAM system provided in Embodiment 2.

[0060] Multi-robot SLAM systems need to optimize data transmission under low bandwidth conditions and ensure global coverage. Figure 1 To ensure consistency and meet real-time requirements, this invention designs a collaborative mapping mechanism based on keyframe sharing and pose fusion. In specific implementation, each robot runs a local BLO-SLAM system. Through the "visual front-end system," data is acquired, other visual data is processed, and visual features are fused. Combined with BM3D image processing, image preprocessing, feature extraction, and feature tracking, keyframes containing ORB features, depth information, and pose estimation are generated.

[0061] In the backend system, the "Natural Intelligence Algorithm" provides the intelligent decision-making foundation for the entire collaborative mapping process, assisting in optimizing feature processing and pose calculation logic; the "Guiding Law Algorithm" plans the action guidance strategy for the robot mapping process based on environmental perception and task requirements, such as the timing of key frame acquisition and transmission; the "Consensus Algorithm" ensures that multiple robots reach a consistent understanding of key map construction parameters (such as pose and map points) in a distributed environment; and the "Relative Pose Fusion" fuses the accurate relative poses between robots based on the local poses and interaction matching information of each robot, supporting the unification of the global map.

[0062] In the system operation process, the robot first attempts to "establish a communication link". If the communication is abnormal, it triggers the detection of "whether the working environment is abnormal", which covers the abnormal judgment of communication, mapping and other dimensions. If the environment is abnormal, "loop closure detection" is started to correct the pose and map deviation to ensure the accuracy of mapping. When the mapping process is advanced, and all links are completed in coordination and the predetermined conditions are met, the "task is completed" is determined.

[0063] To reduce communication overhead, the system quantizes and compresses ORB feature descriptors, generates compressed keyframes, and prioritizes transmitting the number of matching pairs. Keyframes exceeding a preset threshold. The receiving robot performs ORB feature matching on the shared keyframes, estimates the relative poses between robots based on the matching results, and uses an incremental graph optimization algorithm to update the poses and map points of each robot by minimizing the global reprojection error, generating a consistent global 3D point cloud map. This mechanism effectively alleviates communication bottlenecks and improves the accuracy and real-time performance of map building in multi-robot collaborative scenarios through efficient keyframe compression and incremental optimization.

[0064] This invention has the following advantages: 1. Application of the optimized BM3D algorithm By introducing the BM3D (Block-Matching and 3D Filtering) denoising algorithm, image noise in complex dynamic scenes is effectively removed while preserving edge and texture details, improving the stability and matching accuracy of ORB feature point extraction, and significantly enhancing the robustness of the SLAM system in low-light or motion-blurred environments.

[0065] 2. Lightweight YOLO depth segmentation combined with LK optical flow tracing and frame difference method.

[0066] This invention designs a lightweight YOLO model for dynamic object segmentation, combining LK optical flow tracing and frame difference methods to efficiently identify and eliminate interfering targets in dynamic scenes. Compared to traditional heavy semantic segmentation models, this method reduces computational complexity, is suitable for resource-constrained mobile robots, and simultaneously improves the accuracy of feature matching and the robustness of pose estimation.

[0067] 3. Dynamic detection keyframe selection strategy This invention proposes a keyframe screening mechanism based on dynamic intensity. It estimates the scene dynamic intensity by calculating the inter-frame gray-level difference matrix, and generates a dynamic judgment map by combining threshold adaptive selection YOLO or frame difference method. It then filters candidate keyframes corresponding to static features, suppresses residual interference in dynamic regions, and improves the mapping accuracy and real-time performance in highly dynamic scenes.

[0068] 4. Multi-robot collaborative mapping mechanism This invention designs a multi-robot collaborative mapping strategy based on keyframe sharing and pose fusion, which is achieved by quantizing and compressing ORB feature descriptors and prioritizing the transmission of high-matching-pair numbers. Keyframes reduce communication overhead in low-bandwidth environments, enabling relative pose estimation and global positioning between robots. Figure 1 Consistency, improving the efficiency of collaborative mapping.

[0069] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in the mobile robot visual SLAM method embodiment. Specific implementation methods can be found in the method embodiment, and will not be repeated here.

[0070] Furthermore, the present invention also provides a non-transitory computer-readable storage medium containing instructions on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in the embodiments of the mobile robot visual SLAM method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0071] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A visual SLAM method for mobile robots, characterized in that, include: Collect raw environmental image sequences of the mobile robot's location; The original environmental image sequence is divided into multiple non-overlapping pixel blocks, denoted as reference blocks; A search window is set around each reference block, and a three-dimensional block group is formed by the reference block and other image blocks within the search window that have similar pixel distribution to the reference block. The three-dimensional block group is mapped to the frequency domain through a three-dimensional transformation, and hard thresholding is applied to the three-dimensional block group in the frequency domain. The frequency domain three-dimensional block group after hard thresholding is then transformed inversely to convert the frequency domain data back into a three-dimensional block group in the spatial domain, generating the estimated pixel values ​​of the pixel blocks after the first stage of denoising. A new block group is formed by combining the estimated pixel values ​​of the pixel blocks after the first stage of denoising with the original environmental image sequence. Wiener filtering is applied to the new block group, and the block group results after Wiener filtering are weighted and averaged to generate a denoised image. By removing dynamic feature points from the denoised image, the final static feature point set is obtained; Based on the final static feature point set, optimize the 3D map points and the robot's camera pose to generate a local 3D point cloud map.

2. The mobile robot visual SLAM method according to claim 1, characterized in that, The step of setting a search window around each reference block and forming a three-dimensional block group by using the reference blocks and other image blocks within the search window with similar pixel distributions to the reference blocks includes the following steps: Set a search window around each reference block, and search for each reference block. Search for similar blocks in the search window. Calculate the reference block based on weighted Euclidean distance Similar blocks similarity : ； In the formula, Indicates the side length of the pixel block; Similarity with a fixed threshold To make a comparison, if and only if Then, the selected similar blocks are added to the three-dimensional block group to obtain the final three-dimensional block group.

3. The mobile robot visual SLAM method according to claim 2, characterized in that, The process involves mapping the three-dimensional block group to the frequency domain through a three-dimensional transformation, performing hard threshold filtering on the three-dimensional block group in the frequency domain, and then performing an inverse transformation on the frequency domain three-dimensional block group after hard threshold filtering to convert the frequency domain data back into a spatial domain three-dimensional block group. The process of generating estimated pixel values ​​for the first-stage denoised pixel blocks includes the following steps: Through three-dimensional transformation The three-dimensional block group is mapped to the frequency domain; hard thresholding is then applied to the three-dimensional block group in the frequency domain to obtain the hard-thresholded frequency domain three-dimensional block group. Y first : ； In the formula, Y Represents the frequency domain coefficients after three-dimensional transformation. This indicates a hard threshold operation. , Expressed as the standard deviation of noise, These are empirical coefficients used to adjust the filter strength; The frequency domain 3D block group after hard threshold filtering is inversely transformed, and the inversely transformed 3D block group is aggregated by weighted average to obtain the estimated pixel value of the pixel block after the first stage of denoising: ； ； In the formula, This represents the estimated pixel value of the pixel block after the first stage of denoising; x Indicates the pixel position in the image. Indicates the weighting coefficient. A three-dimensional block group representing a spatial domain; This represents the number of non-zero coefficients retained in the frequency domain after the three-dimensional block group has been transformed and thresholded.

4. The mobile robot visual SLAM method according to claim 3, characterized in that, The process of removing dynamic feature points from the denoised image to obtain the final static feature point set includes the following steps: A dynamic target segmentation mask for the denoised image is generated using the YOLO-fastestV2 model. The ORB feature points in the dynamic region of the denoised image are then removed using the dynamic target segmentation mask to generate a static feature point set. The LK optical flow algorithm is used to track feature points in the static feature point set, and feature points with speeds exceeding a threshold are removed to obtain an updated point set. Calculate the frame difference between adjacent frames in the update point set, generate a binary difference map based on the frame difference, and remove residual feature points in the dynamic region based on the binary difference map to obtain the final static feature point set.

5. The mobile robot visual SLAM method according to claim 4, characterized in that, The formula for calculating the frame difference is: ； In the formula, This represents the frame difference between frame t and frame (t-1) at coordinates (x, y), reflecting the change in pixel brightness between the two frames; Let represent the pixel brightness at position (x,y) in the t-th frame of the image. This represents the pixel brightness at position (x,y) in the (t-1)th frame of the image.

6. The mobile robot visual SLAM method according to claim 5, characterized in that, The process of optimizing 3D map points and robot camera poses based on the final static feature point set to generate a local 3D point cloud map includes the following steps: Extract ORB feature points from the current frame and the previous keyframe from the final static feature point set, and calculate the number of feature matching pairs. Combining dynamic object depth information and tracking results, and based on the adaptively adjusted scene view and robot movement speed threshold, it is determined whether the current frame meets the keyframe insertion conditions; if multiple frames consecutively meet the insertion conditions, keyframes with an overlap rate greater than a set value are merged by comparing feature matching overlap rates. After inserting keyframes, the camera pose and 3D map points are locally bundled and adjusted by minimizing the reprojection error using features from the final static feature point set to generate a local 3D point cloud map.

7. The mobile robot visual SLAM method according to claim 6, characterized in that, It also includes processing image sequences of multiple robots in the same scene to generate keyframes containing ORB features, depth information, and pose estimation; quantizing and compressing ORB feature descriptors to generate compressed keyframes; and prioritizing the transmission of keyframes with more than a preset threshold of matching pairs. The receiving robot performs ORB feature matching on the shared keyframes, estimates the relative poses between robots based on the matching results, and uses an incremental graph optimization algorithm to update the poses and map points of each robot by minimizing the global reprojection error, thereby generating a consistent global 3D point cloud map.

8. A mobile robot visual SLAM system, characterized in that, include: A vision sensor, which is configured on a mobile robot, is used to acquire a sequence of raw environmental images of the location of the mobile robot. The image denoising module is used to divide the original environmental image sequence into multiple non-overlapping pixel blocks, denoted as reference blocks; a search window is set around each reference block, and a three-dimensional block group is formed by the reference blocks and other image blocks within the search window with similar pixel distribution to the reference blocks; The three-dimensional block group is mapped to the frequency domain through a three-dimensional transformation, and hard thresholding is applied to the three-dimensional block group in the frequency domain. The frequency domain three-dimensional block group after hard thresholding is then transformed inversely to convert the frequency domain data back into a three-dimensional block group in the spatial domain, generating the estimated pixel values ​​of the pixel blocks after the first stage of denoising. A new block group is formed by combining the estimated pixel values ​​of the pixel blocks after the first stage of denoising with the original environmental image sequence. Wiener filtering is applied to the new block group, and the block group results after Wiener filtering are weighted and averaged to generate a denoised image. The map generation module is used to remove dynamic feature points from the denoised image to obtain the final static feature point set. Based on the final static feature point set, optimize the 3D map points and the robot's camera pose to generate a local 3D point cloud map.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 7.

Images