Fish-eye Camera SLAM Method and Device, System, Storage Medium

Abstract

The present invention discloses a fish-eye camera SLAM method, device, system, and storage medium, including: a wide-angle image obtained by a binocular fish-eye camera; preprocessing the wide-angle image, and using an ORB feature extractor to extract visual feature points from the preprocessed image; according to the visual feature points, performing dense optical flow estimation on adjacent image frames through a fish-eye image optical flow estimation network model to obtain pixel-level motion information; based on the optical flow estimation result, formulating a region proposal strategy for feature matching, obtaining a preliminary pose estimation result through ORB feature matching, performing image reprojection using the preliminary pose result, and combining the optical flow estimation result to eliminate outliers; using the optimized matching point pairs to re-estimate the pose, and performing local optimization and global optimization; outputting the global pose transformation result and the constructed point cloud map. By adopting the technical solution of the present invention, the matching accuracy in the fish-eye camera environment is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a fisheye camera SLAM method, device, system, and storage medium. Background Technique

[0002] Simultaneous Localization and Mapping (SLAM) is a technology that enables robots or intelligent devices to simultaneously perform localization and map construction in unknown environments, and is widely used in fields such as robot navigation, augmented reality, and autonomous driving. Most traditional SLAM methods rely on ordinary pinhole cameras for feature extraction and matching. However, due to the wide-angle field of view and strong environmental perception ability of fisheye cameras, which can provide more extensive perception information in complex or narrow environments, the application of fisheye cameras in SLAM has gradually become a research hotspot, and the technologies required mainly include computer vision, deep learning, video understanding, computational geometry, and optimization techniques.

[0003] Computer vision technology: In visual SLAM, computer vision algorithms first extract stable feature points from the images captured by the camera. Commonly used feature extraction methods such as ORB, SIFT, and SURF, etc. These feature points are matched between adjacent frames to calculate the relative pose of the camera. Through the matching of these feature points and the application of geometric models, visual SLAM can estimate the motion trajectory of the camera in space. At the same time, computer vision also further improves the accuracy of pose estimation and map construction through image reprojection and error optimization.

[0004] Deep learning technology: It is mainly reflected in aspects such as feature extraction, optical flow estimation, depth map prediction, and optimization. Traditional visual SLAM relying on manually designed feature extraction methods may perform poorly in complex environments and dynamic scenes. Deep learning can automatically learn useful features in images, thereby improving the robustness and accuracy of feature extraction. For example, convolutional neural networks (CNNs) can be used to extract high-level semantic features from images, enabling the SLAM system to recognize and track more details in complex environments. Deep learning also plays an important role in optical flow estimation, by training deep models to accurately estimate pixel-level motion information, especially performing excellently in dense scenes.

[0005] Video understanding technology: Its applications in visual SLAM are mainly reflected in aspects such as dynamic scene analysis, motion estimation, and multi-frame information fusion. By analyzing the temporal information in a continuous image sequence, video understanding technology can effectively capture and understand the motion laws of objects, thus providing more accurate positioning and mapping capabilities for visual SLAM. Specifically, video understanding technology can obtain pixel-level motion information of objects or cameras in images through optical flow estimation, providing strong support for feature matching. In terms of multi-frame information fusion, video understanding technology can effectively integrate visual data between different time frames, and through temporal modeling and semantic understanding, improve the stability and accuracy of pose estimation. Especially in dynamic scenes, video understanding can help the SLAM system distinguish static backgrounds from dynamic objects, reducing the interference of dynamic objects on map construction and positioning.

[0006] Computational geometry and optimization technology: Its applications in visual SLAM are mainly reflected in aspects such as pose estimation, map optimization, and error minimization. The SLAM system processes feature point matching in images and camera pose calculation through computational geometry methods, using geometric models such as the fundamental matrix and homography matrix to estimate the relative motion of the camera. In addition, computational geometry also plays a key role in three-dimensional reconstruction, helping to construct accurate three-dimensional maps. Optimization technology, especially Bundle Adjustment, optimizes the camera pose and the positions of map points by minimizing the image reprojection error, thereby improving the positioning accuracy and map quality. Loop detection and global optimization use optimization methods to reduce error accumulation, ensuring the stability and robustness of the SLAM system during long-term operation. Computational geometry and optimization technology enable visual SLAM to achieve high-precision positioning and mapping in complex environments.

[0007] Existing visual SLAM methods often face problems of insufficient matching accuracy and error accumulation in large field-of-view environments. Especially when using a fisheye camera, wide-angle images are prone to large distortions, resulting in poor performance of traditional feature extraction and matching algorithms in complex scenes. In addition, due to large field-of-view images containing more dynamic elements, existing technologies often cannot effectively eliminate these outliers, causing error accumulation in the map and positioning drift. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a fisheye camera SLAM method, device, system, and storage medium. By introducing optical flow estimation to guide feature matching, the matching accuracy is improved, and outliers in low-quality matches are effectively eliminated, thereby significantly improving the accuracy of pose estimation and map construction, and ensuring efficient positioning and mapping in large field-of-view and dynamic environments.

[0009] To achieve the above object, the present invention adopts the following technical solutions:

[0010] A fish-eye camera SLAM method, comprising:

[0011] Wide-angle images obtained by a binocular fish-eye camera;

[0012] Preprocess the wide-angle images, and use an ORB feature extractor to extract visual feature points from the preprocessed images;

[0013] According to the visual feature points, perform dense optical flow estimation on adjacent image frames through a fish-eye image optical flow estimation network model to obtain pixel-level motion information;

[0014] Based on the optical flow estimation results, formulate a region proposal strategy for feature matching, and obtain a preliminary pose estimation result through ORB feature matching,

[0015] Use the preliminary pose result for image reprojection, and combine the optical flow estimation result to eliminate outliers;

[0016] Use the optimized matching point pairs to re-estimate the pose, and perform local optimization and global optimization;

[0017] Output the global pose transformation result and the constructed point cloud map.

[0018] Preferably, based on the optimized matching point pairs, use the bundle adjustment method for local optimization to adjust the camera pose and the positions of 3D map points, and minimize the reprojection error in all images.

[0019] Preferably, perform loop detection on key frames and execute global optimization; wherein, when an image is determined to be a key frame, use the loop detection algorithm to identify whether the current frame has an overlapping area with a previous frame, and if a loop is detected, feedback the loop information to the optimization framework for global optimization.

[0020] The present invention also provides a fish-eye camera SLAM device, comprising:

[0021] A first processing module for obtaining wide-angle images through a binocular fish-eye camera;

[0022] A second processing module for preprocessing the wide-angle images and using an ORB feature extractor to extract visual feature points from the preprocessed images;

[0023] A third processing module for performing dense optical flow estimation on adjacent image frames according to the visual feature points through a fish-eye image optical flow estimation network model to obtain pixel-level motion information;

[0024] A fourth processing module for formulating a region proposal strategy for feature matching based on the optical flow estimation results and obtaining a preliminary pose estimation result through ORB feature matching,

[0025] The fifth processing module is used to perform image reprojection using the preliminary pose result and eliminate abnormal points in combination with the optical flow estimation result;

[0026] The sixth processing module is used to re - estimate the pose using the optimized matching point pairs and perform local optimization and global optimization;

[0027] The seventh processing module is used to output the global pose transformation result and the constructed point cloud map.

[0028] Preferably, the sixth processing module is used to perform local optimization based on the optimized matching point pairs by using the bundle adjustment method, adjust the camera pose and the positions of 3D map points, and minimize the reprojection error in all images.

[0029] Preferably, the sixth processing module is used to perform loop closure detection on key frames and perform global optimization; wherein, when an image is determined to be a key frame, the loop closure detection algorithm is used to identify whether the current frame has an overlapping area with a previous frame. If a loop closure is detected, the loop closure information is fed back to the optimization framework for global optimization.

[0030] An embodiment of the present invention also provides a fisheye camera SLAM system, including: a memory and a processor. A computer program is stored on the memory and run by the processor, and the computer program executes the fisheye camera SLAM method when run by the processor.

[0031] An embodiment of the present invention also provides a storage medium, on which a computer program is stored, and the computer program executes the fisheye camera SLAM method when running.

[0032] Compared with the prior art, the present invention has the following advantages and technical effects:

[0033] 1. By introducing the strategy of using optical flow estimation to guide feature matching, the present invention effectively improves the matching accuracy of fisheye cameras in large - field - of - view environments.

[0034] 2. Through the differential analysis of the reprojection and optical flow estimation results, abnormal points are effectively identified and eliminated. In traditional SLAM systems, incorrect matching points often occur during the feature matching process, especially in dynamic environments, which affects the positioning accuracy and map quality. By analyzing the differences in optical flow estimation of matching points, the present invention can detect abnormal matching points in advance and eliminate them in a timely manner, avoiding the influence of abnormal points on subsequent pose estimation and map construction, thereby improving the robustness and reliability of the SLAM system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0036] Figure 1 Flowchart of the fisheye camera SLAM method according to an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of data processing of the fisheye camera SLAM method according to an embodiment of the present invention;

[0038] Figure 3 Schematic diagram of the feature matching result according to an embodiment of the present invention;

[0039] Figure 4 Schematic diagram of the optical flow estimation result according to an embodiment of the present invention;

[0040] Figure 5 Schematic diagram of the experimental result display according to an embodiment of the present invention. Detailed implementation manners

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0042] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0043] Embodiment 1:

[0044] As Figure 1 , 2 shown, this embodiment provides a fisheye camera SLAM method guided by optical flow estimation, including the following steps:

[0045] Step 1, data acquisition: Take the wide-angle images obtained by the binocular fisheye camera and the camera configuration parameters as inputs. Specifically, it includes a binocular fisheye camera with a horizontal field of view angle range of 170 degrees to 180 degrees and a vertical field of view angle range of 140 degrees to 160 degrees. Then, through the calibration process, obtain the internal and external parameter matrices of the camera. Use the binocular fisheye camera to collect images.

[0046] Step 2, Image Preprocessing: Preprocess the image, specifically including image grayscale processing, image pyramid construction, and extracting visual feature points from the preprocessed image using an ORB feature extractor. The matching results are as Figure 3 shown;

[0047] Step 3, Dense Optical Flow Estimation: Apply a deep learning method optimized for fisheye images to perform dense optical flow estimation on adjacent image frames to obtain pixel-level motion information. This step requires the following sub-steps: data preparation, network model construction, model training, and model application.

[0048] Step 3-1 Data Preparation: Use the existing fisheye image optical flow estimation dataset SynWoodScape as the data source and split it into a training set and a test set according to a ratio of 7:3.

[0049] Step 3-2 Network Model Construction: The fisheye image optical flow estimation network consists of an encoder, information fusion, optical flow prediction, and a decoder:

[0050] (1) Feature Encoding: Extract the adjacent frame correlation feature, fisheye image dynamic edge feature, and context feature respectively, as follows:

[0051] Adjacent Frame Correlation Feature: First, concatenate the previous frame I1 and the current frame I2, and then use a weight-sharing convolutional neural network N to extract the correlation feature to obtain the dense downsampled features F1 and F2:

[0052] F1 = N(I1), F2 = N(I2)

[0053] where D represents the feature dimension, and W and H represent the width and height respectively. The correlation of all pixel points is represented by calculating the cosine similarity between the two feature maps:

[0054]

[0055] where each element in Cor ijkl represents the correlation between the coordinate P1=(i,j) in F1 and the coordinate P1=(k,l) in F2. Finally, perform pooling operations of different sizes on the last two dimensions of Cor to obtain:

[0056] Cor a = {Cor1, Cor2, Cor3, Cor4}

[0057] where

[0058] Dynamic Edge Features and Context Features of Fish-Eye Images: In the present invention, by designing a graphics processing method, a dynamic object mask for the edge region is obtained and used as prior information for the network. First, I1 and I2 are converted into grayscale images, and the motion relationship between the two frames, I, is obtained by calculating the absolute frame difference. D Then, the position of the strongly distorted edge region is determined according to the polynomial model and the image size. Specifically, a coordinate system is established with the center of the image as the origin, and the major axis a and minor axis b of the elliptical region are determined according to the distortion degrees of the pixel points on the x-axis and y-axis. The region outside the ellipse is regarded as the strongly distorted edge region, and the calculation formula is as follows:

[0059]

[0060] Where X t represents the number of pixels of the pixel point on the positive direction of the x-axis from the origin, and X t represents the number of pixels of the pixel point after correction using the polynomial model from the origin. Y c and Y c are the same. τ represents the deviation rate threshold. Finally, after multiple rounds of denoising, connected component analysis, and opening and closing operations, the edge dynamic object mask I t is obtained after removing noise points and most of the background. mask .

[0061] The deformable residual network is used to perform feature encoding on I mask and I2 respectively, and the dynamic edge feature F motion and the context feature F context are obtained;

[0062] (2) Information Fusion: The convolutional block attention mechanism module is used to fuse the context feature and the dynamic edge feature to obtain the dynamic context enhanced feature;

[0063] (3) Optical Flow Prediction: The above features are fused through a gated recurrent unit, and the optical flow offset is iteratively updated to obtain the optical flow estimation result at low resolution;

[0064] (4) Feature Decoding: A series of upsampling layers and convolutional layers are used to gradually restore the spatial resolution of the feature map to obtain the dense optical flow estimation result as Figure 4 shown;

[0065] Step 3-3 Model Training: The present invention is implemented based on PyTorch. The learning rate is set to 1.25e-4, and the AdamW optimizer is used to iterate 60k times on the training set. The gated recurrent unit performs 32 rounds of optical flow updates. The model is trained on two Titan V graphics cards with a batch size of 16.

[0066] Step 3-4 Model Application: Perform optical flow estimation on adjacent fisheye image frames to obtain pixel-level motion information.

[0067] Step 4. Based on the optical flow estimation results, formulate a region proposal strategy for the feature matching to improve the matching accuracy. Specifically, assume that the feature point positions of adjacent image frames are (x1, y1) and (x2, y2) respectively. Then the motion vector given by the optical flow estimation is (u, v), that is, the horizontal and vertical displacement amounts of a certain point in the image, satisfying the following relationship:

[0068] (x2, y2) = (x1 + u, y1 + v)

[0069] Set the region within a distance of 10 pixels from the center with (x2, y2) as the center as the candidate region. For the feature point with the highest similarity within the candidate feature region, if the similarity is greater than 0.7, it is directly set as a successful match. If not, perform global feature matching again.

[0070] Step 5. Calculate the rough pose transformation result: Use the ORB feature matching result obtained in Step 3 to calculate the preliminary camera pose transformation using the fundamental matrix or the homography matrix. This rough pose estimation process obtains the preliminary pose relationship between two frames by minimizing the feature point projection error.

[0071] Step 6. Abnormal feature point elimination: Analyze the difference between the motion trajectory of each feature point determined by the optical flow estimation and the reprojection position. If the actual motion of a certain feature point is too different from the predicted motion, it indicates that there may be an incorrect match or interference for this point. Specifically, if the motion difference of some feature points exceeds the preset threshold (set to 50%), these points are considered abnormal points and are removed from the matching list.

[0072] Step 7. Local and global optimization: Use the optimized matching point pairs to re-estimate the pose, adopt the bundle adjustment method for local optimization to improve the localization and mapping accuracy, and perform loop detection and global optimization according to whether the image is a key frame. This step requires the following sub-steps: reprojection, local optimization, global optimization:

[0073] Step 7-1 Reprojection: After completing the feature matching and abnormal point elimination, the remaining matching point pairs are used to re-estimate the camera pose. The pose estimation here is mainly based on the optimized matching point pairs and is optimized using the method of minimizing the reprojection error. Specifically, for a feature point p i = (x i , y i ), its three-dimensional space coordinates in the current frame are P i = (X i , Y i , Z i), and its reprojection position in the previous frame is:

[0074] p i ' = K · (R · P i + t)

[0075] where K is the intrinsic matrix of the camera, and p i ' is the image coordinate in the coordinate system of the previous frame after reprojection.

[0076] Step 7-2 Local optimization: Minimizing the reprojection error is achieved by the least squares method. By summing the reprojection errors of all matching points, the optimization objective function is obtained:

[0077]

[0078] where p i ′ is the predicted position obtained through the current pose estimation, and p i is the actually observed matching point.

[0079] Step 7-3 Global optimization: The global optimization after loop detection is used to further reduce the global error and improve the accuracy of the entire map. By constructing an optimization graph that includes all frame poses and map points, and according to the constraints of each node (frame pose and map point) in the graph, global optimization is performed. The goal of the optimization is to minimize the reprojection errors of all frames, making the entire map and pose consistent globally, which can effectively eliminate pose drift, reduce error accumulation, and ensure the consistency and accuracy of the map and pose estimation throughout the trajectory.

[0080] Step Eight, Output: Through the optimized camera pose estimation, the global pose transformation result of the camera during the entire process is output. This result represents the spatial position and orientation transformation of the camera between different time frames, and can reflect the overall movement trajectory of the camera in the environment; According to the optimized 3D map points and camera poses, a complete point cloud map is constructed. The point cloud map is composed of multiple feature points in 3D space, representing the structural information of the environment captured by the camera, such as Figure 5 shown, where the black square represents the camera pose transformation process. This map can be used for subsequent path planning, environmental modeling, visualization, etc. tasks, and at the same time provides an accurate description of the geometric characteristics of the environment.

[0081] The present invention aims to improve the accuracy of simultaneous localization and mapping in a large field of view environment, including: taking the wide-angle images acquired by a binocular fisheye camera and the camera configuration parameters as inputs; preprocessing the images, and using an ORB feature extractor to extract visual feature points from the preprocessed images; applying a deep learning method optimized for fisheye images to perform dense optical flow estimation on adjacent image frames to obtain pixel-level motion information; based on the optical flow estimation results, formulating a region proposal strategy for feature matching, thereby improving the matching accuracy; obtaining a preliminary pose estimation result through ORB feature matching, performing image reprojection using the preliminary pose result, and combining the optical flow estimation results to eliminate outliers; re-estimating the pose using the optimized matching points, and adopting a bundle adjustment method for local optimization to improve the map accuracy; performing loop detection on key frames and executing global optimization to reduce drift and error accumulation; finally outputting the global pose transformation result and the constructed point cloud map. By introducing optical flow estimation to guide feature matching, the present invention significantly improves the matching accuracy in a fisheye camera environment, and effectively eliminates outliers in low-quality matches, making SLAM highly practical and applicable in a wide field of view.

[0082] The innovation of this embodiment lies in:

[0083] A deep learning network built for the characteristics of fisheye images is more suitable for the optical flow estimation task of fisheye images with strong distortion and wide field of view.

[0084] The optical flow estimation technology is introduced to guide the feature matching process. Using dense optical flow estimation to provide pixel-level motion information significantly improves the feature matching accuracy in a fisheye camera environment. Through the optical flow information, the problem of false matching in dynamic scenes can be effectively solved, and high matching stability can be maintained under low-texture conditions.

[0085] After feature matching, the present invention combines the image reprojection and the optical flow estimation results, and effectively eliminates outliers through motion consistency analysis. Compared with traditional methods, the optical flow-guided outlier elimination strategy can more accurately identify incorrect matches, avoiding the interference of low-quality matches on subsequent pose estimation and map construction, thereby improving the robustness and accuracy of the system.

[0086] Experiments have proven that the optical flow estimation-guided fisheye camera SLAM method of the present invention achieves high-precision positioning and mapping in a large field of view environment. Through experimental testing, the average absolute trajectory error of the system in various complex scenarios reaches 0.213 meters, significantly lower than the error level of traditional SLAM systems. In addition, in practical applications, the present invention successfully reduces error accumulation and drift and improves the stability and consistency of the map by introducing optical flow estimation and local optimization. In the comparative test, the SLAM system of the present invention performs better than other existing solutions in terms of feature matching accuracy and positioning accuracy in a fisheye camera environment and can effectively cope with changes in dynamic scenarios. Experiments have proven that the present invention can provide efficient and accurate positioning and mapping results in a wide field of view, providing reliable technical support for tasks such as navigation, robot positioning, and environmental modeling in practical applications.

[0087] Embodiment 2:

[0088] The embodiment of the present invention further provides a fisheye camera SLAM device, including:

[0089] A first processing module for obtaining wide-angle images through a binocular fisheye camera;

[0090] A second processing module for preprocessing the wide-angle images and extracting visual feature points from the preprocessed images using an ORB feature extractor;

[0091] A third processing module for performing dense optical flow estimation on adjacent image frames through a fisheye image optical flow estimation network model according to the visual feature points to obtain pixel-level motion information;

[0092] A fourth processing module for formulating a regional proposal strategy for feature matching based on the optical flow estimation result and obtaining a preliminary pose estimation result through ORB feature matching,

[0093] A fifth processing module for performing image reprojection using the preliminary pose result and removing outliers in combination with the optical flow estimation result;

[0094] A sixth processing module for re-estimating the pose using the optimized matching point pairs and performing local optimization and global optimization;

[0095] A seventh processing module for outputting the global pose transformation result and the constructed point cloud map.

[0096] As an implementation manner of the embodiment of the present invention, the sixth processing module is used to perform local optimization based on the optimized matching point pairs by using the bundle adjustment method, adjust the camera pose and the positions of 3D map points, and minimize the reprojection error in all images.

[0097] As an implementation manner of an embodiment of the present invention, the sixth processing module is used to perform loop detection on key frames and execute global optimization; wherein, when an image is determined to be a key frame, the loop detection algorithm is used to identify whether the current frame has an overlapping area with a certain previous frame. If a loop is detected, the loop information is fed back to the optimization framework for global optimization.

[0098] Embodiment 3:

[0099] An embodiment of the present invention further provides a fisheye camera SLAM system, including: a memory and a processor. A computer program is stored on the memory and run by the processor. When the computer program is run by the processor, it executes the fisheye camera SLAM method.

[0100] Embodiment 4:

[0101] An embodiment of the present invention further provides a storage medium. A computer program is stored on the storage medium. When the computer program runs, it executes the fisheye camera SLAM method.

[0102] The above-described embodiments are only descriptions of the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A fisheye camera SLAM method, characterized in that, Including: Wide-angle images obtained by a binocular fisheye camera; Preprocess the wide-angle images, and use an ORB feature extractor to extract visual feature points from the preprocessed images; According to the visual feature points, use a fisheye image optical flow estimation network model to perform dense optical flow estimation on adjacent image frames to obtain pixel-level motion information; Based on the optical flow estimation results, formulate a region proposal strategy for feature matching, and obtain a preliminary pose estimation result through ORB feature matching; Use the preliminary pose result for image reprojection, and combine the optical flow estimation result to eliminate outliers; Use the optimized matching point pairs to re-estimate the pose, and perform local optimization and global optimization; Output the global pose transformation result and the constructed point cloud map; Apply an optimized deep learning method for fisheye images to perform dense optical flow estimation on adjacent image frames to obtain pixel-level motion information, specifically including: Step 3-1 Data preparation: Use the existing fisheye image optical flow estimation dataset SynWoodScape as the data source, and split it into a training set and a test set according to a ratio of 7:3; Step 3-2 Network model construction: The fisheye image optical flow estimation network consists of an encoder, information fusion, optical flow prediction, and a decoder: (1) Feature encoding: Extract the adjacent frame correlation feature, fisheye image dynamic edge feature, and context feature respectively, as follows: Extract the adjacent frame correlation feature: Concatenate the previous frame I1 and the current frame I2, and use a weight-sharing convolutional neural network N to extract the correlation feature to obtain the dense downsampled features F1 and F2: Represent the correlation Cor of all pixel points by calculating the cosine similarity between the two feature maps; Perform pooling processing on the last two dimensions of Cor with different sizes to obtain the adjacent frame correlation feature; Extract the dynamic edge features and context features of the fish-eye image: Convert I1 and I2 into grayscale images, and obtain the motion relationship I between two frames by calculating the absolute frame difference D ; Determine the position of the strongly distorted edge region according to the polynomial model and the image size; After multiple rounds of denoising, connected component analysis, and opening and closing operations, obtain the edge dynamic object mask I after removing noise points and most of the background mask ; Use the deformable residual network to perform feature encoding on I mask and I2 respectively to obtain the dynamic edge feature F motion and the context feature F context ; (2) Information fusion: Use a convolutional block attention mechanism module to fuse the context feature and the dynamic edge feature to obtain a dynamic context enhanced feature; (3) Optical flow prediction: Fuse the above features through a gated recurrent unit, iteratively update the optical flow offset, and obtain the optical flow estimation result at low resolution; (4) Feature decoding: Through the upsampling layer and the convolutional layer, gradually restore the spatial resolution of the feature map to obtain the dense optical flow estimation result; Step 3-3 Model training: Set the learning rate to 1.25e-4, use the AdamW optimizer to iterate 60k times on the training set, and perform 32 rounds of optical flow updates for the gated recurrent unit; Step 3-4 Model application: Perform optical flow estimation on adjacent fisheye image frames to obtain pixel-level motion information.

2. The fish-eye camera SLAM method according to claim 1, wherein, Based on the optimized matching point pairs, use the bundle adjustment method for local optimization, adjust the camera pose and the positions of the 3D map points, and minimize the reprojection error in all images.

3. The fish-eye camera SLAM method according to claim 2, characterized in that, Perform loop detection on the key frames and execute global optimization; Among them, when an image is determined to be a key frame, use the loop detection algorithm to identify whether the current frame has an overlapping area with a previous frame. If a loop is detected, feedback the loop information to the optimization framework for global optimization.

4. A fisheye camera SLAM device for implementing the fisheye camera SLAM method according to claim 1, characterized in that, Including: A first processing module for wide-angle images obtained by a binocular fisheye camera; The second processing module is used to preprocess the wide-angle image and extract visual feature points from the preprocessed image using an ORB feature extractor; The third processing module is used to perform dense optical flow estimation on adjacent image frames based on visual feature points through a fisheye image optical flow estimation network model to obtain pixel-level motion information; The fourth processing module is used to formulate a region proposal strategy for feature matching based on the optical flow estimation results, and obtain preliminary pose estimation results through ORB feature matching. The fifth processing module is used to reproject the image using the preliminary pose results and remove abnormal points in combination with the optical flow estimation results; A sixth processing module is used to use the optimized matching point pairs to re-estimate the pose and perform local optimization and global optimization; The seventh processing module is used to output the global pose transformation results and the constructed point cloud map.

5. The fish-eye camera SLAM device according to claim 4, characterized in that, The sixth processing module is used to perform local optimization based on the optimized matching point pairs using the bundle optimization method to adjust the camera pose and the position of the 3D map points to minimize the reprojection error in all images.

6. The fish-eye camera SLAM device according to claim 5, characterized in that The sixth processing module is used to perform loop detection on key frames and perform global optimization. When an image is determined to be a key frame, a loop detection algorithm is used to identify whether the current frame has an overlapping area with a previous frame. If a loop is detected, the loop information is fed back to the optimization framework for global optimization.

7. A fisheye camera SLAM system, characterized in that, include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the fisheye camera SLAM method according to any one of claims 1 to 3 is executed.

8. A storage medium, characterized in that, The storage medium stores a computer program, which executes the fisheye camera SLAM method according to any one of claims 1 to 3 when running.

Images