Fisheye camera SLAM method, device and system, and storage medium

Abstract

The invention discloses a fisheye camera SLAM method, device and system and a storage medium. The fisheye camera SLAM method comprises the steps that a wide-angle image is acquired through a binocular fisheye camera; preprocessing the wide-angle image, and extracting visual feature points from the preprocessed image by using an ORB feature extractor; according to the visual feature points, dense optical flow estimation is carried out on adjacent image frames through a fisheye image optical flow estimation network model so as to acquire pixel-level motion information; on the basis of an optical flow estimation result, a region suggestion strategy is made for feature matching, a preliminary pose estimation result is obtained through ORB feature matching, image re-projection is performed by using the preliminary pose result, and abnormal points are eliminated in combination with the optical flow estimation result; re-estimating the pose by using the optimized matching point pair, and carrying out local optimization and global optimization; and outputting a global pose transformation result and the constructed point cloud map. By adopting the technical scheme of the invention, the matching precision in a fisheye camera environment is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a fisheye camera SLAM method and device, a system, and a storage medium. Background Art

[0002] Simultaneous Localization and Mapping (SLAM) is a technology that enables robots or smart devices to simultaneously locate and map in unknown environments. It is widely used in robot navigation, augmented reality, autonomous driving and other fields. Traditional SLAM methods mostly rely on ordinary pinhole cameras for feature extraction and matching. However, since fisheye cameras have a wide-angle field of view and strong environmental perception capabilities, they can provide more extensive perception information in complex or narrow environments. Therefore, the application of fisheye cameras in SLAM has gradually become a research hotspot. The technologies required mainly include computer vision, deep learning, video understanding, computational geometry and optimization technology.

[0003] Computer vision technology: In visual SLAM, the computer vision algorithm first extracts stable feature points from the images captured by the camera. Common feature extraction methods such as ORB, SIFT and SURF are used to match these feature points between adjacent frames to calculate the relative position and posture of the camera. Through the matching of these feature points and the application of geometric models, visual SLAM can estimate the movement trajectory of the camera in space. At the same time, computer vision also further improves the accuracy of pose estimation and map construction through image reprojection and error optimization.

[0004] Deep learning technology: mainly reflected in feature extraction, optical flow estimation, depth map prediction and optimization. Traditional visual SLAM relies on manually designed feature extraction methods, which may not perform well in complex environments and dynamic scenes. Deep learning can automatically learn useful features in images, thereby improving the robustness and accuracy of feature extraction. For example, convolutional neural networks (CNNs) can be used to extract high-level semantic features from images, allowing SLAM systems to identify and track more details in complex environments. Deep learning also plays an important role in optical flow estimation, by training deep models to accurately estimate pixel-level motion information, especially in dense scenes.

[0005] Video understanding technology: Its application in visual SLAM is mainly reflected in dynamic scene analysis, motion estimation and multi-frame information fusion. By analyzing the timing information in continuous image sequences, video understanding technology can effectively capture and understand the motion patterns of objects, thereby providing more accurate positioning and mapping capabilities for visual SLAM. Specifically, video understanding technology can obtain pixel-level motion information of objects or cameras in images through optical flow estimation, providing strong support for feature matching. In terms of multi-frame information fusion, video understanding technology can effectively integrate visual data between different time frames, and improve the stability and accuracy of pose estimation through timing modeling and semantic understanding. Especially in dynamic scenes, video understanding can help the SLAM system distinguish between static backgrounds and dynamic objects, and reduce the interference of dynamic objects on map construction and positioning.

[0006] Computational geometry and optimization technology: Its application in visual SLAM is mainly reflected in pose estimation, map optimization and error minimization. The SLAM system processes feature point matching and camera pose calculation in the image through computational geometry methods, and uses geometric models such as basic matrix and homography matrix to estimate the relative motion of the camera. In addition, computational geometry also plays a key role in 3D reconstruction, helping to build accurate 3D maps. Optimization technology, especially bundle adjustment, optimizes the camera pose and the position of map points by minimizing image reprojection errors, thereby improving positioning accuracy and map quality. Loop detection and global optimization use optimization methods to reduce error accumulation and ensure the stability and robustness of the SLAM system during long-term operation. Computational geometry and optimization technology enable visual SLAM to achieve high-precision positioning and mapping in complex environments.

[0007] Existing visual SLAM methods often face problems of insufficient matching accuracy and error accumulation in large field of view environments, especially when using fisheye cameras, wide-angle images are prone to large distortion, resulting in poor performance of traditional feature extraction and matching algorithms in complex scenes. In addition, since large field of view images contain more dynamic elements, existing technologies are often unable to effectively remove these outliers, resulting in map error accumulation and positioning drift. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a fisheye camera SLAM method and device, system, and storage medium, which improves the matching accuracy by introducing optical flow estimation to guide feature matching, and effectively eliminates abnormal points in low-quality matching, thereby significantly improving the accuracy of pose estimation and map construction, ensuring efficient positioning and mapping in a large field of view and dynamic environment.

[0009] To achieve the above object, the present invention adopts the following technical solution:

[0010] A fisheye camera SLAM method, comprising:

[0011] Wide-angle images acquired by binocular fisheye cameras;

[0012] Preprocess the wide-angle image, and use the ORB feature extractor to extract visual feature points from the preprocessed image;

[0013] Based on the visual feature points, the fisheye image optical flow estimation network model is used to perform dense optical flow estimation on adjacent image frames to obtain pixel-level motion information.

[0014] Based on the optical flow estimation results, a region proposal strategy is formulated for feature matching, and a preliminary pose estimation result is obtained through ORB feature matching.

[0015] Use the preliminary pose results to reproject the image, and remove abnormal points in combination with the optical flow estimation results;

[0016] Use the optimized matching point pairs to re-estimate the pose and perform local and global optimization;

[0017] Output the global pose transformation results and the constructed point cloud map.

[0018] Preferably, based on the optimized matching point pairs, a bundle optimization method is used for local optimization to adjust the camera pose and the position of the 3D map points to minimize the reprojection error in all images.

[0019] Preferably, loop detection is performed on key frames and global optimization is performed; when an image is determined to be a key frame, a loop detection algorithm is used to identify whether the current frame has an overlapping area with a previous frame. If a loop is detected, the loop information is fed back to the optimization framework for global optimization.

[0020] The present invention also provides a fisheye camera SLAM device, comprising:

[0021] A first processing module, for obtaining a wide-angle image through a binocular fisheye camera;

[0022] The second processing module is used to preprocess the wide-angle image and extract visual feature points from the preprocessed image using an ORB feature extractor;

[0023] The third processing module is used to perform dense optical flow estimation on adjacent image frames according to the visual feature points through a fisheye image optical flow estimation network model to obtain pixel-level motion information;

[0024] The fourth processing module is used to formulate a region proposal strategy for feature matching based on the optical flow estimation results, and obtain a preliminary pose estimation result through ORB feature matching.

[0025] The fifth processing module is used to perform image reprojection using the preliminary pose results and remove abnormal points in combination with the optical flow estimation results;

[0026] A sixth processing module is used to use the optimized matching point pairs to re-estimate the pose and perform local optimization and global optimization;

[0027] The seventh processing module is used to output the global pose transformation results and the constructed point cloud map.

[0028] Preferably, the sixth processing module is used to perform local optimization based on the optimized matching point pairs using a bundle optimization method to adjust the camera pose and the position of the 3D map points to minimize the reprojection error in all images.

[0029] Preferably, the sixth processing module is used to perform loop detection on key frames and perform global optimization; wherein, when an image is determined to be a key frame, a loop detection algorithm is used to identify whether the current frame has an overlapping area with a previous frame. If a loop is detected, the loop information is fed back to the optimization framework for global optimization.

[0030] An embodiment of the present invention further provides a fisheye camera SLAM system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a fisheye camera SLAM method when executed by the processor.

[0031] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes a fisheye camera SLAM method when running.

[0032] Compared with the prior art, the present invention has the following advantages and technical effects:

[0033] 1. The present invention effectively improves the matching accuracy of fisheye cameras in a large field of view environment by introducing a strategy of optical flow estimation to guide feature matching.

[0034] 2. Effectively identify and remove abnormal points through difference analysis of reprojection and optical flow estimation results. Traditional SLAM systems often have erroneous matching points during feature matching, which is more serious in dynamic environments, affecting positioning accuracy and map quality. By analyzing the differences in optical flow estimation of matching points, the present invention can detect abnormal matching points in advance and remove them in time, avoiding the impact of abnormal points on subsequent pose estimation and map construction, thereby improving the robustness and reliability of the SLAM system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 This is a flow chart of the fisheye camera SLAM method according to an embodiment of the present invention;

[0037] Figure 2 This is a data processing schematic diagram of a fisheye camera SLAM method according to an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of feature matching results according to an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of an optical flow estimation result according to an embodiment of the present invention;

[0040] Figure 5 The figure is a schematic diagram showing the experimental results of an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] Embodiment 1:

[0044] like Figure 1 , 2 As shown, this embodiment provides a fisheye camera SLAM method guided by optical flow estimation, comprising the following steps:

[0045] Step 1, data acquisition: The wide-angle image acquired by the binocular fisheye camera and the camera configuration parameters are used as input, specifically including a binocular fisheye camera with a horizontal field of view angle range of 170 degrees to 180 degrees and a vertical field of view angle range of 140 degrees to 160 degrees, and then the camera internal and external parameter matrix is ​​obtained through the calibration process. Use the binocular fisheye camera to collect images.

[0046] Step 2: Image preprocessing: Preprocess the image, including grayscale processing, image pyramid construction, and extract visual feature points from the preprocessed image using the ORB feature extractor. The matching results are as follows: Figure 3 As shown;

[0047] Step 3: Dense optical flow estimation: Use a deep learning method optimized for fisheye images to perform dense optical flow estimation on adjacent image frames to obtain pixel-level motion information. This step requires the following steps: data preparation, network model construction, model training, and model application.

[0048] Step 3-1 Data preparation: Use the existing fisheye image optical flow estimation dataset SynWoodScape as the data source and split it into a training set and a test set in a ratio of 7:3.

[0049] Step 3-2 Network model construction: The fisheye image optical flow estimation network consists of an encoder, information fusion, optical flow prediction, and a decoder:

[0050] (1) Feature encoding: extract the adjacent frame correlation features, fisheye image dynamic edge features and context features respectively, as follows:

[0051] Correlation features of adjacent frames: First, concatenate the previous frame I1 and the current frame I2, and then use a weight-shared convolutional neural network N to extract correlation features to obtain dense downsampling features F1 and F2:

[0052] F1=N(I1),F2=N(I2)

[0053] Where D represents the feature dimension, W and H represent the width and height respectively. The correlation of all pixels is expressed by calculating the cosine similarity between two feature maps:

[0054]

[0055] Among them, Cor ijkl Each element in represents the correlation between the coordinates P1 = (i, j) in F1 and the coordinates P1 = (k, l) in F2. Finally, the last two dimensions of Cor are pooled with different sizes to obtain:

[0056] Cor a ={Cor1,Cor2,Cor3,Cor4}

[0057] in

[0058] Dynamic edge features and context features of fisheye images: The present invention obtains the dynamic object mask of the edge area by designing a graphics processing method and uses it as network prior information. First, I1 and I2 are converted into grayscale images, and the motion relationship between the two frames I is obtained by calculating the absolute frame difference. D Then, the location of the strong edge distortion area is determined based on the polynomial model and the image size. Specifically, a coordinate system is established with the center of the image as the origin, and the major axis a and minor axis b of the ellipse area are determined based on the degree of distortion of the pixels on the x and y axes. The area outside the ellipse is considered as the strong edge distortion area. The calculation formula is as follows:

[0059]

[0060] Where X t Indicates the number of pixels from the origin in the positive direction of the axis, X t Represents Y c The number of pixels from the origin after the polynomial model correction, Y c and Y t Similarly, τ represents the deviation rate threshold. Finally, after multiple rounds of denoising, connected domain analysis, and opening and closing operations, the edge dynamic object mask I is obtained after removing noise points and most backgrounds. mask .

[0061] Use deformable residual network to mask And I2 are used for feature encoding to obtain the dynamic edge feature F motion and context feature F context ;

[0062] (2) Information fusion: Use the convolutional block attention mechanism module to fuse context features and dynamic edge features to obtain dynamic context enhanced features;

[0063] (3) Optical flow prediction: The above features are integrated through the gated recurrent unit, and the optical flow offset is iteratively updated to obtain the optical flow estimation result at low resolution.

[0064] (4) Feature decoding: A series of upsampling layers and convolutional layers are used to gradually restore the spatial resolution of the feature map to obtain the dense optical flow estimation result. Figure 4 As shown;

[0065] Step 3-3 Model training: The present invention is implemented based on PyTorch, the learning rate is set to 1.25e-4, the AdamW optimizer is used to iterate 60k times on the training set, and the gated recurrent unit performs 32 rounds of optical flow updates. The whole model is trained on two TitanV graphics cards with a batch size of 16.

[0066] Step 3-4 Model application: Perform optical flow estimation on adjacent fisheye image frames to obtain pixel-level motion information.

[0067] Step 4: Based on the optical flow estimation result, a region proposal strategy is formulated for the feature matching to improve the matching accuracy. Specifically, assuming that the positions of the feature points of adjacent image frames are (x1, y1) and (x2, y2), respectively, the motion vector given by the optical flow estimation is (u, v), that is, the horizontal and vertical displacement of a point in the image, which satisfies the following relationship:

[0068] (x2,y2)=(x1+u,y1+v)

[0069] The area within 10 pixels of (x2, y2) as the center is set as the candidate area. If the feature point with the highest similarity in the candidate feature area is greater than 0.7, it is directly set as a successful match. If it does not exist, global feature matching is performed again.

[0070] Step 5: Calculate the rough pose transformation result: Using the ORB feature matching result obtained in step 3, use the basic matrix or homography matrix to calculate the preliminary camera pose transformation. The rough pose estimation process obtains the preliminary pose relationship between the two frames by minimizing the feature point projection error.

[0071] Step 6. Abnormal feature point removal: Analyze the difference between the motion trajectory determined by the optical flow estimation and the reprojected position of each feature point. If the actual motion of a feature point differs too much from the predicted motion, it indicates that the point may be mismatched or interfered with. Specifically, if the motion difference of some feature points exceeds the preset threshold (set to 50%), these points are considered abnormal points and are removed from the matching list.

[0072] Step 7: Local and global optimization: Use the optimized matching point pairs to re-estimate the pose, use the bundle optimization method to perform local optimization, improve the positioning and mapping accuracy, and perform loop detection and global optimization based on whether the image is a key frame. This step requires the following steps: reprojection, local optimization, and global optimization:

[0073] Step 7-1 Reprojection: After completing feature matching and outlier removal, the remaining matching point pairs are used to re-estimate the camera pose. The pose estimation here is mainly based on the optimized matching point pairs, which are optimized by minimizing the reprojection error. Specifically, for a feature point p i =(x i ,y i ), whose three-dimensional space coordinates in the current frame are P i =(X i ,Y i ,Z i), whose reprojected position in the previous frame is:

[0074] p i '=K·(R·P i +t)

[0075] Among them, K is the intrinsic parameter matrix of the camera, p i ' is the image coordinates in the previous frame coordinate system after reprojection.

[0076] Step 7-2 Local Optimization: Minimizing the reprojection error is achieved through the least squares method. By summing the reprojection errors of all matching points, the optimization objective function is obtained:

[0077]

[0078] Among them, p i ′ is the predicted position estimated by the current pose, p i is the actual observed matching point.

[0079] Step 7-3 Global Optimization: Global optimization after loop detection is used to further reduce global errors and improve the accuracy of the entire map. Global optimization is performed by constructing an optimization graph containing all frame poses and map points, and based on the constraints of each node (frame pose and map point) in the graph. The goal of the optimization is to minimize the reprojection error of all frames so that the entire map and pose are consistent globally, which can effectively eliminate pose drift, reduce error accumulation, and ensure the consistency and accuracy of the map and pose estimation over the entire trajectory.

[0080] Step 8. Output: Through the optimized camera pose estimation, the global pose transformation result of the camera in the whole process is output. This result represents the spatial position and orientation transformation of the camera between different time frames, which can reflect the overall motion trajectory of the camera in the environment; according to the optimized 3D map points and camera pose, a complete point cloud map is constructed. The point cloud map is composed of multiple feature points in 3D space, which represents the structural information of the environment photographed by the camera, such as Figure 5 As shown in Figure 1, the black box represents the camera pose transformation process. This map can be used for subsequent path planning, environment modeling, and visualization tasks, while providing an accurate description of the environment's geometric characteristics.

[0081] The present invention aims to improve the accuracy of instant positioning and mapping in a large field of view environment, including: taking a wide-angle image obtained by a binocular fisheye camera and camera configuration parameters as input; preprocessing the image, and extracting visual feature points from the preprocessed image using an ORB feature extractor; applying a deep learning method optimized for fisheye images to perform dense optical flow estimation on adjacent image frames to obtain pixel-level motion information; formulating a region suggestion strategy for feature matching based on the optical flow estimation result, thereby improving the matching accuracy; obtaining a preliminary pose estimation result through ORB feature matching, using the preliminary pose result to reproject the image, and removing abnormal points in combination with the optical flow estimation result; re-estimating the pose using the optimized matching point pairs, and using a bundle optimization method to perform local optimization to improve the map accuracy; performing loop detection on key frames and performing global optimization to reduce drift and error accumulation; and finally outputting the global pose transformation result and the constructed point cloud map. The present invention significantly improves the matching accuracy in the fisheye camera environment by introducing optical flow estimation to guide feature matching, and effectively removes abnormal points in low-quality matching, making SLAM more practical and promising in a wide field of view.

[0082] The innovation of this embodiment is:

[0083] The deep learning network built based on the characteristics of fisheye images is more suitable for the task of optical flow estimation of fisheye images with strong distortion and wide field of view.

[0084] Optical flow estimation technology is introduced to guide the feature matching process. Dense optical flow estimation is used to provide pixel-level motion information, which significantly improves the feature matching accuracy in the fisheye camera environment. Through optical flow information, it can effectively solve the mismatching problem in dynamic scenes and maintain high matching stability under low-texture conditions.

[0085] After feature matching, the present invention combines image reprojection and optical flow estimation results to effectively remove outliers through motion consistency analysis. Compared with traditional methods, the optical flow-guided removal strategy can more accurately identify incorrect matches, avoiding the interference of low-quality matches on subsequent pose estimation and map construction, thereby improving the robustness and accuracy of the system.

[0086] Experiments have shown that the optical flow estimation-guided fisheye camera SLAM method of the present invention achieves high-precision positioning and mapping in a large field of view environment. Through experimental tests, the average absolute trajectory error of the system in a variety of complex scenes reached 0.213 meters, which is significantly lower than the error level of traditional SLAM systems. In addition, in practical applications, the present invention successfully reduced error accumulation and drift by introducing optical flow estimation and local optimization, and improved the stability and consistency of the map. In comparative tests, the SLAM system of the present invention performs better in feature matching accuracy and positioning accuracy in a fisheye camera environment than other existing solutions, and can effectively cope with changes in dynamic scenes. Experiments have shown that the present invention can provide efficient and accurate positioning and mapping results in a wide field of view, providing reliable technical support for tasks such as navigation, robot positioning and environmental modeling in practical applications.

[0087] Embodiment 2:

[0088] The embodiment of the present invention further provides a fisheye camera SLAM device, comprising:

[0089] A first processing module, for obtaining a wide-angle image through a binocular fisheye camera;

[0090] The second processing module is used to preprocess the wide-angle image and extract visual feature points from the preprocessed image using an ORB feature extractor;

[0091] The third processing module is used to perform dense optical flow estimation on adjacent image frames according to the visual feature points through a fisheye image optical flow estimation network model to obtain pixel-level motion information;

[0092] The fourth processing module is used to formulate a region proposal strategy for feature matching based on the optical flow estimation results, and obtain a preliminary pose estimation result through ORB feature matching.

[0093] The fifth processing module is used to perform image reprojection using the preliminary pose results and remove abnormal points in combination with the optical flow estimation results;

[0094] A sixth processing module is used to use the optimized matching point pairs to re-estimate the pose and perform local optimization and global optimization;

[0095] The seventh processing module is used to output the global pose transformation results and the constructed point cloud map.

[0096] As an implementation of an embodiment of the present invention, the sixth processing module is used to perform local optimization based on the optimized matching point pairs using a bundle optimization method to adjust the camera pose and the position of the 3D map points to minimize the reprojection error in all images.

[0097] As an implementation method of an embodiment of the present invention, the sixth processing module is used to perform loop detection on key frames and perform global optimization; wherein, when an image is determined to be a key frame, a loop detection algorithm is used to identify whether the current frame has an overlapping area with a previous frame. If a loop is detected, the loop information is fed back to the optimization framework for global optimization.

[0098] Embodiment 3:

[0099] An embodiment of the present invention further provides a fisheye camera SLAM system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a fisheye camera SLAM method when executed by the processor.

[0100] Embodiment 4:

[0101] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes a fisheye camera SLAM method when running.

[0102] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A fisheye camera SLAM method, characterized in that: include: Wide-angle images acquired by binocular fisheye cameras; Preprocess the wide-angle image, and use the ORB feature extractor to extract visual feature points from the preprocessed image; Based on the visual feature points, the fisheye image optical flow estimation network model is used to perform dense optical flow estimation on adjacent image frames to obtain pixel-level motion information. Based on the optical flow estimation results, a region proposal strategy is formulated for feature matching, and a preliminary pose estimation result is obtained through ORB feature matching. Use the preliminary pose results to reproject the image, and remove abnormal points in combination with the optical flow estimation results; Use the optimized matching point pairs to re-estimate the pose and perform local and global optimization; Output the global pose transformation results and the constructed point cloud map.

2. The fisheye camera SLAM method as claimed in claim 1, characterized in that: Based on the optimized matching point pairs, the bundle optimization method is used for local optimization to adjust the camera pose and the position of the 3D map points to minimize the reprojection error in all images.

3. The fisheye camera SLAM method as claimed in claim 2, characterized in that: Perform loop detection on key frames and perform global optimization. When an image is determined to be a key frame, a loop detection algorithm is used to identify whether the current frame has an overlapping area with a previous frame. If a loop is detected, the loop information is fed back to the optimization framework for global optimization.

4. A fisheye camera SLAM device, characterized in that: include: A first processing module, for obtaining a wide-angle image through a binocular fisheye camera; The second processing module is used to preprocess the wide-angle image and extract visual feature points from the preprocessed image using an ORB feature extractor; The third processing module is used to perform dense optical flow estimation on adjacent image frames according to the visual feature points through a fisheye image optical flow estimation network model to obtain pixel-level motion information; The fourth processing module is used to formulate a region proposal strategy for feature matching based on the optical flow estimation results, and obtain preliminary pose estimation results through ORB feature matching. The fifth processing module is used to perform image reprojection using the preliminary pose results and remove abnormal points in combination with the optical flow estimation results; A sixth processing module is used to use the optimized matching point pairs to re-estimate the pose and perform local optimization and global optimization; The seventh processing module is used to output the global pose transformation results and the constructed point cloud map.

5. The fisheye camera SLAM device as claimed in claim 4, characterized in that: The sixth processing module is used to perform local optimization based on the optimized matching point pairs by using the bundle optimization method to adjust the camera pose and the position of the 3D map points to minimize the reprojection errors in all images.

6. The fisheye camera SLAM device as claimed in claim 5, characterized in that: The sixth processing module is used to perform loop detection on key frames and perform global optimization; when an image is determined to be a key frame, a loop detection algorithm is used to identify whether the current frame has an overlapping area with a previous frame. If a loop is detected, the loop information is fed back to the optimization framework for global optimization.

7. A fisheye camera SLAM system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the fisheye camera SLAM method according to any one of claims 1 to 3 is executed.

8. A storage medium, characterized in that: The storage medium stores a computer program, which executes the fisheye camera SLAM method according to any one of claims 1 to 3 when running.

Images