Panoramic ring lens based visual slam system and method

Abstract

The application discloses a kind of vision SLAM systems and methods based on panoramic annular lens, the system includes: acquisition unit, by the camera of the panoramic annular lens is acquired image sequence at a certain frame rate after;Identification unit, by analyzing the result after image processing, then the scene features of the annular effective area of PAL image are identified and extracted, and interframe matching is carried out;Positioning unit, corresponding tracking model is selected to estimate pose and is optimized;Mapping unit, by acquisition unit, positioning unit draws the driving trajectory of the motion carrier where the camera is located, and the spatial coordinates of feature points in the environment are calculated to reconstruct the scene map.The panoramic annular lens is used to obtain more scene features, and the positioning accuracy is effectively improved by screening the features of the effective area and using adaptive threshold to remove points with large errors.

Description

TECHNICAL FIELD

[0001] The present application relates to a panoramic annular lens-based visual SLAM (Simultaneous Localization and Mapping) system and method. BACKGROUND

[0002] With the rapid development of mobile robots, self-driving technology and augmented reality technology, simultaneous localization and mapping (SLAM) as a core module has become a research hotspot in the industry, which can realize the positioning and mapping function of mobile robots (or mobile devices) in an unknown environment. Compared with using a laser radar, using a visual sensor has the advantages of simple model, large amount of information, low cost, easy installation and low power consumption, but it cannot be ignored that it is easily affected by the environment as a passive sensor, such as changes in light, dust and fog. Therefore, improving the stability and accuracy of the visual SLAM system has become an important work in this field.

[0003] Most traditional visual SLAMs are based on traditional pinhole cameras, and the limited field of view angle will cause the camera to move quickly, rotate or exist dynamic objects in the environment, which cannot be positioned and mapped correctly. Therefore, people consider introducing a large field of view camera into the application of SLAM. The distortion of the fisheye lens is difficult to control at the edge of the field of view, which will cause the image quality to decrease. The multi-camera panoramic imaging system synthesizes a panoramic view by splicing images captured by several cameras at the same time, which may result in a large and expensive system as a whole. In addition, the multi-camera system must pay attention to the synchronization and data fusion between different cameras, which will increase the computational cost. The mirror used in the catadioptric camera is usually large and irregular, which is difficult to manufacture and requires high-precision alignment, which hinders the widespread use of such catadioptric systems. SUMMARY

[0004] In view of the above problems, in order to overcome the defects of the prior art, the present application provides a panoramic annular lens-based visual SLAM system and method.

[0005] The present application solves the above technical problems by the following technical solutions: a panoramic annular lens-based visual SLAM system, characterized in that it comprises:

[0006] An acquisition unit acquires an image sequence at a certain frame rate through a camera carrying the panoramic annular lens, and then performs some preliminary image processing;

[0007] The recognition unit recognizes and extracts the scene features of the annular effective area of the PAL image by analyzing the processed image, and performs inter-frame matching.

[0008] The positioning unit estimates the pose and performs optimization by selecting the corresponding tracking model according to the data obtained by the acquisition unit and the recognition unit, so as to determine the positioning information of the motion carrier in the environment.

[0009] The mapping unit draws the driving trajectory of the motion carrier and calculates the spatial coordinates of the feature points in the environment to reconstruct the scene map.

[0010] Preferably, the acquisition unit comprises an image acquisition module and an image processing module, the image acquisition module acquires the image sequence, and the image processing module processes the image acquired by the image acquisition module, and the processing mode comprises gamma correction and highlight suppression.

[0011] Preferably, the recognition unit comprises a blind area removal module, a feature extraction module and a feature matching module, the blind area removal module removes the blind area to avoid extracting and matching feature points in the area; the feature extraction module extracts the feature points in the effective area; and the feature matching module matches the extracted feature points.

[0012] Preferably, the positioning unit comprises an initialization module, a feature tracking module and a pose optimization module, the initialization module initializes the system to obtain the initial pose and map points, the feature tracking module estimates the pose by selecting the corresponding tracking model according to the data obtained by the acquisition unit and the recognition unit, and the pose optimization module optimizes the pose to perform the positioning process of the motion carrier.

[0013] Preferably, the mapping unit comprises a trajectory generation module and a map construction module, the map construction module calculates the spatial coordinates of the feature points in the environment and continuously updates and optimizes, and reconstructs the scene map, the scene map matches the spatial coordinate system established by the panoramic annular lens-based visual SLAM system, and the trajectory generation module draws the driving trajectory of the motion carrier in the coordinate system.

[0014] The application also provides a panoramic annular lens-based visual SLAM method, which comprises the following steps:

[0015] Step one, initialization;

[0016] Step two, after initialization, the panoramic annular lens-based visual SLAM system will perform a tracking process by analyzing the features of the current frame effective position and obtaining a rough value of the camera pose using a corresponding tracking model, then matching the feature points of the current frame with the local map points, further optimizing the current pose using bundle adjustment, and then calculating the number of inliers that meet the requirements, if the number is sufficient, it is considered as a successful tracking, and step three is performed, otherwise, relocalization will be performed according to the next panoramic image;

[0017] Step three, if the tracking in step two is successful, it is determined whether the current frame is a key frame, if yes, it is inserted into the key frame sequence and step four is performed, if not, it is not inserted into the key frame sequence;

[0018] Step four, in the map construction thread, the map points are updated according to the co-visibility relationship between the current key frame and the existing key frames, redundant key frames are removed, and bundle adjustment is performed to further update the map points and the linking relationship between the map points and the corresponding key frames;

[0019] Step five, in the loop closure detection thread, the most matching point number of the loop key frame with the current key frame is obtained using continuity verification, and the matching feature points of the current key frame and the loop key frame are searched based on the bag-of-words model, if the number of the matching feature points is sufficient, it is determined that a loop is detected, and more accurate map points and camera poses are obtained after subsequent optimization.

[0020] Preferably, the step one comprises the following steps:

[0021] Step eleven, the camera carrying the panoramic annular lens is installed on a moving carrier, and the orientation of the camera is adjusted; the orientation of the camera when installed depends on the specific use environment;

[0022] Step twelve, after the camera is installed and adjusted, calibration is performed to obtain the required parameters of the camera model, and then the corresponding projection model and inverse projection model, i.e. the transformation relationship between the pixel coordinate system and the camera coordinate system, are established;

[0023] Step thirteen, the camera carrying the panoramic annular lens is used to collect image sequences at a certain frame rate, and the image sequences are sequentially input into the panoramic annular lens-based visual SLAM system;

[0024] Step fourteen, the panoramic annular lens-based visual SLAM system extracts and matches feature points from the effective area of the first input several panoramic pictures, and calculates the pose between two frames according to the matching;

[0025] Step fifteen, the adaptive threshold is a threshold used for screening inliers in the initialization process of the panoramic annular lens based visual SLAM system, and the size of the threshold depends on the distance from the corresponding feature point to the optical center.

[0026] The positive progress effect of the present application is that the panoramic annular lens of the present application can realize 360-degree imaging without splicing multiple cameras, so that the entire panoramic annular lens based visual SLAM system is more portable and efficient, and has wide application prospects in the fields of automatic driving and intelligent mobile robots. The present application adopts a panoramic annular lens to obtain more scene features, and effectively improves the positioning accuracy by screening the features in the effective area and using an adaptive threshold to remove points with large errors, and has wide application prospects in the fields of mobile robots and automatic driving. The present application has good robustness in fast cornering and other areas, and can effectively identify the closed loop and perform global optimization even in reverse motion, and does not need to perform unwrapping of annular images, which can ensure the real-time performance of the algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The figure is a principle block diagram of the panoramic annular lens based visual SLAM system of the present application.

[0028] Figure 2 The figure is a flowchart of the panoramic annular lens based visual SLAM method of the present application.

[0029] Figure 3 The figure is an optical path imaging schematic diagram of the camera carrying the panoramic annular lens in the present application.

[0030] Figure 4 The figure is a schematic diagram of tracking map points. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments.

[0032] Compared with these fisheye lenses, catadioptric cameras and multi-camera panoramic imaging systems, the Panoramic Annular Lens (PAL) structure is compact and can effectively reduce stray light. It has the advantages of large field of view, low distortion and high imaging quality. Through special optical design, as much lateral light as possible is collected into the camera. In addition, the annular image removes the upper and lower parts of the spherical image, focusing more on the middle part of the image containing important location visual clues. Since PAL can project a 360° field of view onto a planar annular image at one time without rotating the lens, even if the camera moves and rotates quickly, it can ensure that there is enough overlap between adjacent frames. Therefore, based on the camera equipped with the panoramic annular lens, the present application designs a panoramic annular lens-based visual SLAM method and system, which uses a simple projection formula to process a single panoramic view, and its compact structure and reasonable calculation cost ensure the practical application of the panoramic visual SLAM technology in automatic driving, augmented reality and intelligent robots.

[0033] As shown in Figure 1 , the panoramic annular lens-based visual SLAM system of the present application comprises:

[0034] An acquisition unit acquires an image sequence at a certain frame rate through a camera equipped with the panoramic annular lens, and then performs some preliminary image processing.

[0035] An identification unit identifies and extracts the scene features of the annular effective area of the PAL image by analyzing the results of image processing, and performs inter-frame matching.

[0036] A positioning unit estimates the pose and performs optimization by selecting a corresponding tracking model based on the data obtained by the acquisition unit and the identification unit, so as to determine the positioning information of the motion carrier in the environment.

[0037] A mapping unit draws the driving trajectory of the motion carrier of the camera and calculates the spatial coordinates of the feature points in the environment to reconstruct a scene map.

[0038] The acquisition unit comprises an image acquisition module and an image processing module. The image acquisition module acquires an image sequence, and the image processing module processes the images acquired by the image acquisition module. The processing methods include gamma correction, high light suppression, etc.

[0039] The identification unit comprises a blind area removal module, a feature extraction module and a feature matching module. The blind area removal module identifies the blind area to avoid extracting and matching feature points in this area. The feature extraction module extracts feature points in the effective area, and the feature matching module matches the extracted feature points.

[0040] The positioning unit comprises an initialization module, a feature tracking module and a pose optimization module, the initialization module initializes the system to obtain an initial pose and a map point, the feature tracking module selects a corresponding tracking model according to the data obtained by the acquisition unit and the recognition unit to estimate the pose, and the pose optimization module optimizes the pose, so that the positioning process of the motion carrier where the camera is located is performed.

[0041] The mapping unit comprises a trajectory generation module and a map construction module, the map construction module calculates the spatial coordinates of feature points in the environment and continuously updates and optimizes the spatial coordinates, and reconstructs a scene map, the scene map matches a spatial coordinate system established by the panoramic annular lens-based visual SLAM system, and the trajectory generation module draws a driving trajectory of the motion carrier where the camera is located in the coordinate system.

[0042] The panoramic annular lens-based visual SLAM system comprises the following steps:

[0043] Step one, initialization, the camera carrying the panoramic annular lens acquires an image sequence at a certain frame rate, the image sequence is sequentially input into the panoramic annular lens-based visual SLAM system, and the image is appropriately preprocessed if necessary. The panoramic annular lens-based visual SLAM system extracts and matches feature points in the effective area of the first input several panoramic pictures, calculates the pose between two frames according to the matching result. In this process, an adaptive threshold related to the distance of the feature point to the optical center is used to screen the inliers, so as to obtain more accurate pose estimation, then the spatial point coordinates are obtained by using triangulation, and the optimization is performed by using the bundle adjustment, the initial pose and the initial map point are obtained, and the initial map is drawn.

[0044] Step two, after the initialization is completed, the panoramic annular lens-based visual SLAM system will perform a tracking process, by analyzing the features of the current frame effective position and using a corresponding tracking model to obtain a rough value of the camera pose, then the feature points of the current frame are matched with the local map points, the current pose is further optimized by using the bundle adjustment, then the number of inliers meeting the requirements is calculated, if the number is sufficient, it is considered that the tracking is successful, step three is performed, otherwise, the next frame panoramic image is repositioned.

[0045] The tracking process selects a corresponding tracking model according to the number of matched points, such as Figure 4As shown, P1 and P2 are map points. When the camera moves from the first origin O1 of the camera coordinate system to the second origin O2, a tracking model needs to be selected to estimate the current camera pose. A constant velocity model is preferred for tracking the map points of the previous frame. If the number of matching points is insufficient, a bag-of-words model is used to track the previous keyframe. If the previous keyframe cannot be tracked, an attempt is made to calculate the initial pose of the current frame; otherwise, relocalization is performed to obtain a coarse value of the current camera pose. After obtaining the coarse value of the current camera pose, the objective function is optimized by matching the effective region feature points and local map points of the current frame. To obtain the current pose optimization result, where ξ is the camera pose represented by the Lie algebra, and x c It is the unit vector of the feature point in the camera coordinate system, x w These are the coordinates of the corresponding map points, n is the number of currently valid matching points, and i represents the index of these points.

[0046] Step 3: If the tracking in Step 2 is successful, determine whether the current frame is a keyframe. If it is, insert a keyframe sequence and proceed to Step 4; otherwise, do not insert a keyframe sequence.

[0047] Step 4: In the map building thread, update the map points based on the co-view relationship between the current keyframe and existing keyframes, remove redundant keyframes, and perform bundle adjustment to further update the map points and the link relationship between map points and corresponding keyframes.

[0048] Step 5: In the loop closure detection thread, the continuity test is used to obtain the loop closure keyframe with the most matching points to the current keyframe. The matching feature points between the current keyframe and the loop closure keyframe are searched based on the bag-of-words model. If the number of matching feature points is large enough, it is determined that a loop closure has been detected. After subsequent optimization, more accurate map points and camera poses are obtained.

[0049] Figure 3 This is a schematic diagram of the optical path imaging of a camera equipped with the panoramic annular lens disclosed in an embodiment of the present invention. Figure 3 As shown, aperture 2 is located between panoramic annular lens 1 and relay lens 3, image 4 is located between image region 5 and relay lens 3, and blind zone 6 is located on image region 5. The panoramic annular lens used in this embodiment is a double-refractive type. Panoramic annular lenses also include other types such as single-refractive types, and this embodiment of the invention does not impose any limitations on them.

[0050] Step one includes the following steps:

[0051] Step eleven, install the camera with the panoramic annular lens on the moving carrier and adjust the orientation of the camera; the orientation of the camera when installed depends on the specific use environment, for example, when placed on a small ground mobile robot, the camera should be pointed upwards to capture more scene features because the camera is close to the ground. When the camera with the panoramic annular lens is installed on a drone, it should be pointed downwards to capture more scene features;

[0052] Step twelve, after the camera is installed and adjusted, calibrate to obtain the parameters required by the camera model, and then establish the corresponding projection model and back projection model, i.e. the conversion relationship between the pixel coordinate system and the camera coordinate system;

[0053] Step thirteen, use the camera with the panoramic annular lens to capture image sequences at a certain frame rate, and input the image sequences into the panoramic annular lens-based visual SLAM system in turn, and if necessary, perform appropriate preprocessing on the images;

[0054] Step fourteen, the panoramic annular lens-based visual SLAM system extracts and matches feature points in the effective area of the first input several panoramic pictures, and calculates the pose between two frames according to the matching. In this process, an adaptive threshold related to the distance of the feature point to the optical center is used to screen inliers, so as to obtain more accurate pose estimation, and then use triangulation to obtain spatial point coordinates and use bundle adjustment for optimization. This step will obtain the initial pose and initial map points, and at the same time, draw the initial map; the effective area in the panoramic picture refers to the area excluding the blind area, and the blind area should be avoided when extracting feature points to reduce false matching. If dynamic object removal is used, the effective area includes the static scene part after the dynamic object is removed in addition to the part excluding the blind area. Extracting feature points and matching in the effective area can further improve the accuracy of positioning.

[0055] Step fifteen, the adaptive threshold is a threshold used to screen inliers in the initialization process of the panoramic annular lens-based visual SLAM system, and the size of the threshold depends on the distance of the corresponding feature point to the optical center. After calculating the essential matrix according to the matching feature points of two pictures, it is necessary to further screen inliers according to the epipolar constraint , where x1 and x2 are the projections of the matching points in the two pictures on the unit sphere, E is the fundamental matrix, and T represents matrix transposition. Then, it is determined whether the essential matrix meets the requirements according to the number of inliers.

[0056] Due to the noise in the image, the epipolar constraint is not exactly zero, but there is a certain error, which is related to the distance of the feature point to the optical center, so an adaptive threshold is used to screen inliers, as shown in the following formula (1):

[0057]

[0058] Due to the difference in the distance from the matching feature points on the panoramic image to the optical center, T represents matrix transposition, and theta represents the projection of the inlier filtering threshold on the unit sphere. The size of theta is related to the projection position of x1 and x2 on the unit sphere. The adaptive threshold can more effectively eliminate points with large errors, thereby improving the positioning accuracy.

[0059] In summary, the present application can solve the problem of poor robustness in the case of fast movement and turning of the camera caused by the limited field of view angle of the traditional camera-based visual SLAM. The panoramic annular lens can realize 360-degree imaging without splicing multiple cameras, so the entire visual SLAM system based on the panoramic annular lens is more portable and efficient, and has wide application prospects in the fields of autonomous driving and intelligent mobile robots. The present application uses a panoramic annular lens to obtain more scene features, and effectively improves the positioning accuracy by screening effective area features and using an adaptive threshold to eliminate points with large errors, and has wide application prospects in the fields of mobile robots and autonomous driving. The present application has good robustness in the area of fast turning, and can effectively identify the closed loop and perform global optimization even in reverse motion, and does not need to perform unwrapping of the annular image, which can ensure the real-time performance of the algorithm.

[0060] Some of the blocks and / or flowcharts in the drawings are shown. It should be understood that some of the blocks in the block diagram and / or flowchart or combination thereof can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, so that these instructions can create a device for implementing the functions / operations described in the block diagram and / or flowchart when executed by the processor.

[0061] Therefore, the technology of the present application can be realized in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present application can take the form of a computer program product on a computer readable storage medium storing instructions, which can be used by or in conjunction with an instruction execution system. In the context of the present application, the computer readable storage medium can be any medium capable of containing, storing, communicating, propagating or transmitting instructions. For example, the computer readable storage medium can include but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices, devices or propagation media. Specific examples of computer readable storage media include: magnetic storage devices such as magnetic tapes or hard disks (HDD); optical storage devices such as optical discs (CD-ROM); memories such as random access memories (RAM) or flash memories; and / or wired / wireless communication links.

[0062] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0063] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0064] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer accessible memory. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of steps for causing a computer device (which can be a personal computer, a server or a network device, etc., and specifically can be a processor in the computer device) to execute the above-mentioned methods of each embodiment of the present application.

[0065] In addition, it should be understood that, although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that those skilled in the art can understand.

Claims

1. A visual SLAM system based on a panoramic annular lens, characterized in that, The visual SLAM system based on a panoramic ring lens includes: The acquisition unit acquires an image sequence at a certain frame rate using a camera equipped with the panoramic ring lens, and then performs some preliminary image processing. The recognition unit analyzes the results of image processing, then identifies and extracts the scene features of the ring-shaped effective area of ​​the PAL image, and performs inter-frame matching. The positioning unit estimates and optimizes the pose by selecting the appropriate tracking model based on the data obtained by the acquisition unit and the identification unit, thereby determining the positioning information of the moving vehicle where the camera is located in the environment. The mapping unit, through the acquisition unit and the positioning unit, draws the trajectory of the moving vehicle where the camera is located, and calculates the spatial coordinates of feature points in the environment to reconstruct the scene map; The adaptive threshold is used to filter interior points during the initialization of the panoramic ring lens-based visual SLAM system. The value of this threshold depends on the distance from the corresponding feature point to the optical center. After calculating the essential matrix based on the matching feature points of two frames, it is necessary to apply epipolar constraints. Further filter the interior points, where x1 and x2 are the projections of the matching points in the two frames onto the unit sphere, E is the fundamental matrix, and T represents the matrix transpose; then determine whether the fundamental matrix meets the requirements based on the number of interior points.

2. The visual SLAM system based on a panoramic annular lens as described in claim 1, characterized in that, The acquisition unit includes an image acquisition module and an image processing module. The image acquisition module acquires an image sequence, and the image processing module processes the images acquired by the image acquisition module. The processing methods include gamma correction and specular suppression.

3. The visual SLAM system based on a panoramic annular lens as described in claim 1, characterized in that, The identification unit includes a blind spot removal module, a feature extraction module, and a feature matching module. The blind spot removal module will identify the blind spot area to avoid extracting and matching feature points in this area; the feature extraction module will extract feature points in the effective area; and the feature matching module will match the extracted feature points.

4. The visual SLAM system based on a panoramic annular lens as described in claim 1, characterized in that, The positioning unit includes an initialization module, a feature tracking module, and a pose optimization module. The initialization module initializes the system to obtain the initial pose and map points. The feature tracking module selects the appropriate tracking model to estimate the pose based on the data obtained by the acquisition unit and the recognition unit. The pose optimization module optimizes the pose to perform the positioning process of the moving vehicle where the camera is located.

5. The visual SLAM system based on a panoramic annular lens as described in claim 1, characterized in that, The mapping unit includes a trajectory generation module and a map building module. The map building module calculates the spatial coordinates of feature points in the environment and continuously updates and optimizes them, and reconstructs a scene map. This scene map is matched with the spatial coordinate system established by the visual SLAM system based on the panoramic ring lens. The trajectory generation module draws the driving trajectory of the moving vehicle where the camera is located in this coordinate system.

6. A visual SLAM method based on a panoramic annular lens, characterized in that, This method employs the visual SLAM system based on a panoramic ring lens as described in claim 1, and includes the following steps: Step 1: Initialize; Step 2: After initialization, the tracking process will be executed. By analyzing the features of the effective position in the current frame and using the corresponding tracking model, a rough value of the camera pose will be obtained. Then, the feature points of the current frame will be matched with the local map points. Bundle adjustment will be used to further optimize the current pose. Then, the number of inliers that meet the requirements will be calculated. If the number is sufficient, the tracking is considered successful and Step 3 will be executed. Otherwise, relocalization will be performed based on the panoramic image of the next frame. Step 3: If the tracking in Step 2 is successful, determine whether the current frame is a keyframe. If it is, insert a keyframe sequence and proceed to Step 4; otherwise, do not insert a keyframe sequence. Step 4: In the map building thread, update the map points based on the co-view relationship between the current keyframe and existing keyframes, remove redundant keyframes, and perform bundle adjustment to further update the map points and the link relationship between map points and corresponding keyframes. Step 5: In the loop closure detection thread, the continuity test is used to obtain the loop closure keyframe with the most matching points to the current keyframe. The matching feature points between the current keyframe and the loop closure keyframe are searched based on the bag-of-words model. If the number of matching feature points is large enough, it is determined that a loop closure has been detected. After subsequent optimization, more accurate map points and camera poses are obtained.

7. The visual SLAM method based on a panoramic annular lens as described in claim 6, characterized in that, Step one includes the following steps: Step 1: Install the camera equipped with the panoramic ring lens on the moving vehicle and adjust the camera's orientation; the orientation of the camera during installation depends on the specific usage environment. Steps one and two: After installing and adjusting the camera, calibration is performed to obtain the parameters required for the camera model. Then, the corresponding projection model and back projection model are established, that is, the transformation relationship between the pixel coordinate system and the camera coordinate system. Step 13: Use the camera equipped with the panoramic ring lens to acquire image sequences at a certain frame rate, and input the image sequences sequentially into the visual SLAM system based on the panoramic ring lens; Step 14: The visual SLAM system based on the panoramic ring lens extracts and matches feature points in the effective areas of the first input panoramic images, and calculates the pose between two frames based on the matching. Step 15, the adaptive threshold is the threshold used to filter interior points during the initialization of the visual SLAM system based on the panoramic ring lens. The size of this threshold depends on the distance from the corresponding feature point to the optical center.

Images