Visual SLAM method for dark scene

Abstract

The invention provides a visual SLAM method for a dark scene, and belongs to the technical field of visual SLAM.The method comprises the steps that an input image is processed, angular points are detected from the processed image through an AGAST algorithm, a BRIEF descriptor is generated for the image, complete feature points are formed, the initial pose of a camera is estimated, a tracking thread is matched with a key point pair of a current frame and a local map, and the initial pose of the camera is obtained; inserting the key frame into a local mapping thread; optimizing the poses and map points of the key frames, and deleting redundant key frames and related map points at the same time; a loopback detection thread obtains an accumulative error, corrects the pose estimation value of the current frame through similar transformation, carries out alignment operation on two ends of a loopback, and disperses a loopback closing error into the whole map to complete loopback processing; according to the method, feature points can still be extracted in a weak texture area, high precision is kept, in a dark environment, trajectory estimation errors are remarkably reduced, and higher robustness and precision are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual SLAM, and in particular, relates to a visual SLAM method for dim scenes. Background Art

[0002] With the rapid development of sensors and advanced information technology, the demand for autonomous exploration and mapping of diverse underground environments is increasing. In environments where the Global Positioning System (GPS) signal is limited and there is no pre-map, such as tunnels, urban underground environments, and complex cave networks, Simultaneous Localization and Mapping (SLAM) technology, as a positioning technology that does not rely on external signal sources, has become the key to achieving autonomous navigation.

[0003] In underground scenes such as mines, underground garages or abandoned tunnels, traditional visual SLAM systems face many challenges due to poor lighting conditions and lack of obvious texture features on walls and floors. Images collected in such environments usually have problems with high noise and low contrast, which makes feature point detection and matching difficult, reducing the accuracy of positioning and map construction. In addition, real-time performance is particularly important in these environments. During the operation of unmanned vehicles, robots or drones, excessive delays may cause positioning drift or even navigation failure, directly affecting the safety and reliability of the system.

[0004] With the continuous development of related research, although a variety of visual SLAM methods for dim scenes have been proposed and some progress has been made, building a SLAM system that can run in real time in dim environments is of great significance for improving the exploration efficiency of equipment in underground environments, shortening mission time, and promoting the popularization and development of underground intelligent applications. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention optimizes ORB-SLAM2, combines Retinex image processing technology with the Adaptive General Accelerated Segmentation Detection (AGAST) method, and proposes a visual SLAM method for dim scenes, which effectively improves the positioning accuracy of the system under low-light conditions while ensuring its real-time performance, and solves the positioning error problem caused by the degradation of image quality due to dim scenes in practical applications of the visual SLAM system.

[0006] The present invention is achieved through the following technical solutions:

[0007] A visual SLAM method for dim scenes:

[0008] The method specifically comprises the following steps:

[0009] Step 1, the tracking thread receives the image captured by the camera;

[0010] Step 2, using a Retinex-based image processing algorithm to process the input image;

[0011] Step 3, using the AGAST algorithm to detect corner points from the processed image, and generate BRIEF descriptors to form complete feature points;

[0012] Step 4: Camera initialization; SLAM system initialization, map points generation, key frames generation and sending to local mapping thread,

[0013] Step 5: The local mapping thread processes the keyframes sent by the receiving tracking thread, generates more reliable map points, optimizes the pose of the common view keyframes and their map points; performs local BA optimization, checks and removes redundant local keyframes and their related map points, and inserts the current keyframe into the loop detection thread;

[0014] In step 6, the loop detection thread filters out possible loop frames through the database, calculates the similarity transformation between the current frame and the loop frame, and obtains the cumulative error; then, the pose estimation value of the current frame is corrected through the similarity transformation, and the two ends of the loop are aligned to disperse the loop closure error to the entire map, completing the loop processing.

[0015] Furthermore, in step 1, the average grayscale value AGL of the input image is calculated:

[0016]

[0017] Where M is the height of the image, N is the width of the image, and I is the light intensity of the image.

[0018] When the average grayscale value of the input image is greater than 75, directly proceed to step 3 to extract corner points and descriptors, otherwise execute step 2 to process the image.

[0019] Furthermore, step 2 includes:

[0020] Step 2.1, process the image with an average grayscale value less than 75, and construct the Gaussian formula of the image grayscale map, where I represents the light intensity of the image, representing the lighting effect of the external light source on the object; p = (x, y) and q = (x, y) represent the position of the pixel in the image, I p with I q represents the light intensity at p and q, σ L is the scale parameter of the Gaussian function.

[0021]

[0022] Step 2.2: Construct a cost function and solve the illumination map L of the image by minimizing the cost function; where N(p) represents the area around p, and λ is a control variable. The larger its value, the smoother the illumination map obtained:

[0023]

[0024] Step 2.3: Derivative the cost function to make its derivative equal to 0; where E is the identity matrix, A is an S×S matrix, S=W×H, W is the width of the image, and H is the height of the image;

[0025] L=(Ε+λA) -1 N

[0026] Step 2.4: The above formula A is specifically defined as follows; where N(m) represents the neighborhood of pixel m;

[0027]

[0028] Step 2.5: After obtaining L, the image is processed using an algorithm based on the Retinex theory;

[0029]

[0030] Among them, L represents the illumination map of the image, R represents the reflection component of the object carrying the image detail information, and I represents the illumination intensity of the image, which represents the illumination effect of the external light source on the object.

[0031] Furthermore, in step 3, the corner points are extracted from the image as follows:

[0032] Step 3.1.1: Traverse each pixel in the image and select a group of pixels in a circular area with a radius of 3 around a given pixel p;

[0033] Step 3.1.2: Set a threshold value θ. If there are at least 12 consecutive pixels on the circle whose grayscale value differs from the grayscale value of the center point I(p) by more than the threshold θ, then this pixel is considered to be a corner point. The optimization goal of adaptive threshold adjustment is as follows: solve and obtain the parameter that maximizes the information gain (IG). Where H(D) represents the information entropy of the data set D, |D k | indicates that under a given parameter θ, the kth subset D after segmentation k The size of |D| represents the size of the original data set D, H(D k ) represents the subset D k The information entropy of; p(c) represents the probability of category c appearing in data set D;

[0034] θ=argmax[IG(D,θ)]

[0035]

[0036]

[0037] Step 3.1.3: Build an adaptive decision tree to automatically select the optimal pixel comparison order and determine whether it is a corner point with the least number of comparisons;

[0038] Step 3.1.4: In the decision tree, each node represents a pixel comparison operation; according to the grayscale difference between the current pixel and the center point, select the child node to the left or right until the final judgment result is obtained;

[0039] Step 3.1.5: If a node at a certain layer has determined that the pixel does not meet the corner point condition during the judgment process, the detection is terminated in advance to improve the detection efficiency.

[0040] Furthermore, in step 3, the BRIEF descriptor is generated as follows:

[0041] Step 3.2.1: Gaussian filtering is performed on the relative image;

[0042] Step 3.2.2: Take a certain size image window with the corner point as the center; randomly select a pair of points in the window, compare the size of the two pixels, and perform binary assignment:

[0043]

[0044] Step 3.2.3: Randomly select N pairs of feature points in the window, repeat step 3.2.2, and finally obtain a 256-dimensional binary descriptor.

[0045] Furthermore, in step 4, it includes:

[0046] Step 4.1, the tracking thread estimates the initial pose of the camera through the previous frame image; otherwise, the first initial pose estimation is performed;

[0047] Step 4.2: After the initial pose is determined, the tracking thread further optimizes the current pose estimate by matching feature point pairs between the current frame and the local map; if tracking fails, it attempts to relocalize in the entire map;

[0048] In step 4.3, the keyframes that meet the conditions will be inserted into the local mapping thread. When inserting a keyframe, first close the keyframe insertion operation, check whether there is a keyframe in the local map queue, if so, take the first keyframe from the candidate keyframe queue and delete it from the queue, then calculate the BoW structure of the current keyframe, read all map points of the current keyframe, and finally update the co-visual map.

[0049] Furthermore, in step 5, the local mapping thread is responsible for screening the map point cloud; for map points observed by three key frames at the same time, they are regarded as reliable map points.

[0050] Furthermore, in step 6, it includes:

[0051] Step 6.1: The loop detection thread checks whether there are key frames in the queue. If there are only 10 key frames or there are less than 10 key frames from the last loop, no loop detection is required. Otherwise, possible loop frames are screened out through the database, and the similarity transformation between the current frame and the loop frame is calculated to obtain the cumulative error.

[0052] Step 6.2: When the loop is detected, terminate the global BA, update the co-visual keyframe of the current frame, estimate the pose of the current frame using the loop, update the poses of all the co-visual keyframes, and correct all the map points of the current frame and its co-visual keyframes according to the new pose information;

[0053] Step 6.3, replace the map points corresponding to the current frame with the map points that match the loop frame, fuse the repeated map points, update the co-visual key frame of the current frame, and ensure that the current frame is connected with the loop frame; optimize the essential graph and start a new thread to perform global BA.

[0054] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0055] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps of the above method are implemented.

[0056] Beneficial effects of the present invention

[0057] In dim scenes, traditional visual SLAM methods often have insufficient effective feature points due to the small number of extracted feature points, which leads to tracking failure, and then causes problems such as low positioning accuracy, poor robustness, and system downtime. The present invention proposes a real-time visual SLAM system that combines Retinex image processing technology with the AGAST algorithm; by introducing the Retinex image processing technology to pre-process the input image, the brightness and details of the image are significantly enhanced, and the visual quality of the image in dim scenes is improved. At the same time, the AGAST algorithm is used to extract corner points, thereby ensuring the positioning accuracy and robustness of the system in low-light environments.

[0058] The present invention effectively solves the problem that traditional SLAM systems are difficult to extract sufficient and stable feature points in dim scenes, greatly improves the positioning accuracy and reliability of the system, and enhances its adaptability in complex dim environments; the present invention can still extract feature points in weak texture areas and maintain high accuracy, significantly reduces trajectory estimation errors in dim environments, and has stronger robustness and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a system framework diagram of the present invention.

[0060] Figure 2 Comparison diagram of the original image and the processed image, where (a) is the original image and (b) is the processed image.

[0061] Figure 3 This is a comparison chart of the TUM dataset trajectories.

[0062] Figure 4 This is a comparison chart of the EuRoC dataset trajectories. DETAILED DESCRIPTION

[0063] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The experimental methods used in the following examples are conventional methods unless otherwise specified. The materials, reagents, methods and instruments used are conventional materials, reagents, methods and instruments in the art unless otherwise specified, and can be obtained through commercial channels by those skilled in the art.

[0065] like Figure 1 As shown, the present invention proposes a visual SLAM method for dim scenes, including three main threads: a tracking thread, a local mapping thread, and a loop detection thread.

[0066] Step 1, the tracking thread receives the image captured by the camera;

[0067] In step 1, the average grayscale value AGL of the input image is calculated:

[0068]

[0069] Where M is the height of the image, that is, the number of rows of pixels. N is the width of the image, that is, the number of columns of pixels. I is the light intensity of the image,

[0070] When the average grayscale value of the input image is greater than 75, directly proceed to step 3 to extract corner points and descriptors, otherwise execute step 2 to process the image.

[0071] Step 2, using a Retinex-based image processing algorithm to process the input image;

[0072] Step 2.1, process the image with an average grayscale value less than 75, and construct the Gaussian formula of the image grayscale map, where I represents the light intensity of the image, representing the lighting effect of the external light source on the object; p = (x, y) and q = (x, y) represent the position of the pixel in the image, I p with I q represents the light intensity at p and q, σ L is the scale parameter of the Gaussian function.

[0073]

[0074] Step 2.2: Construct a cost function and solve the illumination map L of the image by minimizing the cost function; where N(p) represents the area around p, and λ is a control variable. The larger its value, the smoother the illumination map obtained:

[0075]

[0076] Step 2.3: Derivative the cost function to make its derivative equal to 0; where E is the identity matrix, A is an S×S matrix, S=W×H, W is the width of the image, and H is the height of the image;

[0077] L=(Ε+λA) -1 N

[0078] Step 2.4: The above formula A is specifically defined as follows; where N(m) represents the neighborhood of pixel m;

[0079]

[0080] Step 2.5: After obtaining L, the image is processed using an algorithm based on the Retinex theory;

[0081]

[0082] Among them, L represents the brightness of the image, R represents the reflection component of the object carrying the image detail information, and I represents the light intensity of the image, which represents the lighting effect of the external light source on the object.

[0083] Step 3, extracting corner points and descriptors from the processed image;

[0084] In step 3, the corner points are extracted from the image as follows:

[0085] Step 3.1.1: Traverse each pixel in the image and select a group of pixels in a circular area with a radius of 3 around a given pixel p;

[0086] Step 3.1.2: Set a threshold value θ. If there are at least 12 consecutive pixels on the circle whose grayscale value differs from the grayscale value of the center point I(p) by more than the threshold θ, then this pixel is considered to be a corner point. The optimization goal of adaptive threshold adjustment is as follows: solve and obtain the parameter that maximizes the information gain (IG). Where H(D) represents the information entropy of the data set D, |D k | indicates that under a given parameter θ, the kth subset D after segmentation k The size of |D| represents the size of the original data set D, H(D k ) represents the subset D k The information entropy of; p(c) represents the probability of category c appearing in data set D;

[0087] θ=argmax[IG(D,θ)]

[0088]

[0089]

[0090] Step 3.1.3: Build an adaptive decision tree to automatically select the optimal pixel comparison order and determine whether it is a corner point with the least number of comparisons;

[0091] Step 3.1.4: In the decision tree, each node represents a pixel comparison operation; according to the grayscale difference between the current pixel and the center point, select the child node to the left or right until the final judgment result is obtained;

[0092] Step 3.1.5: If a node at a certain layer has determined that the pixel does not meet the corner point condition during the judgment process, the detection is terminated in advance to improve the detection efficiency.

[0093] Generate the descriptor as follows:

[0094] Step 3.2.1: First, perform Gaussian filtering on the image;

[0095] Step 3.2.2: Take a certain size image window with the corner point as the center; randomly select a pair of points in the window, compare the size of the two pixels, and perform binary assignment:

[0096]

[0097] Step 3.2.3: Randomly select N pairs of feature points in the window, repeat step 3.2.2, and finally obtain a 256-dimensional binary descriptor.

[0098] Step 4: Camera initialization; SLAM system initialization, map points generation, key frames generation and sending to local mapping thread,

[0099] Step 4.1, the tracking thread estimates the initial pose of the camera through the previous frame image; otherwise, the first initial pose estimation is performed;

[0100] Step 4.2: After the initial pose is determined, the tracking thread further optimizes the current pose estimate by matching feature point pairs between the current frame and the local map; if tracking fails, it attempts to relocalize in the entire map;

[0101] In step 4.3, the keyframes that meet the conditions will be inserted into the local mapping thread. When inserting a keyframe, first close the keyframe insertion operation, check whether there is a keyframe in the local map queue, if so, take the first keyframe from the candidate keyframe queue and delete it from the queue, then calculate the BoW structure of the current keyframe, read all map points of the current keyframe, and finally update the co-visual map.

[0102] Step 5: The local mapping thread processes the keyframes sent by the receiving tracking thread, generates more reliable map points, and optimizes the pose of the common view keyframes and their map points. In the local window, the local mapping thread performs local BA (Bundle Adjustment) optimization, checks and removes redundant local keyframes and their related map points, and inserts the current keyframe into the loop detection thread.

[0103] The local mapping thread is responsible for screening the map point cloud; map points that are simultaneously observed by three key frames are considered reliable map points.

[0104] In step 6, the loop detection thread filters out possible loop frames through the database, calculates the similarity transformation between the current frame and the loop frame, and obtains the cumulative error; then, the pose estimation value of the current frame is corrected through the similarity transformation, and the two ends of the loop are aligned to disperse the loop closure error to the entire map, completing the loop processing.

[0105] Step 6.1: The loop detection thread checks whether there are key frames in the queue. If there are only 10 key frames or there are less than 10 key frames from the last loop, no loop detection is required. Otherwise, possible loop frames are screened out through the database, and the similarity transformation between the current frame and the loop frame is calculated to obtain the cumulative error.

[0106] Step 6.2: When the loop is detected, terminate the global BA, update the co-visual keyframe of the current frame, estimate the pose of the current frame using the loop, update the poses of all the co-visual keyframes, and correct all the map points of the current frame and its co-visual keyframes according to the new pose information;

[0107] Step 6.3, replace the map points corresponding to the current frame with the map points that match the loop frame, fuse the repeated map points, update the co-visual key frame of the current frame, and ensure that the current frame is connected with the loop frame; optimize the essential graph and start a new thread to perform global BA.

[0108] Example:

[0109] Implementation environment: computer with AMD R9-7945hx CPU, 4060GPU, 16GB memory, and Ubuntu20.04 operating system.

[0110] By comparing the ORB-SLAM2 system and the present invention, two public data sets TUM and EuRoC are selected to evaluate the performance. Among them, the TUM data set is provided by the Technical University of Munich in Germany, covering monocular, binocular and RGB-D data. This test selects the weak texture scenes fr3_notexture_near and fr3_notexture_near_validation in the RGB-D data set to verify the performance of the present invention in a weak texture environment. The EuRoC data set is provided by the Swiss Federal Institute of Technology in Zurich and contains rich binocular vision data. This test selects the binocular data set V1_03_difficult in a low-light environment and MH_05_difficult in a fast-moving scene to evaluate the positioning accuracy and robustness of the present invention under dim and fast-moving conditions. The test results fully demonstrate the performance advantages of the present invention in weak texture, low light and dynamic and complex scenes.

[0111] like Figure 2 As shown, the brightness of the processed image is significantly improved, while more detailed textures are retained, improving image quality.

[0112] Table 1 shows the comparison results and improvement ratio of the absolute trajectory error (ATE) between the present invention and ORB-SLAM2. The smaller the value, the higher the system accuracy. The best performance results are bolded. All data are the average values ​​after each system runs five times. For the fr3_notexture_near and fr3_notexture_near_validation data sets, the RMSE accuracy of the present invention is improved by 56.25% and 69.23% respectively, and the average error is reduced by 60.00% and 68.75%, showing excellent results in weak texture environments. On the V1_03_difficult and MH_05_difficult data sets, the RMSE accuracy of the present invention is improved by 19.48% and 17.39% respectively, and the average error is reduced by 24.41% and 15.38%. In general, the present invention is significantly better than ORB-SLAM2 on all test data sets, showing extremely high accuracy and stability.

[0113] In summary, the present invention aims at the problem of difficulty in feature extraction and matching in traditional visual SLAM systems in weak texture and dim scenes, and proposes a visual SLAM system designed for dim scenes. The system first adopts an image processing method based on Retinex theory to effectively improve the brightness and detail quality of the input image; then, the AGAST corner detection method is used for feature tracking, which significantly enhances the accuracy of feature extraction and matching in dim and weak texture scenes, thereby improving the pose estimation accuracy of the system. Experimental results show that in the tests of TUM and EuRoC datasets, the present invention exhibits stronger stability and robustness in dim environments. Compared with ORB-SLAM2, its absolute trajectory error is reduced by at least 15.38%, which fully demonstrates the superior performance of the present invention.

[0114] Table 1 ATE comparison results of some datasets

[0115]

[0116] Figure 3 A comparison chart of the simulation results of the TUM data set is provided, in which the red line segment represents the difference between the real trajectory and the camera trajectory. The longer the line segment, the greater the trajectory error, and the absence of a red line segment indicates that the tracking is lost during this period. As can be seen from the figure, when running the two data sets, ORB-SLAM2 shows a large number of feature tracking losses in the first half. This is because there are fewer image textures in the first half, and ORB-SLAM2 has difficulty extracting enough feature points in weak texture areas, resulting in the system being unable to perform stable feature tracking. In contrast, the present invention can still extract feature points in weak texture areas and maintain high accuracy.

[0117] Figure 4 The trajectory comparison of the EuRoC data set is shown. The orange line segment represents the trajectory of the present invention, the blue line segment represents the trajectory of ORB-SLAM2, and the dotted line represents the real trajectory. In the enlarged detail part, it can be clearly seen that ORB-SLAM2 has a large trajectory deviation, which is mainly due to the violent fluctuation of the camera in this segment of the trajectory and the dim ambient light, resulting in a large number of feature mismatches in ORB-SLAM2, thereby causing significant trajectory errors. When dealing with weak textures and dim environments, the present invention significantly reduces the trajectory estimation error, showing stronger robustness and accuracy.

[0118] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0119] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps of the above method are implemented.

[0120] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory, ROM, a programmable read-only memory, PROM, an erasable programmable read-only memory, EPROM, an electrically erasable programmable read-only memory, EEPROM, or a flash memory. The volatile memory may be a random access memory, RAM, which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory static RAM, SRAM, dynamic random access memory dynamic RAM, DRAM, synchronous dynamic random access memory synchronous DRAM, SDRAM, double data rate synchronous dynamic random access memory double data rate SDRAM, DDR SDRAM, enhanced synchronous dynamic random access memory enhanced SDRAM, ESDRAM, synchronous link dynamic random access memory synchlink DRAM, SLDRAM, and direct memory bus random access memory direct rambus RAM, DR RAM. It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0121] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center through a wired method such as coaxial cable, optical fiber, digital subscriber line digital subscriber line, DSL or wireless such as infrared, wireless, microwave, etc. to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium such as a floppy disk, a hard disk, a tape, an optical medium such as a high-density digital video disc digital video disc, DVD, or a semiconductor medium such as a solid state hard disk solid state disc, SSD, etc.

[0122] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0123] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined to perform. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0124] The above is a detailed introduction to the visual SLAM method for dim scenes proposed in the present invention, and the principles and implementation methods of the present invention are explained. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A visual SLAM method for dim scenes, characterized by: The method specifically comprises the following steps: Step 1, the tracking thread receives the image captured by the camera; Step 2, using a Retinex-based image processing algorithm to process the input image; Step 3, using the AGAST algorithm to detect corner points from the processed image, and generate BRIEF descriptors to form complete feature points; Step 4: Camera initialization; SLAM system initialization, map points generation, key frames generation and sending to local mapping thread, Step 5: The local mapping thread processes the keyframes sent by the receiving tracking thread, generates more reliable map points, optimizes the pose of the common view keyframes and their map points; performs local BA optimization, checks and removes redundant local keyframes and their related map points, and inserts the current keyframe into the loop detection thread; In step 6, the loop detection thread filters out possible loop frames through the database, calculates the similarity transformation between the current frame and the loop frame, and obtains the cumulative error; then, the pose estimation value of the current frame is corrected through the similarity transformation, and the two ends of the loop are aligned to disperse the loop closure error to the entire map, completing the loop processing.

2. The method according to claim 1, characterized in that: In step 1, the average grayscale value AGL of the input image is calculated: Where M is the height of the image, N is the width of the image, and I is the light intensity of the image. When the average grayscale value of the input image is greater than 75, directly proceed to step 3 to extract AGAST corner points and descriptors, otherwise execute step 2 to process the image.

3. The method according to claim 2, characterized in that: In step 2 include: Step 2.1, process the image with an average grayscale value less than 75, and construct the Gaussian formula of the image grayscale map, where I represents the light intensity of the image, representing the lighting effect of the external light source on the object; p = (x, y) and q = (x, y) represent the position of the pixel in the image, I p with I q represents the light intensity at p and q, σ L is the scale parameter of the Gaussian function. Step 2.2: Construct a cost function and solve the illumination map L of the image by minimizing the cost function; where N(p) represents the area around p, and λ is a control variable. The larger its value, the smoother the illumination map obtained: Step 2.3: Derivative the cost function to make its derivative equal to 0; where E is the identity matrix, A is an S×S matrix, S=W×H, W is the width of the image, and H is the height of the image; L=(E+λA) -1 N Step 2.4: The above formula A is specifically defined as follows; where N(m) represents the neighborhood of pixel m; Step 2.5: After obtaining L, the image is processed using an algorithm based on the Retinex theory; Among them, L represents the illumination map of the image; R represents the reflection component of the object that carries the image detail information; I represents the illumination intensity of the image, which represents the illumination effect of the external light source on the object.

4. The method according to claim 3, characterized in that: In step 3, the corner points are extracted from the image as follows: Step 3.1.1: Traverse each pixel in the image and select a group of pixels in a circular area with a radius of 3 around a given pixel p; Step 3.1.2: Set a threshold θ. If there are at least 12 consecutive pixels on the circle whose grayscale value differs from the grayscale value of the center point I(p) by more than the threshold θ, then this pixel is considered to be a corner point. The optimization goal of adaptive threshold adjustment is as follows: to solve the parameter that maximizes the information gain (IG); where H(D) represents the information entropy of the data set D, |D k | indicates that under a given parameter θ, the kth subset D after segmentation k The size of |D| represents the size of the original data set D, H(D k ) represents the subset D k The information entropy of; p(c) represents the probability of category c appearing in data set D; θ=argmax[IG(D,θ)] Step 3.1.3: Build an adaptive decision tree to automatically select the optimal pixel comparison order and determine whether it is a corner point with the least number of comparisons; Step 3.1.4: In the decision tree, each node represents a pixel comparison operation; according to the grayscale difference between the current pixel and the center point, select the child node to the left or right until the final judgment result is obtained; Step 3.1.5: If a node at a certain layer has determined that the pixel does not meet the corner point condition during the judgment process, the detection is terminated in advance to improve the detection efficiency.

5. The method according to claim 4, characterized in that: In step 3, the BRIEF descriptor is generated as follows: Step 3.2.1: First, perform Gaussian filtering on the image; Step 3.2.2: Take a certain size image window with the corner point as the center; randomly select a pair of points in the window, compare the size of the two pixels, and perform binary assignment: Step 3.2.3: Randomly select N pairs of feature points in the window, repeat step 3.2.2, and finally obtain a 256-dimensional binary descriptor.

6. The method according to claim 5, characterized in that: In step 4, include: Step 4.1, the tracking thread estimates the initial pose of the camera through the previous frame image; otherwise, the first initial pose estimation is performed; Step 4.2: After the initial pose is determined, the tracking thread further optimizes the current pose estimate by matching feature point pairs between the current frame and the local map; if tracking fails, it attempts to relocalize in the entire map; In step 4.3, the keyframes that meet the conditions will be inserted into the local mapping thread. When inserting a keyframe, first close the keyframe insertion operation, check whether there is a keyframe in the local map queue, if so, take the first keyframe from the candidate keyframe queue and delete it from the queue, then calculate the BoW structure of the current keyframe, read all map points of the current keyframe, and finally update the co-visual map.

7. The method according to claim 6, characterized in that: In step 5, the local mapping thread is responsible for screening the map point cloud; map points observed by three key frames at the same time are regarded as reliable map points.

8. The method according to claim 7, characterized in that: In step 6, include: Step 6.1: The loop detection thread checks whether there are key frames in the queue. If there are only 10 key frames or there are less than 10 key frames from the last loop, no loop detection is required. Otherwise, possible loop frames are screened out through the database, and the similarity transformation between the current frame and the loop frame is calculated to obtain the cumulative error. Step 6.2: When the loop is detected, terminate the global BA, update the co-visual keyframe of the current frame, estimate the pose of the current frame using the loop, update the poses of all the co-visual keyframes, and correct all the map points of the current frame and its co-visual keyframes according to the new pose information; Step 6.3, replace the map points corresponding to the current frame with the map points that match the loop frame, fuse the repeated map points, update the co-visual key frame of the current frame, and ensure that the current frame is connected with the loop frame; optimize the essential graph and start a new thread to perform global BA.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images