A method for extracting ORB feature points under weak lighting conditions by introducing image enhancement and adaptive threshold

Abstract

The present invention discloses an ORB feature point extraction method under weak illumination conditions that introduces image enhancement and adaptive thresholding. The method converts the input image from the RGB color space to the HSV color space to separate the color information from the brightness information. During the enhancement process, only the brightness component V is processed, while the hue component H and the saturation component S remain unchanged. The brightness component V is processed using an MSR algorithm based on the Retinex theory. Then, the brightness component V after MSR processing is further processed using contrast-limited adaptive histogram equalization to limit the contrast enhancement degree of local areas while maintaining the overall brightness distribution of the image. The unprocessed hue component H and saturation component S are fused with the processed brightness component V and converted back to the RGB color space. At the same time, an adaptive threshold fast corner point extraction strategy is proposed, which can dynamically adjust the threshold according to the grayscale value of the image when the illumination changes, thereby improving the accuracy and robustness of feature point extraction.

Description

Technical Field

[0001] The invention relates to an ORB feature point extraction method under weak illumination conditions by introducing image enhancement and adaptive threshold, and belongs to the field of visual SLAM positioning. Background Art

[0002] Simultaneous localization and mapping (SLAM) is a key technology in robotics, enabling robots to autonomously build maps and locate their positions in real time in unknown environments. With the rapid development of computer vision and sensor technology, visual SLAM has gradually become a key branch of SLAM due to its low cost and rich information. Visual SLAM uses visual sensors (such as cameras) to acquire environmental images, perform pose estimation, and build maps, providing strong support for autonomous navigation and intelligent interaction for robots. However, the performance of visual SLAM technology often faces severe challenges in low-light conditions, which has become a hot and difficult issue in current research. Due to insufficient illumination, key information such as texture, edges, and color in the image becomes blurred or even lost, significantly impacting key steps in visual SLAM, such as feature extraction, matching, tracking, and mapping. Traditional visual SLAM algorithms typically rely on distinct feature points in the image for localization and mapping. However, in low-light conditions, these feature points are often difficult to accurately detect, resulting in a significant decrease in algorithm stability and accuracy. Furthermore, low light can increase image noise, further disrupting the algorithm's proper operation and causing deviations or distortions in localization and mapping results. Because low-light conditions pose a significant challenge to visual SLAM technology, research on this technology is extremely important. First, from a practical application perspective, many scenarios may occur in low light, such as nighttime patrols and underground space exploration. In these scenarios, autonomous robot navigation and map building are crucial for improving mission efficiency and safety. Therefore, research on visual SLAM technology in low-light conditions will help expand its application scope and enhance its adaptability and reliability in real-world scenarios. Summary of the Invention

[0003] The present invention provides an ORB feature point extraction method under weak light conditions by introducing image enhancement and adaptive thresholding, so as to realize ORB feature point extraction under weak light conditions by introducing image enhancement and adaptive thresholding, and further realize visual SLAM optimization under insufficient light scenes based on the extracted ORB feature points.

[0004] The technical solution of the present invention is:

[0005] According to a first aspect of the present invention, a method for extracting ORB feature points under weak lighting conditions by introducing image enhancement and adaptive thresholding is provided, comprising the following steps:

[0006] Step 1: Obtain the original low-light image;

[0007] Step 2: Convert the original low-light image from the red, green, and blue (RGB) color space to the hue, saturation, and brightness (HSV) space;

[0008] Step 3: Extract the brightness component V from the HSV image and perform multi-scale algorithm processing on the brightness component V based on Retinex theory;

[0009] Step 4: Use the contrast-limited adaptive histogram equalization algorithm to optimize the brightness component V after being processed by the multi-scale algorithm based on Retinex theory;

[0010] Step 5: Convert the brightness component V and the unprocessed H and S components optimized by the contrast-limited adaptive histogram equalization algorithm back to the RGB color space to obtain the optimized RGB image;

[0011] Step 6: For the optimized RGB image, ORB feature points are extracted according to the ORB feature extraction algorithm; wherein, the threshold used to extract FAST corner points in the ORB feature extraction algorithm adopts an adaptive threshold.

[0012] Furthermore, the determination of the adaptive threshold is specifically as follows: for each pixel point p in the optimized RGB image, the pixel points with the maximum and minimum grayscale values ​​on the circle with the pixel point p as the center point and the radius R are taken as the first pixel points, and the remaining pixel points are taken as the second pixel points; based on the grayscale value of the second pixel point, the local threshold of the pixel point p is obtained; for the optimized RGB image, a preset number of pixel points with the maximum grayscale value and the minimum grayscale value are found from the optimized RGB image, and the global threshold is obtained based on the preset number of pixel points with the maximum grayscale value and the minimum grayscale value; the local threshold and the global threshold are proportionally summed to obtain the adaptive threshold of the pixel point p.

[0013] Furthermore, the calculation formula of the local threshold is as follows:

[0014]

[0015] Where: T1 is the local threshold; α is the adaptive parameter; I(x i ) represents the second pixel x around pixel p i The gray value of represents the average grayscale value of all second pixels, and n is the number of pixels in a circle with a radius of R and a center point of pixel p.

[0016] Furthermore, the expression of the global threshold is:

[0017]

[0018] Where: T2 is the global threshold; β is the adaptive parameter; I maxj , I minj They represent the grayscale values ​​of the j-th largest and j-th smallest pixel in the optimized RGB image, respectively, and J is a preset number.

[0019] Furthermore, the adaptive threshold is expressed as:

[0020] T sum =λ·T1+(1-λ)T2:

[0021] Where λ is an adaptive parameter and 0<λ<1; T1 is the local threshold; and T2 is the global threshold.

[0022] According to a second aspect of the present invention, a device for extracting ORB feature points under weak light conditions by introducing image enhancement and adaptive thresholding is provided, comprising a module of any one of the above-mentioned methods for extracting ORB feature points under weak light conditions by introducing image enhancement and adaptive thresholding.

[0023] The beneficial effects of the present invention are as follows: the present invention introduces an image enhancement module and an adaptive threshold part based on the ORB-SLAM3 framework, thereby improving the positioning accuracy and robustness in a weak light environment. Specifically, an image enhancement module is introduced into the improved framework to improve the quality of the input image under weak light conditions: first, the input image is converted from the RGB color space to the HSV color space to separate the color information from the brightness information. During the enhancement process, only the brightness component V is processed, while the hue component H and the saturation component S remain unchanged. The brightness component V is processed using an MSR algorithm based on the Retinex theory. In order to further optimize the brightness component V, the present invention applies a CLAHE (contrast-limited adaptive histogram equalization) algorithm to process the brightness component V after MSR processing. CLAHE avoids noise amplification and detail loss caused by excessive enhancement by limiting the contrast enhancement degree of local areas while maintaining the overall brightness distribution of the image. Finally, the unprocessed hue component H, saturation component S and the processed brightness component V are fused and converted back to the RGB color space. At the same time, an adaptive threshold fast corner point extraction strategy is proposed, which can dynamically adjust the threshold according to the grayscale value of the image when the lighting changes, thereby improving the accuracy and robustness of feature point extraction, and using non-maximum suppression for further screening and feature point matching, thereby improving the positioning accuracy and robustness of traditional visual SLAM in low-light environments, and overcoming the interference of insufficient lighting environments on traditional SLAM. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a process framework diagram of the present invention;

[0025] Figure 2 Schematic diagram of the multi-scale algorithm based on Retinex theory of the present invention;

[0026] Figure 3 is a schematic diagram of a circle drawn with pixel point p as the center point and radius R;

[0027] Figure 4 This is a comparison chart of the effects of the present invention after enhancement processing in a low-light scene;

[0028] Figure 5 The figure shows the comparison between the estimated trajectory and the actual trajectory of the MH04 sequence of the EUROC dataset when running ORB-SLAM3 and the method of the present invention; Figure 5 (a) is a comparison chart of ORB-SLAM3; Figure 5 (b) is a comparison diagram of the method of the present invention;

[0029] Figure 6The figure shows the comparison between the estimated trajectory and the actual trajectory of the MH05 sequence of the EUROC dataset when running ORB-SLAM3 and the method of the present invention; Figure 6 (a) is a comparison chart of ORB-SLAM3; Figure 6 (b) is a comparison diagram of the method of the present invention;

[0030] Figure 7 Comparison of absolute trajectory errors of the MH04 sequence of the EUROC dataset using the method of the present invention and ORB-SLAM3;

[0031] Figure 8 This is a comparison chart of the absolute trajectory errors of the MH05 sequence of the EUROC dataset run by the method of the present invention and ORB-SLAM3. DETAILED DESCRIPTION

[0032] To make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other in any way.

[0033] Example 1: Figures 1-8 As shown, according to a first aspect of an embodiment of the present invention, a method for extracting ORB feature points under weak lighting conditions by introducing image enhancement and adaptive thresholding is provided, comprising the following steps:

[0034] Step 1: Obtain the original low-light image;

[0035] Step 2: Convert the original low-light image from the red, green, and blue (RGB) color space to the hue, saturation, and brightness (HSV) space;

[0036] Step 3: Extract the brightness component V from the HSV image and process it with a multi-scale algorithm based on Retinex theory to improve the brightness and contrast of the image;

[0037] Step 4: Use the contrast-limited adaptive histogram equalization algorithm CLAHE to optimize the brightness component V after being processed by the multi-scale algorithm based on Retinex theory;

[0038] Step 5: Convert the brightness component V and the unprocessed H and S components optimized by the contrast-limited adaptive histogram equalization algorithm back to the RGB color space to obtain the optimized RGB image;

[0039] Step 6: For the optimized RGB image, ORB feature points are extracted according to the ORB feature extraction algorithm; wherein, the threshold used to extract FAST corner points in the ORB feature extraction algorithm adopts an adaptive threshold.

[0040] Furthermore, visual SLAM under low-light conditions is implemented based on the extracted ORB feature points, including:

[0041] Step 7: The ORB feature points extracted by the ORB feature point extraction method under weak light conditions with the introduction of image enhancement and adaptive threshold are further screened using non-maximum suppression and feature point matching is performed.

[0042] Step 8: Perform subsequent camera pose estimation and map construction based on the successfully matched feature points.

[0043] Furthermore, the multi-scale algorithm based on Retinex theory is specifically as follows: first extract the illumination component of the image retaining the brightness component V, then subtract the illumination component from the image retaining the brightness component V to obtain the reflection component reflecting the intrinsic properties of the image, and use a multi-scale method to process the image, that is, use Gaussian filtering on the image at different scales, and then average the filtering results at different scales to obtain the estimated illumination image, which can be expressed as the following formula:

[0044]

[0045] Where: I(x,y) represents the image with the brightness component V, R(x,y) is the reflection component, represents the convolution operation, F k (x,y) is the Gaussian surround function at the kth scale, W k The weight value of each scale is generally 1 / N; N is the number of scales, and the value of N in the present invention is 3, and: W1=W2=W3=1 / 3.

[0046] Furthermore, the contrast-limited adaptive histogram equalization algorithm specifically divides the image into several non-overlapping patches. The histogram of each patch is then calculated and equalized to enhance contrast within that area. To avoid over-amplifying noise and creating unnatural contrast, a contrast limit parameter is introduced. This parameter clips the peaks in the histogram and redistributes the clipped pixel values ​​to other parts of the histogram, thereby limiting the contrast enhancement within each patch. Finally, bilinear interpolation is used to smooth the pixel values ​​at the boundaries, resulting in an image with enhanced global contrast and richer details.

[0047] Furthermore, the determination of the adaptive threshold is specifically as follows: for each pixel point p in the optimized RGB image, the pixel points with the maximum and minimum grayscale values ​​on the circle with the pixel point p as the center point and the radius R are taken as the first pixel points, and the remaining pixel points are taken as the second pixel points; based on the grayscale value of the second pixel point, the local threshold of the pixel point p is obtained; for the optimized RGB image, a preset number of pixel points with the maximum grayscale value and the minimum grayscale value are found from the optimized RGB image, and the global threshold is obtained based on the preset number of pixel points with the maximum grayscale value and the minimum grayscale value; the local threshold and the global threshold are proportionally summed to obtain the adaptive threshold of the pixel point p.

[0048] Furthermore, the calculation formula of the local threshold is as follows:

[0049]

[0050] Wherein: T1 is the local threshold; α is the adaptive parameter. For example, the adaptive parameter α can be 1, 2, or 3. In the specific simulation process, the present invention takes 1; I(x i ) represents the second pixel x around pixel p i The gray value of represents the average grayscale value of all second pixels, n is the number of pixels in a circle with a radius of R and a pixel point as the center point; R is 3 pixels in the embodiment of the present invention, so the number of pixels in the circle with a radius of 3 and a pixel point as the center point is 16, the number of first pixels is 2, and the number of second pixels is 14.

[0051] Furthermore, the expression of the global threshold is:

[0052]

[0053] Wherein: T2 is the global threshold; β is an adaptive parameter. For example, the adaptive parameter β can be 1, 2, or 3. In the specific simulation process, the present invention takes 1;

[0054] I maxj , I minj They represent the grayscale values ​​of the jth largest and jth smallest pixel in the optimized RGB image, respectively, where J is a preset number and is set to 10 in the embodiment of the present invention;

[0055] Furthermore, the adaptive threshold is expressed as:

[0056] T sum =λ·T1+(1-λ)T2:

[0057] Here, λ is an adaptive parameter, and 0<λ<1, which can be determined according to specific lighting conditions. In the embodiment of the present invention, 1 / 2 is used.

[0058] Furthermore, the non-maximum suppression retains only the feature point with the largest response value in the local neighborhood of each feature point, and suppresses other feature points with smaller response values, and calculates the response size (score function) V for each detected feature point, where V is defined as the sum of the absolute deviations of the pixel point p and the 16 pixels around it; considers two adjacent feature points and compares their V values; the point with the lower response value V will be deleted; only the feature point with the local maximum response value is retained as the final feature point, thereby reducing redundancy and improving the quality of feature points.

[0059] According to a second aspect of the present invention, a device for extracting ORB feature points under weak light conditions by introducing image enhancement and adaptive thresholding is provided, comprising a module of the method for extracting ORB feature points under weak light conditions by introducing image enhancement and adaptive thresholding as described above; specifically comprising: a first module for executing step 1: acquiring an original weak light image; a second module for executing step 2: converting the original weak light image from a red, green, and blue (RGB) color space to a hue, saturation, and brightness (HSV) space; a third module for executing step 3: extracting a brightness component V from the HSV image, and performing a multi-scale algorithm based on Retinex theory on the brightness component V. Processing; a fourth module is configured to perform step 4: optimizing the luminance component V processed by the multi-scale algorithm based on Retinex theory using a contrast-limited adaptive histogram equalization algorithm; a fifth module is configured to perform step 5: converting the luminance component V optimized by the contrast-limited adaptive histogram equalization algorithm and the unprocessed H and S components back into the RGB color space to obtain an optimized RGB image; and a sixth module is configured to perform step 6: extracting ORB feature points from the optimized RGB image using an ORB feature extraction algorithm; wherein the threshold used for extracting FAST corner points in the ORB feature extraction algorithm adopts an adaptive threshold. As used above, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the system described in the above embodiment is preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated. For portions not described in detail in each module, please refer to the relevant description of this embodiment.

[0060] According to a third aspect of an embodiment of the present invention, a processor is provided, configured to execute an operation including executing any one of the above-described ORB feature point extraction methods under weak lighting conditions that introduce image enhancement and adaptive thresholding.

[0061] The following uses the publicly available EUROC dataset to evaluate the solution of this embodiment. The EUROC dataset is a key robotics vision dataset provided by the European Center for Robotics, primarily used for the evaluation and research of inertial navigation and visual positioning algorithms. The EUROC dataset contains images, IMU data, camera calibration, and ground truth data collected using a micro aerial vehicle (MAV) in an indoor environment. The dataset provides a rich set of indoor scenes and motion patterns. The EUROC dataset is used to illustrate the positioning of visual SLAM using ORB feature points extracted according to the present invention.

[0062] All experiments were run on a computer with a CPU model of R7-7735H, a memory capacity of 16GB, a GPU model of RTX4060, and a system environment of Ubuntu 18.04.

[0063] Figure 4 The upper part shows a randomly selected low-light scene picture from the EUROC dataset. The effect after processing by steps 1 to 5 of the present invention is as follows: Figure 4 As shown in the lower part, through Figure 4 It can be clearly seen from the lower part that after being processed by the present invention, the overall brightness and local contrast of the image are effectively improved, which helps to restore the details of the image under low light conditions.

[0064] Table 1 Experiments on 4 sequences of the EUROC data set yielded the root mean square error (RMSE), median, standard deviation (SD), and improvement percentages corresponding to the ATE index of the method of the present invention and ORB-SLAM3 (where V1_01_easy_ is a normal lighting environment and the other three sequences are weak lighting environments). Compared with ORB-SLAM3, the present invention achieves significant improvements in weak lighting sequences. For example, the MH_04_difficult and MH_05_difficult sequences have a large number of scenes with insufficient lighting, and their root mean square error (RMSE) increases by 47.45% and 44.08%, respectively. At the same time, the performance of the method of the present invention under normal lighting conditions is not significantly improved compared to ORB-SLAM3; however, the method of the present invention can effectively improve the performance of ORB-SLAM3 in weak lighting scenes.

[0065] Table 1. Absolute trajectory error (ATE) comparison unit (m)

[0066]

[0067] Figure 5 、 Figure 6The figures are respectively the comparison of pose estimation trajectories of MH04 and MH05 sequences of EUROC dataset of the method of the present invention and the ORB-SLAM3 algorithm, where the dotted lines (data-MH-04, data-MH-05) are the real trajectories and the solid lines are the trajectories estimated by the algorithm; Figure 5 (a) Figure 6 (a) Comparison of MH04 and MH05 sequences running for ORB-SLAM3 with the real trajectory. Figure 5 (b) Figure 6 (b) Comparison of the MH04 and MH05 sequences run by the method of the present invention with the real trajectory. Figure 5 、 Figure 6 It can be seen that when the camera enters a darker scene, the effect of ORB-SLAM3 is not good, because insufficient light leads to the failure to extract enough and evenly distributed feature points, resulting in fewer key frames in this area and a large error in the local magnification position. The running trajectory of the method of the present invention is closer to the true trajectory and can maintain a small error even in scenes with insufficient light.

[0068] Figure 7 、 Figure 8 The absolute trajectory error comparison diagrams of the MH04 and MH05 sequences of the EUROC dataset run by the method of the present invention and ORB-SLAM3 are shown in the figure (in the figure, benfaming.txt represents the present invention, and ORBSLAM3.txt represents ORB-SLAM3). Figure 7 、 Figure 8 The absolute trajectory errors of ORB-SLAM3 and the method of the present invention at a specific moment are more intuitively shown. Figure 7 and Figure 8 It can be seen that the absolute trajectory error of the method of the present invention is smaller than that of ORB-SLAM3 in low-light scenarios. The method of the present invention improves the accuracy and robustness of visual SLAM in low-light environments.

[0069] By applying the above technical solution, we can see that the present invention converts the input image from the RGB color space to the HSV color space, separating color information from brightness information. During the enhancement process, only the brightness component V is processed, while the hue component H and saturation component S remain unchanged. The brightness component V is processed using the MSR algorithm based on Retinex theory. The brightness component is then further enhanced using contrast-limited adaptive histogram equalization to process the MSR-processed brightness component V. This method maintains the overall brightness distribution of the image while limiting the degree of contrast enhancement in local areas. The unprocessed hue component H and saturation component S are then fused with the processed brightness component V and converted back to the RGB color space. An adaptive threshold fast corner extraction strategy is also proposed, which dynamically adjusts the threshold based on the image's grayscale value when illumination changes, thereby improving the accuracy and robustness of feature point extraction. The feature points extracted by the adaptive threshold are further filtered using non-maximum suppression. Finally, using the Euroc dataset, the present invention demonstrates that it can maintain good localization accuracy and robustness in low-light scenarios.

[0070] The specific embodiments of the present invention are described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A method for extracting ORB feature points under weak lighting conditions by introducing image enhancement and adaptive threshold, characterized in that: The following steps are involved: Step 1: Obtain the original low-light image; Step 2: Convert the original low-light image from the red, green, and blue (RGB) color space to the hue, saturation, and brightness (HSV) space; Step 3: Extract the brightness component V from the HSV image and perform multi-scale algorithm processing on the brightness component V based on Retinex theory; Step 4: Use the contrast-limited adaptive histogram equalization algorithm to optimize the brightness component V after being processed by the multi-scale algorithm based on Retinex theory; Step 5: Convert the brightness component V and the unprocessed H and S components optimized by the contrast-limited adaptive histogram equalization algorithm back to the RGB color space to obtain the optimized RGB image; Step 6: For the optimized RGB image, ORB feature points are extracted according to the ORB feature extraction algorithm; wherein, the threshold for extracting FAST corner points in the ORB feature extraction algorithm adopts an adaptive threshold; The determination of the adaptive threshold is specifically as follows: For each pixel p in the optimized RGB image, the pixel with the maximum and minimum grayscale values ​​on the circle with radius R and centered at pixel p is used as the first pixel, and the remaining pixels are used as the second pixel. Based on the grayscale value of the second pixel, the local threshold of pixel p is obtained. For the optimized RGB image, a preset number of pixels with the largest grayscale value and a preset number of pixels with the smallest grayscale value are found from the optimized RGB image, and a global threshold is obtained based on the preset number of pixels with the largest grayscale value and the preset number of pixels with the smallest grayscale value; The adaptive threshold of pixel p is obtained by proportionally summing the local threshold and the global threshold; The calculation formula of the local threshold is as follows: Where: T1 is the local threshold; α is the adaptive parameter; I(x i ) represents the second pixel x around pixel p i The gray value of represents the average grayscale value of all second pixels, and n is the number of pixels in a circle with a radius of R and a center point of pixel p; The expression of the global threshold is: Where: T2 is the global threshold; β is the adaptive parameter; I maxj , I minj They represent the grayscale values ​​of the j-th largest and j-th smallest pixel in the optimized RGB image, respectively, and J is a preset number.

2. The ORB feature point extraction method under weak light conditions with image enhancement and adaptive thresholding according to claim 1, characterized in that: The adaptive threshold is expressed as: T sum =λ·T1+(1-λ)T2: Where λ is an adaptive parameter and 0<λ<1; T1 is the local threshold; and T2 is the global threshold.

3. An ORB feature point extraction device under weak light conditions that introduces image enhancement and adaptive threshold, characterized in that: A module comprising the method for extracting ORB feature points under weak lighting conditions by introducing image enhancement and adaptive thresholding as described in any one of claims 1-2.

Images