Image-enhanced visual SLAM methods, systems, media, and devices suitable for complex lighting environments.

Abstract

This invention relates to the field of real-time localization and mapping (SLAM), and discloses an image-enhanced visual SLAM method, system, medium, and device suitable for complex lighting environments. The method includes: preprocessing and classifying the acquired original image of the complex lighting environment; applying different adaptive image enhancement strategies to different image categories; inputting the enhanced image into an improved SuperPoint neural network to obtain an enhanced original feature point confidence map (score map) and determining candidate feature points; dynamically adjusting the feature point confidence threshold based on inter-frame matching feedback to select the final feature points from the candidate points; matching the feature points of the current frame with those of the previous frame, performing matching filtering and geometric consistency verification; and improving feature matching estimation accuracy by quality ranking and dispersed point selection to suppress degradation caused by near-collinearity or local clustering of four points; and inputting the selected matching pairs into the SLAM system for front-end tracking and mapping.

Description

Technical Field

[0001] This invention relates to the field of real-time positioning and mapping technology, and in particular to an image-enhanced visual SLAM method, system, medium, and device suitable for complex lighting environments. Background Technology

[0002] SLAM (Simultaneous Localization and Mapping) is a technique that simultaneously performs localization and map building, commonly used for navigation and environmental perception in devices such as drones and robots. ORB-SLAM2, as a classic feature-point SLAM system, exhibits high accuracy and stability in structured environments. However, in practical applications, images captured by cameras are often affected by changes in lighting, overexposure, or shadow occlusion, leading to a decrease in the accuracy of front-end feature extraction, thus impacting the relocalization accuracy and trajectory robustness of the entire system.

[0003] In recent years, image enhancement techniques such as histogram equalization, Retinex, and gamma algorithms have been introduced to improve image quality. However, these techniques have fixed parameters and lack adaptability, failing to adapt to the structural fidelity of different regions of the image. They are prone to introducing false features, amplifying noise, or destroying real textures, making it difficult to meet the stability requirements of SLAM in complex lighting scenarios. For example, global enhancement in low-light environments amplifies sensor noise, leading to distortion of feature point descriptors, such as a high ORB feature matching error rate in underground parking garage scenes. Forced enhancement of high-exposure images destroys the residual details in overexposed areas, causing monocular depth estimation errors. Existing solutions lack illumination perception capabilities, and in scenarios with sudden changes in lighting, such as tunnel entrances and exits, fixed processing parameters can lead to discontinuities in feature point tracking. In addition, the deep learning method SuperPoint introduces an end-to-end feature point extraction mechanism, possessing strong expressive power and structural stability. The network output includes keypoint locations, descriptors, and corresponding confidence maps (score maps), where the score value of each pixel in the score map represents the probability that it is a keypoint. However, under actual lighting conditions, especially in low-light or overexposed images, the overall score value is low, and local maxima often fail to accurately reflect the true corner structure, leading to traditional non-maximum suppression (NMS) mistakenly suppressing potentially valid feature points. Furthermore, SuperPoint's default fixed confidence threshold cannot adapt to the dynamic changes in score distribution across images with varying lighting conditions. In extremely dark or bright images, it may incorrectly delete originally valid feature points or retain low-quality noise points. During image matching, external factors (such as lighting variations and noise) often cause mismatches of feature points, thus affecting matching accuracy. Summary of the Invention

[0004] To address the aforementioned problems, the purpose of this invention is to provide an image enhancement visual SLAM method, system, medium, and device suitable for complex lighting environments, which can improve the system's robustness against dynamic lighting interference while simultaneously enhancing localization and mapping accuracy.

[0005] To achieve the above objectives, in a first aspect, the technical solution adopted by the present invention is as follows: an image enhancement visual SLAM method suitable for complex lighting environments, comprising: preprocessing the acquired original image of the complex lighting environment into a grayscale image; determining the lighting environment of the original image based on the mean and standard deviation of the pixel grayscale values ​​of the grayscale image; classifying the original image based on the lighting environment determination result; applying different adaptive image enhancement strategies to enhance different categories of original images; inputting the enhanced image into an improved SuperPoint neural network; estimating the confidence of the enhanced original feature points through image structure strength estimation; and obtaining the enhanced original feature point confidence map score. The system maps and identifies candidate feature points. Based on the matching feedback between image frames, it dynamically adjusts the feature point confidence threshold to achieve adaptive control over the quantity and quality of feature points, selecting the final feature points from the candidate points. After feature point extraction, it performs feature point matching between the current frame and the previous frame, outputting the keypoint index pairs determined to be matched in the two images and their confidence scores. It then performs matching filtering and geometric consistency verification. Through quality sorting and dispersed point selection, it suppresses degradation caused by near-collinearity or local clustering of four points without increasing computation, steadily improving the feature matching estimation accuracy. The selected matching pairs are then input into the SLAM system for front-end tracking and mapping.

[0006] Furthermore, the lighting environment of the original image is determined based on the pixel grayscale mean and standard deviation of the grayscale image. The original image is then classified based on the lighting environment determination result, including: calculating the grayscale mean and standard deviation std of the image based on the current input image size and pixel grayscale values; performing preliminary lighting environment classification of the current image based on the calculated grayscale mean and standard deviation std, including: classifying the current image into low-contrast images and other images based on the standard deviation std: if the standard deviation std is lower than a set threshold, it is a low-contrast image; otherwise, it is another image; classifying other images into dark images, normal images, and high-exposure images based on the grayscale mean: if the grayscale mean is less than the minimum value of a first set interval range, it is a dark image; if the grayscale mean is within the first set interval range, it is a normal image; if the grayscale mean is greater than the maximum value of the first set interval range, it is a high-exposure image.

[0007] Furthermore, different adaptive image enhancement strategies are applied to different categories of images for enhancement processing, including: normal images are directly input into the improved SuperPoint neural network; low-contrast images undergo secondary CLAHE enhancement processing;

[0008] Dark image enhancement is performed by: extracting the dark regions from the dark image; applying SSR enhancement to these dark regions to obtain the enhanced image; and using the Sobel operator to calculate the sum of gradient magnitudes S in the dark regions of the original image. orig For the same dark area after SSR enhancement, calculate the sum of gradient magnitudes S. SSR According to S SSR With S orig The ratio determines the gain ratio R for detail in dark areas. g Median filtering is used to denoise dark areas. A noise parameter is calculated from the denoised image. Based on the noise parameter and a threshold, it is determined whether to perform a detail gain ratio assessment. The detail gain ratio R in dark areas is then calculated. g Determine if R g If the value is greater than the maximum value of the second set interval, skip the CLAHE enhancement process and output the image directly; if R g If the value is less than or equal to the maximum value of the second set interval range, and greater than the minimum value of the second set interval range, then perform Level 1 CLAHE enhancement processing; if the value is less than or equal to the minimum value of the second set interval range, then perform Level 2 CLAHE enhancement processing.

[0009] Enhancement processing is performed on overexposed images, specifically by calculating the initial highlight ratio. ,in accordance with Adaptive selection of power exponent Used for Gamma darkening of highlights; based on the selected power factor. Gamma correction is applied to the input raw image to obtain the gamma-corrected image. In the image after Gamma correction Redefine the specular mask, statistically analyze the gray-level histogram of the specular mask region, and calculate the normalized probability distribution. p i The information entropy E of the highlight region is then obtained. Based on the information entropy E, a regionalized CLAHE decision is made: if E is greater than the maximum value of the third set interval, then a three-level CLAHE enhancement process is performed; if E is less than or equal to the maximum value of the third set interval but greater than the minimum value of the third set interval, then a four-level CLAHE enhancement process is performed only in the non-highlight region; if E is less than or equal to the minimum value of the third set interval, then CLAHE is skipped, and the image after Gamma correction is used directly. To enhance the results.

[0010] Furthermore, edge overshoot detection is performed on the image after CLAHE enhancement. Specifically, strong edge filtering is performed on the image to extract strong edge pixels, forming a set of strong edges, and the image's edge strength is calculated. x Directional gradient G x , y Directional gradient G y Gradient magnitude G and gradient direction; multi-point sampling along the image normal direction, skipping any pixel if it exceeds the boundary; scalar sample I is obtained by bilinear interpolation based on the sampling points. k Based on scalar sample I k Perform adjacent difference and monotonicity determination to determine whether strong edge pixels satisfy monotonicity violation; based on scalar sample I k Determine the overshoot-free baseline, and based on the overshoot-free baseline, determine the amplitude of the main body and the deviation of the local peak to obtain the overshoot ratio. r ovr If the overshoot ratio r ovr If the value is greater than or equal to the set value and the monotonicity is violated, the local peak or valley is considered to have significant contrast relative to the main body, and the overshoot pixel count is 1; otherwise, the overshoot pixel count is 0. The overshoot rate OVR is calculated based on the overshoot pixel count and the strong edge pixel count. If the overshoot rate OVR is greater than the set amplitude, the contrast limit of the original CLAHE is lowered and CLAHE enhancement processing is performed again, and edge overshoot detection is checked again. If the overshoot rate OVR is less than or equal to the set amplitude, the enhanced image is output.

[0011] Furthermore, the enhanced image is input into an improved SuperPoint neural network. The confidence of the original feature points is enhanced by estimating the image structure strength, resulting in an enhanced original feature point confidence map (score map). Candidate feature points are then identified, including: introducing a structure-aware enhancement mechanism into the SuperPoint neural network to form an improved SuperPoint neural network; inputting the enhanced image into the improved SuperPoint neural network to output the original feature point confidence map (score map); and calculating the structure-aware score based on the original feature point confidence map (score map). Score through structure perception The score map is enhanced to obtain the enhanced score map, and candidate feature points are obtained by optimizing the non-maximum suppression strategy.

[0012] Furthermore, the feature point confidence threshold is dynamically adjusted based on the matching feedback between image frames to achieve adaptive control over the quantity and quality of feature points. The final feature points are then selected from the candidate feature points, including: obtaining the number of matching points M from the previous frame image; and using the ORB-SLAM2 Tracking module to perform feature point matching between the current frame image and the previous frame image to obtain the number of successfully matched points M between the previous and current frames. t Based on the ideal number of matches and the number of successful matches M t The matching deviation ΔM is obtained; based on the matching deviation The threshold adjustment coefficient is calculated, and the mean μ and standard deviation σ of the enhanced score map are calculated. The confidence dynamic threshold T is obtained through the threshold adjustment coefficient, mean μ and standard deviation σ. The final feature points are selected from the candidate feature points by adaptively adjusting the confidence dynamic threshold T.

[0013] Furthermore, after feature point extraction, a neural network-based feature point matching process is performed between the current frame and the previous frame. This outputs the keypoint index pairs determined to be matched in both images, along with their confidence scores. Match filtering and geometric consistency verification are then performed. Specifically, a mismatch elimination method is used for matching filtering and geometric consistency verification, including: assigning a quality factor to each pair of matching points (a quality factor integrating distance ratio, confidence score, and diversity); sorting all matching points in descending order of quality factor and selecting the top n pairs of matching points into a candidate set; combining each pair of n matching points in the candidate set to form a four-point group; calculating the total quality of the four-point group; selecting the top K groups of four-point groups with the highest quality in descending order of total quality; and estimating the homography matrix H of the highest quality four-point group using normalized least squares, calculating the reprojection coordinates, and applying a threshold value. Determine interior points; after determining interior points, count the number of interior points and compare the number of interior points with a preset interior point threshold T. n Comparison: If the number of interior points exceeds the threshold T n If the match is successful, the homography matrix H is updated, and all inlier information is recorded; otherwise, the four feature points ranked second in matching quality are selected, and the iteration continues until a homography matrix that meets the requirements is found; when the number of inliers exceeds the threshold... Update H periodically, where N is the total number of matches and µ is a constant between 0 and 1; terminate the iteration when the termination condition is met, and return the optimal homography matrix H and all matching point pairs.

[0014] Secondly, the technical solution adopted by this invention is as follows: an image enhancement visual SLAM system suitable for complex lighting environments, comprising: an image enhancement preprocessing module, which preprocesses the acquired original image of the complex lighting environment into a grayscale image, determines the lighting environment of the original image based on the pixel grayscale mean and standard deviation, classifies the original image based on the lighting environment determination result, and applies different adaptive image enhancement strategies to enhance different categories of original images; and a candidate point extraction module, which inputs the enhanced image into an improved SuperPoint neural network, estimates the confidence of the enhanced original feature points through image structure strength estimation, and obtains the enhanced original feature point confidence map sco. The system performs several remap operations and identifies candidate feature points. A feature point extraction module dynamically adjusts the feature point confidence threshold based on inter-frame matching feedback, enabling adaptive control of the number and quality of feature points and selecting the final feature points from the candidate pool. A feature matching module performs feature point matching between the current and previous frames after feature point extraction, outputting the index pairs of key points that are determined to be matched in both images, along with their confidence scores. It also performs matching filtering and geometric consistency verification. Through quality sorting and dispersed point selection, it suppresses degradation caused by near-collinearity or local clustering of four points without increasing computation, thus steadily improving the accuracy of feature matching estimation. The selected matching pairs are then input into the SLAM system for front-end tracking and mapping.

[0015] Thirdly, the technical solution adopted by the present invention is: a computer-readable storage medium for storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0016] Fourthly, the technical solution adopted by the present invention is: a computing device comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described above.

[0017] The present invention has the following advantages due to the adoption of the above technical solutions:

[0018] 1. This invention constructs an adaptive image enhancement mechanism suitable for visual SLAM by fusing image illumination classification and region structure perception analysis. It can determine the category of a grayscale image based on its mean and standard deviation, and dynamically schedule the enhancement intensity and applicable region of CLAHE by combining two structural complexity indicators: "detail gain ratio" and "highlight information entropy." This effectively restores details in dark areas, avoids false enhancement of highlight areas, improves the overall texture perceptibility of the image, and provides a stable input foundation for subsequent feature extraction, adapting to complex lighting environments.

[0019] 2. Based on image enhancement, this invention introduces the SuperPoint neural network into the SLAM feature extraction module and designs a dynamic confidence threshold adjustment mechanism based on inter-frame feature point matching feedback. By evaluating the changes in the number of matching feature points between consecutive frames in real time, the confidence screening criteria for feature points are dynamically adjusted, so that the quantity and quality of feature points remain balanced under different lighting conditions, effectively improving the stability, distribution uniformity, and matching accuracy of feature points.

[0020] 3. This invention introduces the structural strength information of the image into the feature point confidence modeling process, and integrates the Sobel gradient and the SuperPoint confidence map to construct a new confidence map after structural enhancement, which serves as the basis for dynamic threshold and non-maximum suppression (NMS) adjustment. This makes feature extraction pay more attention to regions with structural edge features, thereby improving the key point response rate in low light and low contrast images and reducing the false matching rate and the probability of noise.

[0021] 4. This invention employs a mismatch elimination method to achieve efficient and accurate feature matching in SLAM systems under complex lighting conditions, thereby improving feature matching accuracy. Attached Figure Description

[0022] Figure 1 This is a flowchart of an image enhancement visual SLAM method applicable to complex lighting environments in an embodiment of the present invention;

[0023] Figure 2 This is a flowchart illustrating the discrimination process based on image mean and standard deviation in an embodiment of the present invention.

[0024] Figure 3 These are schematic diagrams illustrating image enhancement under different lighting conditions in embodiments of the present invention;

[0025] Figure 4 This is a detailed schematic diagram of low-light image enhancement in an embodiment of the present invention;

[0026] Figure 5 This is a detailed schematic diagram of high-exposure image enhancement in an embodiment of the present invention;

[0027] Figure 6 This is a flowchart of the improved SuperPoint adaptive threshold filtering process in an embodiment of the present invention;

[0028] Figure 7 This is a detailed schematic diagram of the feature point mismatch removal process in an embodiment of the present invention. Detailed Implementation

[0029] To address the problems of low matching accuracy and false suppression of potential effective feature points by traditional non-maximum suppression (NMS) in existing technologies, this invention proposes an image-enhanced visual SLAM method, system, medium, and device suitable for complex lighting environments. It includes: constructing an adaptive image enhancement mechanism suitable for lighting by fusing image lighting classification and region structure perception analysis; dynamically adjusting the feature point confidence threshold according to the scene lighting complexity during feature point extraction, employing a dynamic confidence threshold adjustment mechanism based on inter-frame feature point matching feedback, and a non-maximum suppression optimization strategy based on structure enhancement. This invention achieves an adaptive balance between feature point quality and quantity in highly dynamic lighting scenes; simultaneously, it improves feature point matching accuracy by employing a false match removal method. This invention is applicable to complex lighting scenes, improving the localization robustness of SLAM systems in complex lighting environments.

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0031] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0032] In one embodiment of the present invention, an image enhancement visual SLAM method suitable for complex lighting environments is provided. In this embodiment, as... Figure 1 As shown, the method includes the following steps:

[0033] 1) After preprocessing the original image of the complex lighting environment into grayscale, the lighting environment of the original image is determined based on the mean and standard deviation of the pixel grayscale value of the grayscale image. The original image is then classified based on the lighting environment determination result, and different adaptive image enhancement strategies are applied to enhance the original images of different categories.

[0034] 2) Input the enhanced image into the improved SuperPoint neural network, estimate the confidence of the enhanced original feature points through image structure strength, obtain the enhanced original feature point confidence map score map, and determine candidate feature points;

[0035] 3) Dynamically adjust the feature point confidence threshold based on the matching feedback between image frames to achieve adaptive control of the number and quality of feature points, and select the final feature points from the candidate feature points;

[0036] 4) After feature point extraction, feature point matching between the current frame image and the previous frame image is performed based on the neural network. The index pairs of key points that are judged to be matched in the two images and their confidence scores are output. Matching and geometric consistency verification are performed. Through quality ranking and dispersed point selection, the degradation caused by near collinearity or local clustering of four points is suppressed without increasing the computation, and the feature matching estimation accuracy is steadily improved. The selected matching pairs are input into the SLAM system for front-end tracking and mapping.

[0037] In step 1) above, the lighting environment of the original image is determined based on the mean and standard deviation of the pixel grayscale values. The original image is then classified based on the lighting environment determination result, such as... Figure 2 As shown, it includes the following steps:

[0038] 1.1) Based on the current input image size and pixel grayscale values, calculate the mean and standard deviation of the grayscale values ​​of the image.

[0039] Specifically, let the current input image size be Z×W, and the pixel grayscale value be denoted as I(x, y), where Z is the length of the image, W is the width of the image, and... x , y Let ) represent the pixel coordinates in the image; then its gray-level mean μ and standard deviation σ are defined as follows:

[0040] The formula for the grayscale image mean is:

[0041]

[0042] The formula for the standard deviation (std) of a grayscale image is:

[0043]

[0044] 1.2) Based on the calculated mean and standard deviation (std) of gray levels, perform preliminary lighting environment classification on the current image, including the following steps:

[0045] 1.2.1) Divide the current image into low-contrast images and other images based on the standard deviation std: if the standard deviation std is lower than the set threshold, it is a low-contrast image; otherwise, it is another image.

[0046] The specific division method is as follows:

[0047]

[0048] 1.2.2) Divide other images into dark images, normal images, and high-exposure images based on the mean gray level: if the mean gray level is less than the minimum value of the first set interval, it is a dark image; if the mean gray level is within the first set interval, it is a normal image; if the mean gray level is greater than the maximum value of the first set interval, it is a high-exposure image.

[0049] In this embodiment, the first set interval unit is [min threshold value, max threshold value].

[0050] The specific division method is as follows:

[0051]

[0052] In step 1) above, if Figure 3 As shown, different adaptive image enhancement strategies are applied to images of different categories, including the following steps:

[0053] 1.3) Normal images are directly fed into the improved SuperPoint neural network;

[0054] 1.4) Perform secondary CLAHE enhancement processing on low-contrast images;

[0055] In this embodiment, the secondary CLAHE enhancement process is: CLAHE process with a block size of Tile8×8 and a contrast limit of ClipLimit=40.

[0056] 1.5) Enhance the dark image, such as... Figure 4 As shown, the specific steps include:

[0057] 1.5.1) Obtain the dark areas in the dark image, and perform SSR enhancement on the dark areas to obtain the enhanced image;

[0058] Specifically, the dark region D is defined as: ,in, Represents the grayscale value of a pixel, ( x , y () represents the pixel coordinates.

[0059] Single-scale Retinex (SSR) is:

[0060]

[0061] In the formula, Set the Gaussian kernel (standard deviation) ); prevent Overflow; I(x,y) is abbreviated as I, representing the pixel value of the original input image at coordinates (x,y); This represents the output pixel value at coordinates (x, y) after single-scale Retinex enhancement.

[0062] The output is then linearly normalized to .

[0063] 1.5.2) The Sobel operator is used to calculate the sum of gradient magnitudes S in the dark areas of the original image. orig For the same dark area after SSR enhancement, calculate the sum of gradient magnitudes S. SSR According to S SSR With S orig The ratio determines the gain ratio R for detail in dark areas. g .

[0064] Specifically, the Sobel operator formula is as follows:

[0065] Horizontal convolution kernel :

[0066]

[0067] Vertical convolution kernel :

[0068]

[0069] Gradient magnitude:

[0070]

[0071] The sum of gradient magnitudes for the original dark area is calculated as follows:

[0072]

[0073] Calculate the sum of gradient magnitudes in the same region after SSR processing:

[0074]

[0075] In the formula, This represents the original gradient magnitude of the dark area. This refers to the gradient magnitude of the dark area after SSR processing, which is identical to the original dark area.

[0076] Dark area detail gain for:

[0077]

[0078] 1.5.3) To ensure smooth local structures and suppress the contribution of edges to the residuals without introducing mean shift, median filtering is used to denoise dark areas. A noise parameter is calculated from the denoised image, and a threshold is used to determine whether to perform detail gain ratio (R) assessment. The R value for detail gain ratio in dark areas is then calculated. g Determine if R g If the value is greater than the maximum value of the second set interval, skip the CLAHE enhancement process and output the image directly; if R g If the value is less than or equal to the maximum value of the second set interval range, and greater than the minimum value of the second set interval range, then perform Level 1 CLAHE enhancement processing; if the value is less than or equal to the minimum value of the second set interval range, then perform Level 2 CLAHE enhancement processing.

[0079] To prevent random noise in dark areas from being mistakenly magnified as details during subsequent enhancement, this invention sets a "noise gate" before entering the dark area enhancement decision. Specifically, the input grayscale image is first smoothed using a small window mid-range filter to obtain a reference image. (window , ,default The median operator replaces each pixel with the neighborhood median, which can suppress isolated outliers without introducing mean shift; the residual map is obtained by subtracting the original image from the reference image. and masking in the dark area The median absolute deviation (MAD) of the internal statistical residuals constitutes a robust measure of noise intensity. Considering that dark-area readout noise can be approximated as a zero-mean Gaussian distribution in engineering practice, the noise standard deviation can be taken as... And set an upper limit threshold for noise. (8-bit, default 8 grayscale levels). When If the signal-to-noise ratio in the dark area is deemed insufficient, no further enhancement will be performed. If the noise is deemed controllable, the process continues with structural criteria such as "detail gain ratio" to determine whether and how to perform local contrast enhancement. The aforementioned noise gate effectively suppresses false textures and edges in dark areas, ensuring the effectiveness and stability of the enhanced output for SLAM feature extraction.

[0080] In this embodiment, the first-level CLAHE enhancement process is a CLAHE process with a tile size of 16×16 and a contrast limit of ClipLimit=30. The second-level CLAHE enhancement process is a CLAHE process with a tile size of 8×8 and a contrast limit of ClipLimit=40.

[0081] In this embodiment, Noise estimation: =M(I)(window k×k, k∈{3,5}, default k=3), This is a median operator that replaces each pixel with the median of its neighborhood.

[0082] The formula for the residual ("approximate noise") is:

[0083]

[0084] In the Dark Zone superior, The main component should be noise, and the real texture is greatly weakened by median filtering.

[0085] MAD (Median Absolute Deviation) is:

[0086]

[0087] With zero-mean residuals, it can be simplified to For Gaussian noise , ,therefore .

[0088] Noise standard deviation estimation:

[0089]

[0090] therefore, Set an upper limit for noise. (default (Gray level). If Then reduce the subsequent enhancement intensity (to avoid amplifying noise). Dark areas have inherently high noise; direct enhancement easily amplifies noise and false edges. The image is then further evaluated for detail gain ratio.

[0091] In this embodiment, the second set interval range is (1.8, 2.5). That is, if 2.5, skips CLAHE processing and outputs the image directly; if 1.8 2.5, with a block size of Tile16. 16. Contrast limit ClipLimit=30 for CLAHE; if Version 1.8 executes with a block size of Tile8. 8. Contrast Limit (ClipLimit) 40's CLAHE.

[0092] 1.6) Enhance overexposed images, such as... Figure 5 As shown, the specific steps include:

[0093] 1.6.1) Calculate the initial specular ratio ,in accordance with Adaptive selection of power exponent Used for Gamma darkening of highlights;

[0094] Specifically, calculate the initial highlight ratio. ;in This represents the number of highlight pixels in the original image, i.e., the number of pixels with a grayscale value greater than 240. N represents the total number of pixels in the original image.

[0095] A higher highlight ratio indicates that more of the high-end dynamic range is being used, so a larger highlight ratio should be used. The value can more effectively compress highlight areas and reclaim grayscale space that can be used for enhancement. The segmentation rules are as follows:

[0096]

[0097] 1.6.2) Based on the selected power exponent For the input raw image Perform Gamma correction to obtain the Gamma-corrected image. In the corrected image Redefine the specular mask, statistically analyze the gray-level histogram of the specular mask region, and calculate the normalized probability distribution. p i This leads to the information entropy E of the highlight region;

[0098] Specifically, for the input image Perform Gamma correction:

[0099]

[0100] in Used to darken highlights. First, edit the original image... Convert to floating point and normalize to Perform exponentiation again to prevent overflow due to excessively large integers. After the operation, multiply by 255 and restore to the original value. .

[0101] exist Redefining the specular mask U:

[0102]

[0103] In the formula, =240 represents a stable statistic; first, [the following is an example]... Perform a 3×3 opening operation followed by a 5×5 closing operation to remove isolated noise and fill small holes.

[0104] In this embodiment, the normalized probability distribution is calculated. This refers to the frequency of occurrence of gray levels i ∈ [241, 255]. The "recoverability" of the highlight region is quantified as information entropy. When the information entropy E of the highlight region is large, it indicates that the gray level distribution within the highlight is relatively dispersed and structural details still exist. When the information entropy E of the highlight region is small and close to 0, it indicates that the highlight region is close to a single saturation peak and lacks recoverable details.

[0105] Gray-scale histogram probability:

[0106]

[0107] Where i∈[0,255], =Number of pixels with grayscale value "i" For specular mask The total number of pixels within.

[0108] Calculate the information entropy of the highlight region :

[0109]

[0110] In the formula, To prevent logarithmic overflow.

[0111] 1.6.3) Regionalized CLAHE decision based on the information entropy E of the highlight region: If E is greater than the maximum value of the third set interval range, then perform three-level CLAHE enhancement processing; if E is less than or equal to the maximum value of the third set interval range but greater than the minimum value of the third set interval range, then perform four-level CLAHE enhancement processing only in the non-highlight region; if E is less than or equal to the minimum value of the third set interval range, then skip the CLAHE processing and directly use the image after Gamma correction. To enhance the results.

[0112] In this embodiment, the third-level CLAHE enhancement process is a CLAHE process with a tile size of 64×64 and a contrast limit of 25. The fourth-level CLAHE enhancement process is a CLAHE process with a tile size of 32×32 and a contrast limit of 35.

[0113] The third defined interval is (2.58, 3.32). The specific regionalized CLAHE decision is as follows:

[0114] 1.6.3.1) When At that time, a three-level CLAHE enhancement process was applied to the entire image. Since there were already many recoverable distributions within the highlights, the three-level CLAHE enhancement process could improve the overall contrast while suppressing overshoot.

[0115] 1.6.3.2) When At that time, only in the non-highlight area Perform four levels of CLAHE enhancement processing; in practice, CLAHE can be applied to the entire image first to obtain the image. Then synthesize according to the mask:

[0116]

[0117] In the formula, This represents the non-highlight mask. Specifically, at the pixel level, a value of 1 indicates a non-highlight area, and a value of 0 indicates a highlight area. It represents element-wise (pixel) multiplication, also known as the Hadamard product, which multiplies each corresponding pixel of the image while keeping the size unchanged; This represents the result of applying CLAHE to the entire image to enhance it.

[0118] Specifically, using the above formula, the entire image is first subjected to CLAHE to obtain... (Enhance global and local contrast), then use a non-highlight mask to retain the enhancement of CLAHE only in non-highlight areas, while the highlight areas retain the original gamma correction or other de-highlighting processing. Finally, the images are composited pixel by pixel.

[0119] Under partially recoverable conditions, prioritizing enhancement budgets for non-highlight areas can avoid creating false details in hard saturation areas.

[0120] 1.6.3.3) Skip Enhancement: When At that time, skip CLAHE and directly use As an enhancement result, in this case the highlights are almost perfectly saturated and the histogram is highly concentrated; further enhancement would only introduce false edges and noise.

[0121] In this embodiment, when the 99th percentile of the enhancement result To further suppress extremely bright pixels, the following compression is performed:

[0122]

[0123] The function is monotonic and its derivative decreases at high gray levels, which can gently suppress "white drift" and reduce the risk of halo. This represents the image after Gamma correction and regionalization (CLAHE) according to the information entropy criterion. This indicates the output image after highlighting and compression.

[0124] In this embodiment, to avoid artifacts introduced by excessive CLAHE enhancement, edge overshoot detection is performed on the image after CLAHE enhancement. Specifically, this includes the following steps:

[0125] (1) Perform strong edge filtering on the image to extract strong edge pixels and form a strong edge set. and calculate the image x Directional gradient G x , y Directional gradient G y Gradient magnitude G and gradient direction Strong edges refer to edge pixels whose gradient magnitude is greater than a set value and have undergone non-maximum suppression (NMS). They are a subset of "all edges," representing the group of edge points with the strongest contrast and most reliable localization. Specifically, strong edge pixels are first extracted, and Sobel and Non-Maximum Suppression (NMS) are used to retain only these local maxima along the gradient direction, compressing the edges into a single-pixel-wide centerline to achieve high localization accuracy. This results in a single-pixel-wide edge centerline. Then, a quantile threshold is used to filter strong edges, selecting the strongest approximately 10% of pixels from the candidate edges. It is the 90th percentile of the gradient magnitude G, which is the threshold at which 90% of the samples are not greater than it and about 10% of the samples are greater than it after sorting all the G values ​​involved in the statistics from smallest to largest.

[0126]

[0127] The enhanced image to be detected is set. : ; Let G be the gradient magnitude.

[0128] The Sobel gradient is:

[0129]

[0130] gradient magnitude gradient direction .

[0131] Strong edge set :satisfy The edge center pixel.

[0132] (2) Perform multi-point sampling along the image normal direction. If any sampling point exceeds the boundary, skip that pixel; perform bilinear interpolation based on the sampling points to obtain I. k .

[0133] In this embodiment, 7-point sampling is used as an example for explanation. Specifically, 7-point sampling is performed in the normal direction, for each... Along the normal 7-point directional sampling:

[0134]

[0135] In the formula, This is the kth sampling point; I represents the bilinear interpolation intensity at that location in image J; k It is an interpolated scalar sample, using this sample I k To construct a one-dimensional profile and detect overshoot. If any sampling point goes out of bounds, skip that pixel.

[0136] Wherein, the unit vector of the normal direction (along the direction of the fastest change in intensity):

[0137]

[0138] Normal sampling location and intensity: for each Sampling in the normal direction:

[0139]

[0140] Using bilinear interpolation ;in Located at the center of the edge.

[0141] (3) Based on I k Perform adjacent difference and monotonicity determination to determine whether strong edge pixels satisfy monotonicity violation.

[0142] In this embodiment, monotonicity is first determined, and the pixel is marked as "overshoot hit"; then, amplitude criteria are applied (the local peak deviation and the main amplitude are calculated using the linear baselines at both ends, and the overshoot ratio is determined); when both structure and amplitude are satisfied, the strong edge pixel is finally determined to be an "overshoot pixel". Specifically, the adjacent difference is calculated. :

[0143]

[0144] Define a thresholded sign function. :

[0145]

[0146] Monotonicity violation criterion: Existence make If so, then an "overshoot hit" is recorded at the strong edge pixel.

[0147] "Overshoot hit" is a marker that satisfies monotonicity destruction; "record 1 overshoot hit" is not the final result, but only an intermediate step. The subsequent judgment needs to be combined with the overshoot amplitude detection in step (4), and the local peak deviation and the main amplitude are calculated using the linear baselines at both ends and the overshoot ratio is calculated; when monotonicity and amplitude are satisfied at the same time, the strong edge pixel is finally judged as an "overshoot pixel".

[0148] =1 indicates the guard threshold, which is used to treat small fluctuations or noise as "flat" (denoted as 0).

[0149] (4) Based on I k Determine the overshoot-free baseline, and based on the overshoot-free baseline, determine the amplitude of the main body and the deviation of the local peak to obtain the overshoot ratio. r ovr If the overshoot ratio r ovr If the value is greater than or equal to the set value and the monotonicity is violated, then the local peak or valley is considered to have significant contrast relative to the main body, and the number of overshoot pixels is 1; otherwise, the number of overshoot pixels is 0.

[0150] Specifically, the linear interpolation formed by the two endpoints is used as the overshoot-free baseline. :

[0151]

[0152] Define the deviation of the main body's amplitude. Deviation from local peaks :

[0153]

[0154] Define overshoot ratio :

[0155]

[0156] in, Magnitude criterion: If (default It is believed that local peaks or valleys have significant contrast with the main body.

[0157] Among them, pixel-level overshoot determination is performed. Define indicator variables:

[0158]

[0159] (5) Based on the number of overshoot pixels and the number of strong edge pixels Calculate the overshoot ratio OVR and perform edge overshoot detection: if the overshoot ratio OVR is greater than the set amplitude, reduce the ClipLimit of the original CLAHE process and perform CLAHE enhancement processing again, and check for edge overshoot again; if the overshoot ratio OVR is less than or equal to the set amplitude, output the enhanced image.

[0160] Specifically, calculate the overshoot rate, assuming the number of pixels on the strong edge. Overshooting pixel count OVR is defined as:

[0161]

[0162] In this embodiment, the amplitude is set to 5%. The specific edge overshoot detection judgment is as follows: if OVR > 5%, the ClipLimit of the original CLAHE method is reduced by 10%, and CLAHE processing is performed again, followed by a re-judgment of edge overshoot detection; if OVR ≤ 5%, the final enhanced image is output. In high-exposure image enhancement, if the image has not undergone CLAHE processing, then... After reducing the value by 0.2, image enhancement processing was performed again. OVR is defined as the "contrast overshoot rate" within 3 pixels of a pixel near a strong edge.

[0163] In step 2) above, such as Figure 6 As shown, the enhanced image is input into the improved SuperPoint neural network. The confidence of the original feature points is enhanced by estimating the image structure strength, resulting in an enhanced original feature point confidence map (score map). Candidate feature points are then identified, allowing regions with obvious structures to be recognized as key points. The process includes the following steps:

[0164] 2.1) Introduce a structure-aware enhancement mechanism into the SuperPoint neural network to form an improved SuperPoint neural network for feature point extraction;

[0165] 2.2) In the feature point extraction stage, the enhanced image is input into the improved SuperPoint neural network, which outputs the original feature point confidence map (score map).

[0166] 2.3) Calculate the structure-aware score based on the original feature point confidence map. Score through structure perception The score map is enhanced to obtain an enhanced score map, and candidate feature points are obtained by optimizing the non-maximum suppression strategy. In this embodiment, the enhanced image input into the SuperPoint neural network generates a two-dimensional heatmap score map showing the possible confidence of each pixel as a key point. By introducing a structure-aware enhancement mechanism, the SuperPoint neural network is improved, and the score distribution is corrected by analyzing the image structure strength C(x,y).

[0167] Original score map: Formula for calculating structural perception score:

[0168]

[0169] in, The enhanced structure-aware score is used for NMS; This represents the original SuperPoint output score map, at pixel positions. The confidence score of the feature points on the feature point ranges from 1 to 10. It represents the probability that the SuperPoint network considers this location to be a feature point; λ represents the image structure contrast (gradient), normalized to [0,1]; λ is the structure-aware weight, controlling the influence of structural information on the final score. In the structure-aware non-maximum suppression module, the recommended value range for the structure contrast enhancement coefficient λ is 0.5–2.0, preferably 1.0. This coefficient is used to adjust the weight of local structural information in the confidence weighting.

[0170]

[0171] in, These are the image gradients (Sobel operators) in the x and y directions, respectively.

[0172] Since the input image is an 8-bit grayscale image with pixel values ​​ranging from [0, 255], the Sobel convolution kernel weights can be up to 4, resulting in gradient magnitudes... In edge regions, the score can reach hundreds or even higher. Therefore, to ensure consistency and numerical stability between the enhancement operation and the SuperPoint score, this embodiment normalizes the structural strength map.

[0173] Normalization:

[0174]

[0175] In this embodiment, candidate feature points are determined, and structural information is fused during the feature point selection process to optimize the non-maximum suppression strategy and improve the physical consistency of the selected feature points; specifically: in Perform NMS (3×3) on the above and retain the local maximum points as candidate feature points.

[0176] In step 3) above, if Figure 6 As shown, the inter-frame feedback-driven dynamic confidence threshold uses the number of matching points in the previous frame as a feedback indicator to guide the current frame confidence threshold to adjust adaptively, forming a closed loop.

[0177] Specifically, the feature point confidence threshold is dynamically adjusted based on the matching feedback between image frames to achieve adaptive control over the quantity and quality of feature points, and the final feature points are selected from the candidate feature points, including the following steps:

[0178] 3.1) Obtain the number of matching points in the previous frame image. By using the Tracking module of ORB-SLAM2, feature point matching is performed between the current frame image and the previous frame image to obtain the number of successfully matched points between the previous frame image and the current frame image. Based on the ideal number of matches and the number of successful matches Get matching deviation ;

[0179] Among them, the matching deviation is calculated. for: . This represents the ideal number of matches.

[0180] Ideal number of matches The target number of matching points, Mtarget, can be between 300 and 800, with 500 points being the preferred setting. This parameter reflects the system's desired stable number of matches and can be obtained through a pre-evaluation dataset or offline calibration. If there are too few matching points, the threshold should be lowered. >0 indicates too many matching points; increase the threshold to reduce redundant points.

[0181] 3.2) Based on matching deviation The threshold adjustment coefficient was calculated. The mean μ and standard deviation σ of the enhanced score map in step 2.3) are calculated, and the threshold adjustment coefficient is used. The mean μ and standard deviation σ are used to obtain the dynamic confidence threshold T;

[0182] Among them, based on matching deviation The threshold adjustment coefficient was calculated. for:

[0183]

[0184] In the formula, This is the initial confidence level adjustment coefficient. The acceptable range is [0.2, 1.0], preferably 0.5 to 0.7, used for feature point selection amplitude control under the baseline frame matching condition; this parameter can be empirically adjusted according to the system's image contrast sensitivity. k is the feedback sensitivity coefficient, a positive real number, preferably between 50 and 300; this parameter is used to balance the system's response speed to the deviation in the number of matching points, where a smaller k value makes the threshold adjustment more sensitive, suitable for low-texture or dynamic scenes. This function is designed so that the worse the matching (the smaller ΔM), the smaller α, and the lower the threshold to retain more feature points.

[0185] Dynamic threshold calculation:

[0186]

[0187] In the formula, σ is the mean of the score map for the current frame; σ is the standard deviation of the score map; T is the dynamic threshold for the confidence of key points in the current frame.

[0188] 3.3) By adaptively adjusting the dynamic confidence threshold T, the final feature points are selected from the candidate feature points, and the selected feature points are output. In this embodiment, by introducing a dynamic confidence threshold, the system lowers the threshold to retain more candidate points when the matching quality deteriorates (such as in low light or blur), and raises the threshold to suppress redundant points when there are sufficient matches, thus selecting feature points and ensuring a stable number and balanced quality of feature points.

[0189] In step 4) above, the improved SuperPoint based on the neural network receives feature data from two images to be matched, including the keypoint pixel coordinates, corresponding descriptor vectors, and optional keypoint quality scores and image size information for each image. Then, an initial cross-image association score is generated based on the similarity of the descriptor vectors to characterize the probability of each keypoint pair becoming a match. Subsequently, the above score and keypoint features are updated in multiple iterations: on the one hand, contextual information of neighboring keypoints within the same image is aggregated to improve discriminative power; on the other hand, information is exchanged between the two images to strengthen mutual constraints, thereby gradually improving the score of correct matches and suppressing interference. To improve efficiency, an adaptive calculation mechanism is introduced during the iteration process: firstly, keypoints with smaller contributions are gradually pruned or downweighted according to the current score to reduce subsequent calculations; secondly, the iteration is terminated early when the improvement in overall matching quality is lower than a preset threshold. After the iteration, matching relationships are constructed based on one-to-one constraints, a bidirectional consistency check strategy is used to eliminate conflicts, and a confidence score is assigned to each pair of matches. Subsequently, low-confidence results are filtered based on a configurable threshold, and the keypoint index pairs determined to be matched in the two images and their confidence scores are output.

[0190] In this embodiment, by estimating the homography matrix between adjacent keyframes, the geometric relationships of local planar regions in the scene can be uniformly described on the image plane, thus providing stable constraints for camera pose determination and map construction. Specifically, feature points are first extracted and matched between adjacent keyframes. The homography matrix is ​​then obtained through robust estimation using these corresponding points, and initial relative pose values ​​satisfying the assumptions of a planar or near-planar scene are obtained through homography decomposition. These initial values ​​serve two purposes: firstly, to provide a good initialization for subsequent nonlinear optimizations based on 3D reprojection errors (such as pose optimization and local BA), improving the convergence speed and stability of the optimization; secondly, the homography model can effectively eliminate geometrically inconsistent outliers during matching and filtering, improving the robustness of front-end tracking from the source. This enhances the overall accuracy and robustness of the SLAM system in complex lighting environments.

[0191] Specifically, after feature point extraction, feature point matching based on neural networks is performed between the current frame image and the previous frame image. The key point index pairs that are determined to be matched in the two images and their confidence scores are output. Match filtering and geometric consistency verification are then performed, specifically: a mismatch elimination method is used for match filtering and geometric consistency verification.

[0192] like Figure 7 As shown, the mismatch elimination method is used for match filtering and geometric consistency verification, including the following steps:

[0193] 4.1) Set a quality factor for each pair of matching points. This quality factor A quality factor that integrates distance ratio, confidence level, and diversity;

[0194] To enhance the suppression of mismatches and degradation (such as near-collinearity of four points or concentration in local regions), a quality factor integrating distance ratio, confidence level, and diversity is introduced. Specifically, for each pair of matching points... Defined as:

[0195]

[0196] in, This is the distance ratio of the match (the ratio of the smallest Euclidean distance to the second smallest Euclidean distance). To match the confidence level of the network output; and This measures the spatial diversity between the source and target images, with a value range of [value range missing]. Diversity measures were obtained using normalized grid statistics: image coordinates were normalized to... Within the scope and divided into Grid, recording matching points The count of the grid is , The maximum count in the entire map is ,but:

[0197]

[0198] To select matching points in sparse regions. Parameters The weights for β, confidence, and diversity are controlled separately, where, To avoid dividing by zero, small constants (The strength of the effect of β) (Confidence level temperature) (Diversity weight); Grid ,default .

[0199] 4.2) Assign all matching points to a quality factor Sort in descending order and select the first few. The matching point is added to the candidate set, and in the candidate set By pairing the matching points into groups of four, the total mass of the four-point groups can be calculated. ; and according to total mass Before selecting descending order The four-point set with the highest quality is selected, and the homography matrix of the four-point set is solved; among them, The preferred number of candidate matching pairs is n=600 and K=16. Four-point group .

[0200] The total mass of the four-point group is defined as follows:

[0201]

[0202] 4.3) For the four-point set with the highest quality, the homography matrix is ​​estimated using the normalized least squares method. Calculate the reprojection coordinates and based on the threshold Determine interior points;

[0203] in, η( ), and These represent the length and width of the current frame image, respectively. η is a constant, usually with a value of 0.1.

[0204] 4.4) After determining the interior points, count the number of interior points and compare the number of interior points with the preset interior point threshold. Comparison:

[0205] If the number of interior points exceeds the threshold If the match is successful, the homography matrix H is updated, and all interior point information is recorded; otherwise, the four feature points ranked second in matching quality are selected, and the iteration continues until a homography matrix that meets the requirements is found.

[0206] When the number of internal points exceeds the threshold Updated regularly ,in The total number of matches is denoted as μ, which is a constant between 0 and 1, typically ranging from 0.5 to 0.7, to enhance the algorithm's versatility.

[0207] 4.5) When the termination condition is met, terminate the iteration and return the optimal homography matrix H and all matching point pairs, and output the feature point matching results.

[0208] The termination condition is: the iteration terminates when the maximum number of iterations is reached.

[0209] In one embodiment of the present invention, an image enhancement visual SLAM system suitable for complex lighting environments is provided, comprising:

[0210] The image enhancement preprocessing module preprocesses the original image of the complex lighting environment by performing grayscale image preprocessing. It then determines the lighting environment of the original image based on the mean and standard deviation of the pixel grayscale values. Based on the lighting environment determination results, the original image is classified, and different adaptive image enhancement strategies are applied to enhance the original images of different categories.

[0211] The candidate point extraction module inputs the enhanced image into the improved SuperPoint neural network, estimates the confidence of the enhanced original feature points through image structure strength estimation, obtains the enhanced original feature point confidence map score map, and determines candidate feature points;

[0212] The feature point extraction module dynamically adjusts the feature point confidence threshold based on the matching feedback between image frames, thereby achieving adaptive control over the quantity and quality of feature points and selecting the final feature points from the candidate feature points.

[0213] The feature matching module performs feature point matching between the current frame image and the previous frame image after feature point extraction. It outputs the key point index pairs determined to be matched in the two images and their confidence scores, and performs match filtering and geometric consistency verification. Through quality ranking and dispersed point selection, it suppresses degradation caused by near collinearity or local clustering of four points without increasing computation, and steadily improves the feature matching estimation accuracy. The selected matching pairs are input into the SLAM system for front-end tracking and mapping.

[0214] In the above embodiments, the lighting environment of the original image is determined based on the mean and standard deviation of the pixel grayscale values ​​of the grayscale image, and the original image is classified based on the lighting environment determination result, including:

[0215] Based on the current input image size and pixel grayscale values, calculate the mean and standard deviation of the image's grayscale values.

[0216] Based on the calculated mean and standard deviation (std) of gray levels, a preliminary lighting environment classification is performed on the current image, including:

[0217] The current image is divided into low-contrast images and other images based on the standard deviation (std): if the standard deviation (std) is below a set threshold, it is a low-contrast image; otherwise, it is another image.

[0218] Other images are classified into dark images, normal images, and overexposed images based on the mean gray level: if the mean gray level is less than the minimum value of a first set interval, it is a dark image; if the mean gray level is within the first set interval, it is a normal image; if the mean gray level is greater than the maximum value of the first set interval, it is an overexposed image.

[0219] In the above embodiments, different adaptive image enhancement strategies are applied to enhance images of different categories, including:

[0220] Normal images are directly input into the improved SuperPoint neural network;

[0221] Low-contrast images undergo secondary CLAHE enhancement processing;

[0222] The dark image is enhanced by performing the following process:

[0223] Obtain the dark areas in the dark image, and perform SSR enhancement on the dark areas to obtain the enhanced image;

[0224] The Sobel operator is used to calculate the sum of gradient magnitudes S in the dark areas of the original image. orig For the same dark area after SSR enhancement, calculate the sum of gradient magnitudes S. SSR According to S SSR With S orig The ratio determines the gain ratio R for detail in dark areas. g ;

[0225] Median filtering is used to denoise dark areas. A noise parameter is calculated from the denoised image. Based on the noise parameter and a threshold, it is determined whether to apply a detail gain ratio (R) to the dark areas. g Determine if R g If the value is greater than the maximum value of the second set interval, skip the CLAHE enhancement process and output the image directly; if R g If the value is less than or equal to the maximum value of the second set interval range, and greater than the minimum value of the second set interval range, then perform Level 1 CLAHE enhancement processing; if the value is less than or equal to the minimum value of the second set interval range, then perform Level 2 CLAHE enhancement processing.

[0226] Enhancement processing is performed on overexposed images, specifically as follows:

[0227] Calculate the initial specular ratio ,in accordance with Adaptive selection of power exponent Used for Gamma darkening of highlights;

[0228] Based on the selected power index Gamma correction is applied to the input raw image to obtain the gamma-corrected image. In the image after Gamma correction Redefine the specular mask, statistically analyze the gray-level histogram of the specular mask region, and calculate the normalized probability distribution. p i This leads to the information entropy E of the highlight region;

[0229] Regionalized CLAHE decision is made based on the information entropy E of the highlight region: if E is greater than the maximum value of the third set interval, then three-level CLAHE enhancement processing is performed; if E is less than or equal to the maximum value of the third set interval but greater than the minimum value of the third set interval, then four-level CLAHE enhancement processing is performed only in the non-highlight region; if E is less than or equal to the minimum value of the third set interval, then CLAHE is skipped and the image after Gamma correction is used directly. To enhance the results.

[0230] In the above embodiments, edge overshoot detection is performed on the image after CLAHE enhancement, specifically as follows:

[0231] The image is filtered for strong edges to extract the pixels of strong edges and form a set of strong edges. and calculate the image x Directional gradient G x , y Directional gradient G y Gradient magnitude G and gradient direction ;

[0232] Multi-point sampling is performed along the image normal direction. If any sampling point exceeds the boundary, the pixel is skipped. Scalar sample I is obtained by bilinear interpolation based on the sampling points. k ;

[0233] Based on scalar sample I k Perform adjacent difference and monotonicity determination to determine whether strong edge pixels satisfy monotonicity violation.

[0234] Based on scalar sample I k Determine the overshoot-free baseline, and based on the overshoot-free baseline, determine the amplitude of the main body and the deviation of the local peak to obtain the overshoot ratio. r ovr If the overshoot ratio r ovr If the value is greater than or equal to the set value and the monotonicity is violated, then the local peak or valley is considered to have significant contrast relative to the main body, and the number of overshoot pixels is 1; otherwise, the number of overshoot pixels is 0.

[0235] Based on the number of overshoot pixels and the number of strong edge pixels Calculate the overshoot rate OVR. If the overshoot rate OVR is greater than the set range, lower the contrast limit of the original CLAHE and perform CLAHE enhancement processing again, and check the edge overshoot detection again. If the overshoot rate OVR is less than or equal to the set range, output the enhanced image.

[0236] In the above embodiments, the enhanced image is input into the improved SuperPoint neural network. The confidence of the enhanced original feature points is estimated through image structure strength estimation to obtain the enhanced original feature point confidence map (score map), and candidate feature points are determined, including:

[0237] An improved SuperPoint neural network is formed by introducing a structure-aware enhancement mechanism into the SuperPoint neural network.

[0238] The enhanced image is input into the improved SuperPoint neural network, which outputs the original feature point confidence map (score map).

[0239] Calculate the structure-aware score based on the original feature point confidence map. Score through structure perception The score map is enhanced to obtain the enhanced score map, and candidate feature points are obtained by optimizing the non-maximum suppression strategy.

[0240] In the above embodiments, the feature point confidence threshold is dynamically adjusted based on the matching feedback between image frames to achieve adaptive control of the number and quality of feature points, and the final feature points are selected from the candidate feature points, including:

[0241] Get the number of matching points in the previous frame image By using the Tracking module of ORB-SLAM2, feature point matching is performed between the current frame image and the previous frame image to obtain the number of successfully matched points between the previous frame image and the current frame image. Based on the ideal number of matches and the number of successful matches Get matching deviation ;

[0242] Based on matching bias The threshold adjustment coefficient was calculated. The mean μ and standard deviation σ of the enhanced score map are calculated, and the threshold adjustment coefficient is used. The mean μ and standard deviation σ are used to obtain the dynamic confidence threshold T;

[0243] By adaptively adjusting the dynamic confidence threshold T, the final feature points are selected from the candidate feature points.

[0244] In the above embodiments, after feature point extraction, feature point matching based on a neural network is performed between the current frame image and the previous frame image. The resulting keypoint index pairs determined to be matched in the two images, along with their confidence scores, are output. Match filtering and geometric consistency verification are then performed, specifically as follows:

[0245] A mismatch elimination method is used for match filtering and geometric consistency verification, including:

[0246] Set a quality factor for each pair of matching points. This quality factor A quality factor that integrates distance ratio, confidence level, and diversity;

[0247] All matching points by quality factor Sort in descending order and select the first few. The matching point is added to the candidate set, and in the candidate set By pairing the matching points into groups of four, the total mass of the four-point groups can be calculated. ; and according to total mass Before selecting descending order The four-point group with the highest quality;

[0248] For the four-point set with the highest quality, the homography matrix is ​​estimated using the normalized least squares method. Calculate the reprojection coordinates and based on the threshold Determine interior points;

[0249] After determining the interior points, count the number of interior points and compare the count with a preset interior point threshold. Comparison:

[0250] If the number of interior points exceeds the threshold If the match is successful, the homography matrix H is updated, and all interior point information is recorded; otherwise, the four feature points ranked second in matching quality are selected, and the iteration continues until a homography matrix that meets the requirements is found.

[0251] When the number of internal points exceeds the threshold Updated regularly ,in is the total number of matches, and µ is a constant between 0 and 1;

[0252] When the termination condition is met, the iteration terminates and returns the optimal homography matrix H and all matching point pairs.

[0253] The system provided in this embodiment is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0254] In one embodiment of the present invention, a computing device is provided. This computing device can be a terminal and may include a processor, a communication interface, memory, a display screen, and an input device. The processor, communication interface, and memory communicate with each other via a communication bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. When the computer programs are executed by the processor, they implement the methods described in the above embodiments. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals. Wireless communication can be achieved through Wi-Fi, a management network, NFC (Near Field Communication), or other technologies. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad mounted on the casing of the computing device, or an external keyboard, touchpad, or mouse. The processor can call logical instructions stored in the memory.

[0255] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0256] In one embodiment of the present invention, a computer program product is provided, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer is able to perform the methods provided in the above-described method embodiments.

[0257] In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided, which stores server instructions that cause a computer to perform the methods provided in the above embodiments.

[0258] The computer-readable storage medium provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0259] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0260] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0261] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0262] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visual SLAM method for image enhancement suitable for complex lighting environments, characterized in that, include: After preprocessing the original image of the complex lighting environment into a grayscale image, the lighting environment of the original image is determined based on the mean and standard deviation of the pixel grayscale value. The original image is then classified based on the lighting environment determination result, and different adaptive image enhancement strategies are applied to enhance the original images of different categories. The enhanced image is input into an improved SuperPoint neural network. The confidence of the original feature points is enhanced by estimating the image structure strength, resulting in an enhanced original feature point confidence map (score map). Candidate feature points are then determined, including: introducing a structure-aware enhancement mechanism into the SuperPoint neural network to form an improved SuperPoint neural network; inputting the enhanced image into the improved SuperPoint neural network to output the original feature point confidence map (score map); calculating the structure-aware score based on the original feature point confidence map (score map); enhancing the score map using the structure-aware score to obtain the enhanced score map; and obtaining candidate feature points by optimizing the non-maximum suppression strategy. The confidence threshold of feature points is dynamically adjusted based on the matching feedback between image frames to achieve adaptive control of the number and quality of feature points, and to select the final feature points from the candidate feature points. After feature point extraction, feature point matching is performed between the current frame image and the previous frame image. The index pairs of key points that are determined to be matched in the two images and their confidence scores are output. Match filtering and geometric consistency verification are performed. Through quality sorting and dispersed point selection, degradation caused by near collinearity of four points or local clustering is suppressed without increasing the computation, and the feature matching estimation accuracy is steadily improved. The selected matching pairs are input into the SLAM system for front-end tracking and mapping.

2. The image enhancement visual SLAM method for complex lighting environments as described in claim 1, characterized in that, The lighting environment of the original image is determined based on the mean and standard deviation of the pixel grayscale values. The original image is then classified based on the lighting environment determination results, including: Based on the current input image size and pixel grayscale values, calculate the mean and standard deviation of the image's grayscale values. Based on the calculated mean and standard deviation (std) of gray levels, a preliminary lighting environment classification is performed on the current image, including: The current image is divided into low-contrast images and other images based on the standard deviation (std): if the standard deviation (std) is below a set threshold, it is a low-contrast image; otherwise, it is another image. Other images are classified into dark images, normal images, and overexposed images based on the mean gray level: if the mean gray level is less than the minimum value of a first set interval, it is a dark image; if the mean gray level is within the first set interval, it is a normal image; if the mean gray level is greater than the maximum value of the first set interval, it is an overexposed image.

3. The image enhancement visual SLAM method for complex lighting environments as described in claim 2, characterized in that, Different adaptive image enhancement strategies are applied to different categories of images for enhancement processing, including: Normal images are directly input into the improved SuperPoint neural network; Low-contrast images undergo secondary CLAHE enhancement processing; The dark image is enhanced as follows: Extract the dark areas from the dark image, perform SSR enhancement on the dark areas, and obtain the enhanced image; The Sobel operator is used to calculate the sum of gradient magnitudes S in the dark areas of the original image. orig The sum of gradient magnitudes S is calculated for the same dark areas after SSR enhancement. SSR According to S SSR With S orig The ratio determines the gain ratio R for detail in dark areas. g ; Median filtering is used to denoise dark areas. A noise parameter is calculated from the denoised image. Based on the noise parameter and a threshold, it is determined whether to apply a detail gain ratio (R) to the dark areas. g Determine if R g If the value is greater than the maximum value of the second set interval, skip the CLAHE enhancement process and output the image directly; if R g If the value is less than or equal to the maximum value of the second set interval range, and greater than the minimum value of the second set interval range, then perform Level 1 CLAHE enhancement processing; if the value is less than or equal to the minimum value of the second set interval range, then perform Level 2 CLAHE enhancement processing. Enhancement processing is performed on overexposed images, specifically as follows: Calculate the initial specular ratio ,in accordance with Adaptive selection of power exponent Used for Gamma darkening of highlights; Based on the selected power index Gamma correction is applied to the input raw image to obtain the gamma-corrected image. In the image after Gamma correction Redefine the specular mask, statistically analyze the gray-level histogram of the specular mask region, and calculate the normalized probability distribution p. i This leads to the information entropy E of the highlight region; Regionalized CLAHE decision is made based on the information entropy E of the highlight region: if E is greater than the maximum value of the third set interval, then three-level CLAHE enhancement processing is performed; if E is less than or equal to the maximum value of the third set interval but greater than the minimum value of the third set interval, then four-level CLAHE enhancement processing is performed only in the non-highlight region; if E is less than or equal to the minimum value of the third set interval, then CLAHE is skipped and the image after Gamma correction is used directly. To enhance the results.

4. The image enhancement visual SLAM method for complex lighting environments as described in claim 3, characterized in that, The image after CLAHE enhancement is then subjected to edge overshoot detection, specifically as follows: The image is subjected to strong edge filtering to extract strong edge pixels, forming a set of strong edges, and the image's strength is calculated. x Directional gradient G x , y Directional gradient G y Gradient magnitude G and gradient direction; Multi-point sampling is performed along the image normal direction. If any sampling point exceeds the boundary, the pixel is skipped. scalar sample I is obtained by bilinear interpolation based on the sampling points. k ; Based on scalar sample I k Perform adjacent difference and monotonicity determination to determine whether strong edge pixels satisfy monotonicity violation. Based on scalar sample I k Determine the overshoot-free baseline, and based on the overshoot-free baseline, determine the amplitude of the main body and the deviation of the local peak to obtain the overshoot ratio. r ovr If the overshoot ratio is exceeded r ovr If the value is greater than or equal to the set value and the monotonicity is violated, then the local peak or valley is considered to have significant contrast relative to the main body, and the number of overshoot pixels is 1; otherwise, the number of overshoot pixels is 0. The overshoot ratio OVR is calculated based on the number of overshoot pixels and the number of strong edge pixels. If the overshoot ratio OVR is greater than the set range, the contrast limit of the original CLAHE is lowered and CLAHE enhancement processing is performed again, and edge overshoot detection is checked again. If the overshoot ratio OVR is less than or equal to the set range, the enhanced image is output.

5. The image enhancement visual SLAM method for complex lighting environments as described in claim 1, characterized in that, The feature point confidence threshold is dynamically adjusted based on the matching feedback between image frames to achieve adaptive control over the quantity and quality of feature points. The final feature points are selected from the candidate feature points, including: The number of matching points M in the previous frame image is obtained. Then, using the Tracking module of ORB-SLAM2, feature point matching is performed between the current frame image and the previous frame image to obtain the number of successfully matched points M between the previous and current frame images. t Based on the ideal number of matches and the number of successful matches M t The matching deviation ΔM is obtained; The threshold adjustment coefficient is calculated based on the matching bias ΔM, and the mean μ and standard deviation σ of the enhanced score map are calculated. The confidence dynamic threshold T is obtained through the threshold adjustment coefficient, mean μ and standard deviation σ. By adaptively adjusting the dynamic confidence threshold T, the final feature points are selected from the candidate feature points.

6. The image enhancement visual SLAM method for complex lighting environments as described in claim 1, characterized in that, After feature point extraction, a neural network-based feature point matching process is performed between the current frame and the previous frame. This outputs keypoint index pairs determined to be matches in both images, along with their confidence scores. Match filtering and geometric consistency verification are then performed, specifically as follows: A mismatch elimination method is used for match filtering and geometric consistency verification, including: A quality factor is set for each pair of matching points. This quality factor is an integrated quality factor that combines distance ratio, confidence score and diversity. Sort all matching points in descending order of quality factor, and select the top n pairs of matching points into the candidate set. In the candidate set, combine the n pairs of matching points into four-point groups, calculate the total quality of the four-point groups, and select the top K groups of four-point groups with the highest quality in descending order of total quality. For the four-point set with the highest quality, the homography matrix H is estimated using the normalized least squares method, and the reprojection coordinates are calculated and determined according to the threshold. Determine interior points; After determining the interior points, count the number of interior points and compare the number of interior points with the preset interior point threshold T. n Comparison: If the number of interior points exceeds the threshold T n If the match is successful, the homography matrix H is updated, and all interior point information is recorded; otherwise, the four feature points ranked second in matching quality are selected, and the iteration continues until a homography matrix that meets the requirements is found. When the number of internal points exceeds the threshold T n H is updated when H = µN, where N is the total number of matches and µ is a constant between 0 and 1; When the termination condition is met, the iteration terminates and returns the optimal homography matrix H and all matching point pairs.

7. An image enhancement visual SLAM system suitable for complex lighting environments, characterized in that, include: The image enhancement preprocessing module preprocesses the original image of the complex lighting environment by performing grayscale image preprocessing. It then determines the lighting environment of the original image based on the mean and standard deviation of the pixel grayscale values. Based on the lighting environment determination results, the original image is classified, and different adaptive image enhancement strategies are applied to enhance the original images of different categories. The candidate point extraction module inputs the enhanced image into an improved SuperPoint neural network, enhances the confidence of the original feature points by estimating the image structure strength, obtains an enhanced original feature point confidence map (score map), and determines candidate feature points. This includes: introducing a structure-aware enhancement mechanism into the SuperPoint neural network to form an improved SuperPoint neural network; inputting the enhanced image into the improved SuperPoint neural network to output the original feature point confidence map (score map); calculating a structure-aware score based on the original feature point confidence map (score map); enhancing the score map using the structure-aware score to obtain the enhanced score map; and obtaining candidate feature points by optimizing a non-maximum suppression strategy. The feature point extraction module dynamically adjusts the feature point confidence threshold based on the matching feedback between image frames, thereby achieving adaptive control over the quantity and quality of feature points and selecting the final feature points from the candidate feature points. The feature matching module performs feature point matching between the current frame image and the previous frame image after feature point extraction. It outputs the key point index pairs determined to be matched in the two images and their confidence scores, and performs match filtering and geometric consistency verification. Through quality ranking and dispersed point selection, it suppresses degradation caused by near collinearity or local clustering of four points without increasing computation, and steadily improves the feature matching estimation accuracy. The selected matching pairs are input into the SLAM system for front-end tracking and mapping.

8. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described in claims 1 to 6.

9. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described in claims 1 to 6.

Images