A visual odometry method and related device with enhanced perception of weak texture in mines

Abstract

This invention discloses a visual odometry method and related apparatus with enhanced perception of weak texture in mines. The method involves acquiring mine frame stream images; evaluating the brightness of the mine frame stream images to filter out those requiring or not requiring image restoration processing; processing the mine frame stream images requiring image restoration using a multi-stage image illumination restoration method; employing the ORB feature point extraction framework and based on Tamura theory to perform texture measurement on the restored and unrestored mine frame stream images to obtain feature point distribution density; improving quadtree uniformization based on the texture perception of the mine frame stream images; pre-integrating and initializing the IMU data using all IMU data from the previous frame to the current frame in the mine frame stream images; and tightly coupling the matching motion tracking information and IMU motion information of the mine frame stream images to calculate the relative pose of the device carrying the odometry.

Description

Technical Field

[0001] This invention belongs to the field of intelligent mining robots, specifically relating to a visual odometry method and related device with enhanced perception of weak textures in mines. Background Technology

[0002] Visual odometry (VO) technology is characterized by its non-contact nature, global coverage, and high precision. Based on coordinate transformation and combined with intelligent environmental perception technology, VO is commonly used for positioning tasks in robots and unmanned equipment. However, mine environments often present challenges such as low illumination, uneven illumination, and interference from various artificial light sources and reflective surfaces. Furthermore, the relatively simple internal environment of mines, despite the presence of numerous rocks, supports, pipes, and other objects, lacks sufficient texture information. This poses a challenge to VO technology in feature extraction, potentially affecting its effectiveness from the data source stage.

[0003] Currently, some researchers have integrated Retinex-based image joint denoising and enhancement networks into visual odometry systems, verifying the positive effect of improving frame stream data input quality on enhancing the robustness of visual odometry in low-light environments. Furthermore, image processing based on deep learning techniques can address specific problems and enhance the deep understanding of texture features; however, deep learning models excessively consume computational resources and heavily rely on data sources, making them unsuitable for visual localization tasks of resource-constrained equipment in mines. Other researchers have attempted to utilize additional infrared projection devices to project infrared light onto the perceived scene surface, "enhancing" the scene texture depth from a computer perspective to ensure that visual odometry can perform normal image matching.

[0004] Existing research provides a theoretical basis for the feasibility of texture enhancement in complex downhole environments. Numerous experiments have demonstrated that texture enhancement technology can improve the quality of input images in downhole environments. Visual odometry systems using texture enhancement technology exhibit superior positioning capabilities compared to traditional methods, providing higher positioning accuracy for equipment and personnel. However, existing texture enhancement techniques applied downhole are prone to color differences and noise, posing a significant challenge for feature point extraction in visual odometry. Significant color differences and noise can lead to mismatches of feature points. Furthermore, weakly textured regions are easily overlooked in processing input images, resulting in a lack of clear corner information and insufficient point feature extraction, potentially leading to substantial system errors. However, current research lacks solutions for addressing the weakly textured environment in downhole environments, such as how to extract high-quality feature points, adaptively adjust the number of feature points extracted based on texture conditions, and improve mismatch filtering mechanisms. Summary of the Invention

[0005] To address the existing technical problems, this invention proposes a visual odometry method and related device with enhanced perception of weak texture in mines. This invention can extract high-quality feature points in weak texture environments, adaptively increase the number of feature points extracted based on texture conditions, and improve the mismatch filtering mechanism.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A visual odometry method with enhanced perception of weak texture in mines includes the following steps:

[0008] Acquire frame stream images of the mine;

[0009] The brightness of the mine frame stream images is evaluated, and the mine frame stream images that need image restoration processing and those that do not need image restoration processing are selected.

[0010] A multi-stage image illumination restoration method is used to process the mine frame stream image that needs image restoration to obtain the restored mine frame stream image;

[0011] The ORB feature point extraction framework is used to perform texture measurement on the restored mine frame stream image and the mine frame stream image that does not require image restoration processing, based on Tamura theory, to obtain the feature point distribution density; wherein, contrast and roughness are used as quantization standards.

[0012] Based on the perception of texture in the mine frame stream image, the quadtree homogenization is improved after obtaining the feature point distribution density.

[0013] The IMU data is pre-integrated and initialized using all IMU data within the time period from the previous frame to the current frame in the mine frame stream image, where the current frame is the next frame after the previous frame.

[0014] By tightly coupling the matching motion tracking information and IMU motion information of the mine frame stream image, the relative pose of the main body of the device equipped with the odometer is calculated.

[0015] Preferably, when evaluating the brightness of the mine frame stream image:

[0016] For the low-normal-illuminance mine frame stream image, the low-normal-illuminance mine frame stream image is randomly cropped, and the brightness of the mine frame stream image is evaluated again.

[0017] Preferably, the multi-stage image illumination restoration method is used to process the mine frame stream image that needs image restoration to obtain the restored mine frame stream image, including the following process:

[0018] Adaptive histogram equalization based on contrast limitation is used to initially improve the contrast of mine frame stream images;

[0019] The contrast of the mine frame stream image was initially improved by Gaussian smoothing filtering to remove noise.

[0020] Adaptive gamma correction with exposure suppression and pixel grayscale normalization functions is used to process the denoised mine frame stream image to obtain the restored mine frame stream image.

[0021] Preferably, when performing denoising on the mine frame stream image with initially improved contrast through Gaussian smoothing filtering, the operator used is a Gaussian-Laplacian operator combining the Gaussian and Laplacian operators. :

[0022]

[0023] in, These are the pixel coordinates of the mine frame stream image. Represents the Gaussian standard deviation;

[0024] The adaptive gamma correction is as follows:

[0025] The exposure status of the current pixel's grayscale value is determined by the following formula:

[0026]

[0027] in, Represents the normalized grayscale value of the current pixel. Represents the standard for exposure brightness;

[0028] like If the value is greater than 0, then the adaptive gamma correction Defined as:

[0029]

[0030] in, Represents the exposure suppression index;

[0031] like If the value is greater than 0, then the adaptive gamma correction Defined as:

[0032]

[0033] in, X This represents the average grayscale value of all pixels in the mine frame stream image.

[0034] Preferably, when using the ORB feature point extraction framework to perform texture measurement on the mine frame stream restored image and the mine frame stream image that does not require image restoration processing, based on Tamura theory, to obtain the feature point distribution density:

[0035] Contrast ratio using Weber contrast ratio :

[0036]

[0037] in, Represents the pixels to be traversed p grayscale value, This represents the average grayscale value of the image pixels. n Represents the number of pixels;

[0038] For each pixel Surrounding size is × ( The average texture intensity is calculated using a window of (0, 1, ..., 5). :

[0039]

[0040] At each pixel, calculate the texture intensity in the horizontal direction between non-overlapping average pairs. Texture intensity in the vertical direction :

[0041]

[0042]

[0043] Compare the various texture intensities to find the one that maximizes the texture intensity. ,Right now:

[0044]

[0045]

[0046] The following normalization operation is performed on the roughness:

[0047]

[0048] For images with excessively high roughness, the radius of the search circle is determined by expanding the keypoints. This method explores the significant differences in grayscale between keypoints and other pixels within a larger neighborhood, defining:

[0049]

[0050] in, This represents the floor function, which adjusts the floor function when the roughness is too high. Maintain the number of feature points extracted, and maximize It is limited to 7 or less;

[0051] Next, the key point filtering threshold is adjusted based on texture features. The threshold is adaptively selected based on the different texture structures of each mine image:

[0052]

[0053] in, , They represent contrast ratios respectively. and roughness Threshold selection coefficient.

[0054] Preferably, the improvement of quadtree homogenization based on the perceived texture of the mine frame stream image, after obtaining the feature point distribution density, includes the following process:

[0055] After the first split of the quadtree, assuming the total number of feature points to be extracted is... Count the number of feature points of the four nodes, denoted as . , =1,2,3,4, and the following judgment operation is performed:

[0056] like This means that the feature points of this node are too sparse due to the presence of weak texture regions. In the subsequent splitting process, nodes with 1 or 2 feature points will not be split again.

[0057] Preferably, the process of tightly coupling the matching motion tracking information and IMU motion information of the mine frame stream image to calculate the relative pose of the device body equipped with the odometer includes the following steps:

[0058] Using the optimized state from the previous frame and the IMU pre-integration, the initial state value of the current frame is predicted. The state prediction value from the previous frame is used as the initial value. The visual reprojection error and the IMU pre-integration error are jointly optimized to obtain the optimal relative pose information.

[0059] The present invention also provides a visual odometry system with enhanced perception of weak textures in mines, for implementing the visual odometry method with enhanced perception of weak textures in mines as described above, comprising:

[0060] Data acquisition unit: used to acquire mine frame stream images;

[0061] Evaluation unit: Used to evaluate the brightness of mine frame stream images and filter out mine frame stream images that need image restoration processing and those that do not need image restoration processing;

[0062] Restoration Unit: Used to process the mine frame stream image that needs image restoration using a multi-stage image illumination restoration method, to obtain the mine frame stream restored image;

[0063] Feature extraction unit: Used to perform texture measurement on the restored mine frame stream image and the mine frame stream image that does not require image restoration processing based on the ORB feature point extraction framework and Tamura theory to obtain the feature point distribution density; wherein, contrast and roughness are used as quantization standards;

[0064] Optimization unit: used to improve quadtree homogenization based on the perception of texture in the mine frame stream image, given the feature point distribution density;

[0065] First computing unit: used to pre-integrate and initialize IMU data using all IMU data within the time period from the previous frame to the current frame in the mine frame stream image, wherein the current frame is the next frame after the previous frame;

[0066] The second calculation unit is used to tightly couple the matching motion tracking information and IMU motion information of the mine frame stream image to calculate the relative pose of the main body of the device equipped with the odometer.

[0067] The present invention also provides an electronic device, comprising:

[0068] One or more processors;

[0069] A storage device on which one or more programs are stored;

[0070] When the one or more programs are executed by the one or more processors, the one or more processors implement the visual odometry method with enhanced weak texture perception in mines as described above.

[0071] The present invention also provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the visual odometry method with enhanced weak texture perception in mines as described above.

[0072] The present invention has the following beneficial effects:

[0073] This invention presents a visual odometry method for mines with enhanced weak texture perception, specifically addressing the challenges of complex environments characterized by low illumination, weak texture, and multiple interferences. Through end-to-end collaborative design, it significantly improves the positioning accuracy and environmental adaptability of visual odometry. Specifically, this invention first evaluates the brightness of the mine frame stream images, filtering out images that require restoration and those that do not, avoiding over-processing of high-quality images to preserve true texture. Then, a multi-stage image illumination restoration method is used to balance brightness compensation and noise suppression in stages, effectively reducing color differences and noise that are easily generated during image restoration in existing technologies, thus eliminating interference for feature point extraction at the data source level. Building upon this, this invention utilizes the ORB feature point extraction framework, combined with Tamura theory, using contrast and roughness as quantification standards to accurately measure the texture features of mine images, clearly capturing information in weak texture areas. Simultaneously, it improves quadtree homogenization based on texture perception, adaptively adjusting the extraction strategy according to the feature point distribution density. This strengthens feature point search in weak texture areas and reasonably controls the amount of feature points in dense texture areas, ensuring sufficient high-quality corner point information is obtained across all scenarios, solving the problems of easily overlooked weak texture areas and insufficient feature points in mines. Furthermore, this invention utilizes all IMU data from the previous frame to the current frame for pre-integration and initialization, tightly coupling image matching motion tracking information with IMU motion information. Through the complementary advantages of visual and inertial information, it compensates for visual information gaps caused by weak textures and mismatches, effectively suppressing positioning drift. Moreover, it does not rely on deep learning models or additional infrared equipment, maintaining lightweight characteristics and adapting to the equipment needs of mines with limited resources. This comprehensively improves the positioning reliability and practicality of visual odometry in complex mine environments. This invention can improve the adaptability of visual odometry in low-light, uneven-light, and weak-texture mine scenes through basic grayscale transformation methods, aiming to provide robust and accurate pose input for subsequent visual positioning systems. Attached Figure Description

[0074] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 This is a schematic diagram of the overall frame of the visual odometry system provided in an embodiment of the present invention;

[0076] Figure 2 This is a comparison of image restoration indicators provided in the embodiments of the present invention;

[0077] Figure 3 This is a schematic diagram of different texture features of a mine provided in an embodiment of the present invention; Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0079] like Figure 1 As shown, the present invention provides a visual odometry method for enhancing weak texture perception in mines, comprising the following steps:

[0080] Step 1: Construct a model for judging the illumination of mine frame stream images using the gray-scale variance method.

[0081] The brightness of the image is evaluated to determine whether image restoration processing is necessary. Taking into account uneven illumination, this invention further randomly crops images with evaluation values ​​within a certain threshold (defined as weak normal illumination) and performs local brightness detection and evaluation again. The standard for evaluating image brightness is the gray-scale variance method.

[0082]

[0083]

[0084] in, Represents the variance of image gray levels. This represents the average grayscale value of the image pixels. Indicates the area size of the image. Represents pixel coordinates, This represents the corresponding grayscale value.

[0085] Step 2: Design a multi-stage image illumination restoration method. First, in Contrast Limited Adaptive Histogram Equalization (CLAHE), the image to be restored is divided into several sub-blocks, and then histogram cropping is performed on each sub-block. Then, histogram equalization is adjusted for each sub-block. Finally, different interpolation operations are performed according to different positions of the sub-blocks to obtain the transformed grayscale values, thereby achieving contrast-limited adaptive histogram equalization.

[0086] Next, Gaussian smoothing filtering is introduced to denoise the image. Then, the Gaussian operator is combined with the Laplacian operator to enhance edge features and improve details while denoising the image, resulting in the Laplacian of Gaussian (LoG) operator.

[0087]

[0088] in, This represents the Gaussian standard deviation.

[0089] Finally, an adaptive gamma correction with exposure suppression and pixel grayscale normalization functions was designed to evenly improve the perceptible range of mine frame stream images:

[0090] Determine the exposure status of the current pixel's grayscale value:

[0091]

[0092] in, Represents the normalized grayscale value of the current pixel. This represents the standard for exposure brightness, which is set to 0.8 in this invention. If... If greater than 0, then define for

[0093]

[0094] in, This represents the exposure suppression index, which is set to 3 in this invention. If... If the value is less than 0, and the average normalized grayscale value of all pixels in the mine frame stream image is set to 0.33 (denoted as the image grayscale center), then the gamma value to be corrected should make the average normalized pixel grayscale value of the target image close to 0.33. Let the average grayscale value be X, then the value to be corrected is obtained through mathematical transformation. value

[0095]

[0096] Figure 2Figures (a) to (e) in the figure show an intuitive macroscopic comparison and detailed comparison with the improved open-source image restoration algorithm. The comparison includes four algorithms: automatedMSRCR, IAGCWD, LIME, and DUAL. The non-reference evaluation metrics used are: (1) Discrete Entropy (DE); (2) Average Gradient (AG); (3) Signal-to-Noise Ratio (SNR); (4) BRISQUEQUILITY; and (5) NIQE. Higher DE, AG, and SNR values ​​indicate better image quality, while lower BRISQUEQUILITY and NIQE values ​​also indicate better image quality. In terms of objective metric scores, the algorithm of this invention (circle with red center and black border in the figure) still outperforms other algorithms in most scenarios. In 50 experiments consisting of five evaluation metrics and ten mine scenarios, it achieved the best results in 40 of them. Even with the average score, it still maintains a significant gap with other algorithms, proving that the algorithm of this invention has greater universal applicability in various mine scenarios. Ultimately, the algorithm of this invention improves the DE, AG and SNR evaluation indicators by 7.42%, 28.6% and 13.41% respectively compared with the improved open source image restoration algorithm, while reducing the BRISQUEQULITY and NIQE evaluation indicators by 40.86% and 16.8% respectively.

[0097] Step 3: Quantify the texture richness of the mine frame stream image, and adaptively adjust the keypoint extraction range and threshold of ORB feature points based on contrast and roughness. In this step, the feature point extraction framework uses ORB feature points, and texture measurement of the mine image is performed based on Tamura theory, using contrast and roughness as quantization standards. For images with excessively high roughness, an expanded keypoint search circle radius is used to explore the significant difference in grayscale between keypoints and other pixels in a larger neighborhood, thereby increasing the perception ability of weak texture areas. Next, the keypoint screening threshold is adjusted based on texture features, and the threshold is adaptively selected according to the different texture structures of each mine image.

[0098] Specifically, the contrast ratio uses Weber contrast ratio:

[0099]

[0100] in, Represents the pixels to be traversed p The grayscale value. This represents the average grayscale value of the image pixels.

[0101] The roughness calculation process takes into account the difference in average texture intensity between non-overlapping windows of different sizes:

[0102] For each pixel Surrounding size is × ( The average texture intensity is calculated using a window of (0, 1, ..., 5).

[0103]

[0104] At each pixel, calculate the texture intensity at the horizontal level between non-overlapping average pairs. Texture intensity in the vertical direction :

[0105]

[0106]

[0107] Compare the various texture intensities in the comparison steps to find the appropriate one. The largest ,Right now

[0108]

[0109]

[0110] The following normalization operation is performed on the roughness:

[0111]

[0112] As shown in Figures (a1) and (a2) of Figure (3), these two images have similar roughness but different contrast; Figures (b1) and (b2) have similar contrast but different roughness. When the image roughness is similar, more feature points can be extracted from the image with high contrast; but when the contrast is similar, more feature points can be extracted from the image with low roughness. That is, under higher contrast and lower roughness, the texture features of the image are richer, and more feature points can be extracted. As shown in Figure (a3) ​​of Figure (2), after introducing the mine image into the multi-stage image illumination restoration method in step 2, it can be found that the number of extracted feature points is increased, which proves the positive role of the image restoration model in enriching image texture in terms of contrast.

[0113] For images with excessively high roughness, the radius of the search circle is determined by expanding the keypoints. This method explores the significant grayscale differences between keypoints and other pixels within a larger neighborhood. In areas with weak texture features, it can, to some extent, "enclose" weakly textured areas, thereby increasing the perception of these areas. Definition

[0114]

[0115] in, This represents the floor function. When the roughness is too high, it can be adjusted... The number of feature points extracted was maintained, and the maximum was increased. It is limited to 7 and below.

[0116] Next, the key point filtering threshold is adjusted based on texture features. The threshold is adaptively selected based on the different texture structures of each mine image:

[0117]

[0118] in, , They represent contrast ratios respectively. and roughness Threshold selection coefficients were tested and 150 and 250 were selected as the threshold selection coefficients for both.

[0119] Step 4: Given the feature point distribution density, to prevent feature points from clustering in texture-rich areas and becoming too sparse in texture-weak areas, the quadtree homogenization is improved based on the perceived texture of the mine image. After the first quadtree split, assuming the total number of feature points to be extracted is... Count the number of feature points of the four nodes, denoted as . , =1,2,3,4, and the following judgment operation is performed:

[0120] like This means that the feature points of this node are too sparse due to the presence of weak texture regions. The original feature point division cannot effectively perceive the features of weak texture regions. Therefore, in the subsequent splitting process, nodes with 1 or 2 feature points are retained and will not be split again, in order to preserve the ability to perceive weak texture regions.

[0121] Step 5: Perform pre-integration and initialization on the IMU data. Use all IMU data from frame k-1 to frame k to perform pre-integration, where frame k-1 is the frame before (or the previous frame) of frame k.

[0122] Step 6: Tightly couple the matching motion tracking information and IMU motion information of the image frame stream to calculate the relative pose of the device body equipped with the odometry. Specifically, use the optimized state of the previous frame and the pre-integration of the IMU to predict the initial state value of the current frame. Then, using the predicted value as the initial value, jointly optimize the visual reprojection error and the IMU pre-integration error to obtain the optimal relative pose information.

[0123] The visual odometry method with enhanced weak texture perception in mines, as described in this invention, is coupled with the ORB-SLAM3 algorithm and run on the OIVIO dataset. The data includes left and right eye images, IMU measurement data, and real trajectories, collecting data on dark environments including mines and tunnels, which is highly consistent with the application scenario of this invention.

[0124] RMSE was used as the quantification metric (unit: m). Table 1 shows the average values ​​of each visual SLAM algorithm after running 10 times in each OIVIO sequence. The bolded values ​​indicate the minimum localization error metric for each sequence.

[0125] Table 1

[0126]

[0127] Table 1 shows that the OUR-SLAM algorithm achieves the best localization results in terms of both accuracy and stability. Compared with the original ORB-SLAM3, OUR-SLAM reduces the average error in the RMSE index by 40.43%. On challenging sequences such as MN_015_GV_01, MN_015_GV_02, and the sequence itself, OUR-SLAM demonstrates significant improvements in both localization accuracy and stability. For the tunnel scene TN, the equipment's operating speed is relatively uniform, and the illumination conditions are sufficient to extract a sufficient number of feature points in feature-rich areas. OUR-SLAM, due to its ability to extract more feature points in less textured ground and wall scenes, still shows improved localization accuracy and stability compared to ORB-SLAM3. While CLAHE-SLAM performs well in reconstructing the MN trajectory in the mine scene, its accuracy in the tunnel scene TN is significantly lower than that of FM-SLAM.

[0128] In the above-mentioned technical solution of the present invention, a multi-stage image restoration method is adopted to improve the problem of limited perception range of visual odometry in the mine environment; by comprehensively considering the constituent factors of mine texture features, texture features are quantified to enhance the perception of visual odometry in the weak texture environment of the mine.

[0129] Furthermore, embodiments of the present invention also provide a system for implementing the above-described visual odometry method with enhanced perception of weak textures in mines, the system comprising:

[0130] Data acquisition unit: used to acquire mine frame stream images;

[0131] Evaluation unit: Used to evaluate the brightness of mine frame stream images and filter out mine frame stream images that need image restoration processing and those that do not need image restoration processing;

[0132] Restoration Unit: Used to process the mine frame stream image that needs image restoration using a multi-stage image illumination restoration method, to obtain the mine frame stream restored image;

[0133] Feature extraction unit: Used to perform texture measurement on the restored mine frame stream image and the mine frame stream image that does not require image restoration processing based on the ORB feature point extraction framework and Tamura theory to obtain the feature point distribution density; wherein, contrast and roughness are used as quantization standards;

[0134] Optimization unit: used to improve quadtree homogenization based on the perception of texture in the mine frame stream image, given the feature point distribution density;

[0135] First computing unit: used to pre-integrate and initialize IMU data using all IMU data within the time period from the previous frame to the current frame in the mine frame stream image, wherein the current frame is the next frame after the previous frame;

[0136] The second calculation unit is used to tightly couple the matching motion tracking information and IMU motion information of the mine frame stream image to calculate the relative pose of the main body of the device equipped with the odometer.

[0137] The embodiments of the present invention also provide corresponding electronic devices and computer-readable storage media for implementing the solutions provided in the embodiments of the present invention.

[0138] The electronic device includes a storage device and one or more processors. The storage device stores instructions or code, and the processors execute the instructions or code to enable the device to perform the visual odometry method with enhanced weak texture perception in mines as described in any embodiment of this application.

[0139] The storage medium stores a computer program, which, when executed by a processor, implements the visual odometry method with enhanced weak texture perception in mines as described in any embodiment of this application.

[0140] Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort should fall within the scope of protection of the present invention.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A visual odometry method with enhanced perception of weak textures in mines, characterized in that, The process includes the following: Acquire frame stream images of the mine; The brightness of the mine frame stream images is evaluated, and the mine frame stream images that need image restoration processing and those that do not need image restoration processing are selected. A multi-stage image illumination restoration method is used to process the mine frame stream image that needs image restoration, and the restored mine frame stream image is obtained. The ORB feature point extraction framework is used to perform texture measurement on the restored mine frame stream image and the mine frame stream image that does not require image restoration processing, based on Tamura theory, to obtain the feature point distribution density; wherein, contrast and roughness are used as quantization standards. Based on the perception of texture in the mine frame stream image, the quadtree homogenization is improved after obtaining the feature point distribution density. The IMU data is pre-integrated and initialized using all IMU data within the time period from the previous frame to the current frame in the mine frame stream image, where the current frame is the next frame after the previous frame. By tightly coupling the matching motion tracking information and IMU motion information of the mine frame stream image, the relative pose of the main body of the device equipped with the odometer is calculated.

2. The visual odometry method with enhanced perception of weak texture in mines according to claim 1, characterized in that, When evaluating the brightness of the mine frame stream image: For the low-normal-illuminance mine frame stream image, the low-normal-illuminance mine frame stream image is randomly cropped, and the brightness of the mine frame stream image is evaluated again.

3. The visual odometry method with enhanced perception of weak texture in mines according to claim 1, characterized in that, The method of using a multi-stage image illumination restoration method to process the mine frame stream image that needs image restoration to obtain the restored mine frame stream image includes the following steps: Adaptive histogram equalization based on contrast limitation is used to initially improve the contrast of mine frame stream images; The contrast of the mine frame stream image was initially improved by Gaussian smoothing filtering to remove noise. Adaptive gamma correction with exposure suppression and pixel grayscale normalization functions is used to process the denoised mine frame stream image to obtain the restored mine frame stream image.

4. The visual odometry method with enhanced perception of weak texture in mines according to claim 3, characterized in that, When denoising the mine frame stream image, which has initially improved contrast, using Gaussian smoothing filtering, the Gaussian-Laplacian operator, which combines the Gaussian and Laplacian operators, is employed. : in, These are the pixel coordinates of the mine frame stream image. Represents the Gaussian standard deviation; The adaptive gamma correction is as follows: The exposure status of the current pixel's grayscale value is determined by the following formula: in, Represents the normalized grayscale value of the current pixel. Represents the standard for exposure brightness; like If the value is greater than 0, then the adaptive gamma correction Defined as: in, Represents the exposure suppression index; like If the value is greater than 0, then the adaptive gamma correction Defined as: in, X This represents the average grayscale value of all pixels in the mine frame stream image.

5. The visual odometry method with enhanced perception of weak texture in mines according to claim 1, characterized in that, When using the ORB feature point extraction framework, based on Tamura theory, to perform texture measurement on the restored mine frame stream image and the mine frame stream image that does not require image restoration processing, and to obtain the feature point distribution density: Contrast ratio using Weber contrast ratio : in, Represents the pixels to be traversed p grayscale value, This represents the average grayscale value of the image pixels. n Represents the number of pixels; For each pixel Surrounding size is × ( The average texture intensity is calculated using a window of (0, 1, ..., 5). : At each pixel, calculate the texture intensity in the horizontal direction between non-overlapping average pairs. Texture intensity in the vertical direction : Compare the various texture intensities to find the one that maximizes the texture intensity. ,Right now: The following normalization operation is performed on the roughness: For images with excessively high roughness, the radius of the search circle is determined by expanding the keypoints. This method explores the significant differences in grayscale between keypoints and other pixels within a larger neighborhood, defining: in, This represents the floor function, which adjusts the floor function when the roughness is too high. Maintain the number of feature points extracted, and maximize It is limited to 7 or less; Next, the key point filtering threshold is adjusted based on texture features. The threshold is adaptively selected based on the different texture structures of each mine image: in, , They represent contrast ratios respectively. and roughness Threshold selection coefficient.

6. The visual odometry method with enhanced perception of weak texture in mines according to claim 1, characterized in that, The improvement of quadtree uniformization based on the perceived texture of the mine frame stream image, after obtaining the feature point distribution density, includes the following process: After the first split of the quadtree, assuming the total number of feature points to be extracted is... Count the number of feature points of the four nodes, denoted as . , =1,2,3,4, and the following judgment operation is performed: like This means that the feature points of this node are too sparse due to the presence of weak texture regions. In the subsequent splitting process, nodes with 1 or 2 feature points will not be split again.

7. A visual odometry method with enhanced perception of weak textures in mines according to claim 1, characterized in that, The process of tightly coupling the matching motion tracking information and IMU motion information of the mine frame stream image to calculate the relative pose of the device body equipped with the odometer includes the following steps: Using the optimized state from the previous frame and the IMU pre-integration, the initial state value of the current frame is predicted. The state prediction value from the previous frame is used as the initial value. The visual reprojection error and the IMU pre-integration error are jointly optimized to obtain the optimal relative pose information.

8. A visual odometry system with enhanced perception of weak textures in mines, characterized in that, A visual odometry method with enhanced perception of weak texture in mines as described in any one of claims 1-7, comprising: Data acquisition unit: used to acquire mine frame stream images; Evaluation unit: Used to evaluate the brightness of mine frame stream images and filter out mine frame stream images that need image restoration processing and those that do not need image restoration processing; Restoration Unit: Used to process the mine frame stream image that needs image restoration using a multi-stage image illumination restoration method, to obtain the mine frame stream restored image; Feature extraction unit: Used to perform texture measurement on the restored mine frame stream image and the mine frame stream image that does not require image restoration processing based on the ORB feature point extraction framework and Tamura theory to obtain the feature point distribution density; wherein, contrast and roughness are used as quantization standards; Optimization unit: used to improve quadtree homogenization based on the perception of texture in the mine frame stream image, given the feature point distribution density; First computing unit: used to pre-integrate and initialize IMU data using all IMU data within the time period from the previous frame to the current frame in the mine frame stream image, wherein the current frame is the next frame after the previous frame; The second calculation unit is used to tightly couple the matching motion tracking information and IMU motion information of the mine frame stream image to calculate the relative pose of the main body of the device equipped with the odometer.

9. An electronic device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the visual odometry method with enhanced weak texture perception in mines as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, It stores a computer program, wherein the computer program, when executed by a processor, implements the visual odometry method with enhanced weak texture perception in mines as described in any one of claims 1-7.

Images