Self-adaptive underwater image ORB feature point extraction method based on image fusion

Through the method of image fusion and adaptive dynamic threshold calculation, the problems of poor underwater image quality and noise sensitivity are solved, and the extraction quality and feature matching effect of underwater image feature points are improved.

CN118941805APending Publication Date: 2024-11-12XI AN JIAOTONG UNIV
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411043503.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing image feature extraction algorithm is poor in the quality of underwater images and is sensitive to noise, resulting in unsatisfactory feature extraction effect. The threshold needs to be adjusted cumbersomely by humans, making it difficult to adapt to complex underwater environments.

Method used

Adaptive method based on image fusion is adopted to improve image quality through preprocessing, calculate dynamic thresholds to extract ORB feature points for underwater images, adaptively adjust the feature extraction threshold, and dynamically adjust the threshold according to the texture richness of the image area.

Benefits of technology

The extraction quality of feature points in underwater image is improved, the problem of missing or poor quality of feature points caused by low noise and contrast is avoided, and the feature matching effect is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941805A_ABST
    Figure CN118941805A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of feature point extraction, and discloses a self-adaptive underwater image ORB feature point extraction method based on image fusion, which comprises the following steps: firstly, improving the quality of an underwater image, then carrying out cell division and self-adaptive dynamic threshold calculation on the basis of a fused image, and in cells with rich textures, carrying out self-adaptive dynamic threshold calculation on the fused image; the coefficient # imgabs0 # of the dynamic threshold value is greater than 1, and the dynamic threshold value is reduced, so that more feature points can be extracted; in the weak texture cells, the coefficient # imgabs1 # of the dynamic threshold is less than 1, and the dynamic threshold is increased, so that the extraction quality of the underwater image feature points is improved, and the problem that the detection number of the feature points is reduced or the quality of the detected feature points is low due to serious reduction of the quality of the underwater image is solved. According to the method, more feature points can be extracted in an area with rich image texture information, the situation that feature points with poor quality are extracted in an area with less image texture information is avoided, and the feature point detection effect of the underwater image is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of feature point extraction and relates to an adaptive underwater image ORB feature point extraction method based on image fusion. Background Art

[0002] Machine vision is widely used underwater. Its main applications include underwater target recognition, such as underwater organisms, mines, pipelines, etc.; underwater terrain detection; underwater robot positioning and path planning, etc. At present, image feature extraction algorithms have become mature, but the current classic image feature extraction algorithms still have limitations for underwater images. For example, the FAST algorithm and the ORB algorithm have made great breakthroughs in speed; the SIFT algorithm and the ORB algorithm are invariant to image changes such as scale and rotation, but these algorithms are often sensitive to noise. When applied to underwater images, the extracted image features are often not ideal due to the poor quality of underwater images. At the same time, since the thresholds of the FAST algorithm and the ORB algorithm need to be set manually in advance, but the underwater environment is very complex, manual adjustments are required for different situations to obtain better results. This process is too cumbersome, which also leads to the fact that such methods are not ideal in the application of actual underwater images.

[0003] In order to solve the problem of poor underwater image quality, researchers have proposed many underwater image processing methods, mainly including traditional underwater image enhancement algorithms, underwater image restoration algorithms based on underwater imaging models, image processing algorithms based on deep learning, and image processing algorithms based on fusion. Traditional image enhancement algorithms do not consider the physical model of the underwater image degradation process, but simply improve the visual quality of the image by adjusting the image pixel value. This type of algorithm is conducive to improving the contrast of the image and enhancing the texture information of the underwater image, but the complexity of the underwater environment often limits the application of such algorithms. The restoration-based method can correct problems such as image color distortion to a certain extent, but due to the complexity of the underwater environment, the establishment of the underwater imaging model is very difficult, which leads to the limited effect of the restoration-based algorithm. Although the deep learning-based method can obtain clear underwater images, it often requires a large amount of data sets for training and has poor universality. Summary of the invention

[0004] The present invention overcomes the deficiencies of the above-mentioned prior art and provides an adaptive underwater image ORB feature point extraction method based on image fusion, which can perform texture enhancement and color correction on underwater images to improve the quality of underwater images, and can adaptively adjust the feature extraction thresholds of different areas of the image to improve the effect of underwater image feature extraction.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] The invention provides an adaptive underwater image ORB feature point extraction method based on image fusion, comprising preprocessing an original image to obtain a color correction image, and saving the image as image 1; converting the color correction image to a Lab color space, filtering and contrast enhancement to obtain image 2; converting image 2 to an RGB color space, calculating the Laplacian contrast weight, local contrast weight and exposure weight of image 1 and image 2; setting the number of pyramid layers to construct a weighted Gaussian pyramid of image 1 and image 2; constructing a Laplacian pyramid according to the weighted Gaussian pyramids of image 1 and image 2 to output a fused image; calculating a FAST dynamic threshold in ORB according to the fused image; and extracting ORB feature points according to the dynamic threshold.

[0007] Furthermore, according to the fused image, the FAST dynamic threshold in ORB is calculated, including: calculating the grayscale standard deviation of the fused image; dividing the fused image into M×N cells, where M is the row and N is the column; calculating the grayscale standard deviation of each cell; calculating the standard deviation ratio coefficient of each cell and the fused image according to the grayscale standard deviation of each cell; and calculating the threshold of each cell according to the standard deviation ratio coefficient.

[0008] Furthermore, the grayscale standard deviation of the fused image is:

[0009]

[0010] Where W is the width of the image, H is the height of the image, and I(x,y) represents the pixel value of the xth row and yth column. is the mean value of the pixel value of the fused image;

[0011] The grayscale standard deviation of each cell is:

[0012]

[0013] Among them, μ is the mean value of each cell pixel value;

[0014] The standard deviation scaling factor for each cell and the fused image is:

[0015]

[0016] The threshold for each cell is:

[0017]

[0018] Among them, Th initial is the preset threshold.

[0019] Furthermore, the original image is preprocessed, including: grayscale world white balance processing; calculating the saturation threshold Threshold of each channel C of the original image C ; Perform histogram contrast adjustment on each channel C.

[0020] Further, calculate the saturation threshold Threshold of each channel C of the original image C ,include:

[0021] Threshold C =0.002×μ C

[0022] μ C =Max / C

[0023] Max=max{R,G,B}

[0024] Among them, R, G, B are the average brightness of red, green, and blue channels, C∈{R,G,B}, μ C is the scaling factor of the corresponding channel.

[0025] Furthermore, the grayscale world white balance process also includes setting an initial FAST threshold Th initial ; The initial FAST threshold Th initial is 20.

[0026] Further, the Laplacian contrast weights of image 1 and image 2 are calculated, including: using the Laplacian operator to filter image 1 and image 2 to obtain the Laplacian contrast weight W L1 Compare the weights W with the Laplacian L2 .

[0027] Further, the local contrast weights of image 1 and image 2 are calculated, including: setting a Gaussian filter h, setting a high-frequency cutoff value whc; using the Gaussian filter to filter image 1 and image 2, applying the high-frequency cutoff value to D(x, y), and obtaining the local contrast weight W LC (x,y)=min(D(x,y),whc), the local contrast weight of image 1 and image 2 is W LC1 , W LC2 .

[0028] Further, the exposure weights of image 1 and image 2 are calculated, including: setting the average brightness and standard deviation σ; according to The calculated exposure weights of image 1 and image 2 are W E1 , W E2 .

[0029] Furthermore, the pixel value C of the Cth channel of the i-th layer of the fused image i for:

[0030] C i =W 1i ·c 1i +W 2i ·c 2i

[0031] Among them, W 1i is the weight coefficient of image 1 in layer i, c 1i is the pixel value of image 1 in channel C of layer i, W 2i is the weight coefficient of image 2 in layer i, c 2i is the pixel value of channel C of image 2 in the i-th layer.

[0032] Compared with the prior art, the present invention has the following beneficial technical effects:

[0033] The present invention discloses an adaptive underwater image ORB feature point extraction method based on image fusion. The method first improves the quality of underwater images, and then calculates a dynamic threshold to perform adaptive ORB feature extraction on the underwater images, thereby effectively improving the quality of the extracted feature points. The traditional ORB feature extraction algorithm is developed by combining the BRIEF and FAST algorithms, and adopts the same method of artificially assigning fixed threshold parameters as the FAST algorithm. However, underwater images often have problems such as low contrast and a large amount of noise. Therefore, if the traditional ORB algorithm is adopted, it will lead to the problem that feature points cannot be extracted or the extracted feature points are of poor quality, thereby affecting the effect of subsequent feature matching. The adaptive underwater image ORB feature point extraction method based on image fusion proposed by the present invention first improves the quality of underwater images, and then performs cell division and adaptive dynamic threshold calculation on the fused image. In cells with rich textures, the coefficient of the dynamic threshold is If it is greater than 1, the dynamic threshold is reduced, which is conducive to extracting more feature points; in cells with weak textures, the coefficient of the dynamic threshold is When it is less than 1, the dynamic threshold increases, which helps to avoid extracting feature points of poor quality, thereby improving the extraction quality of underwater image feature points. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of an adaptive underwater image ORB feature point extraction method based on image fusion according to the present invention;

[0035] Figure 2 is the original image in the embodiment of the present invention;

[0036] Figure 3 is a color correction diagram in an embodiment of the present invention;

[0037] Figure 4 is an image after texture enhancement in an embodiment of the present invention;

[0038] Figure 5 is a fused image in an embodiment of the present invention;

[0039] Figure 6 This is the result map of feature point extraction by the original ORB algorithm;

[0040] Figure 7 This is the result graph of feature point extraction in the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0042] The present invention provides an adaptive underwater image ORB feature point extraction method based on image fusion, such as Figure 1 As shown, it includes: preprocessing the original image to obtain a color correction image and saving it as image 1; converting the color correction image to Lab color space, filtering and contrast enhancement to obtain image 2; converting image 2 to RGB color space, calculating the Laplacian contrast weight, local contrast weight and exposure weight of image 1 and image 2; setting the number of pyramid layers to construct a weighted Gaussian pyramid of image 1 and image 2; constructing a Laplacian pyramid based on the weighted Gaussian pyramid of image 1 and image 2 to output a fused image; calculating the FAST dynamic threshold in ORB based on the fused image; and extracting ORB feature points based on the dynamic threshold.

[0043] Set the initial FAST threshold Th initial , generally set to 20 by default, input the original image, such as Figure 2 As shown. Then the original image is corrected, including gray world white balance processing, specifically: based on the gray world algorithm, the input image is color corrected; the saturation threshold Threshold of each channel C is calculated C , calculated as follows

[0044] Threshold C =0.002×μ C

[0045] μ C =Max / C

[0046] Max=max{R,G,B}

[0047] Among them, R, G, B are the average brightness of red, green, and blue channels, C∈{R,G,B}, μ C is the scaling factor of the corresponding channel; histogram contrast is positively calculated for each channel.

[0048] Determines the percentage position of the low and high ends of the image histogram to be clipped [Threshold C ,1-Threshold C ]. Use the quantile function to find the clipping points tiles of the pixel value distribution. Remove the extreme values ​​at both ends of the histogram to avoid oversaturation or darkening in histogram equalization or contrast enhancement, and improve the overall visual effect of the image. Then perform a linear stretch to expand the pixel value range from [p min ,p max ] is mapped to [0,255] to enhance the contrast. After color correction, we get image 1 as Figure 3 shown.

[0049] The original image is converted from RGB color space to Lab color space to obtain image 1, as shown in Figure 3 shown.

[0050] The original image is converted from RGB color space to Lab color space, and the L channel is subjected to bilateral filtering to remove noise. The L channel of the denoised image 1 is subjected to texture enhancement using the contrast-limited adaptive histogram equalization algorithm CLAHE to obtain image 2, where the parameters clipLimit = 0.25, Distribution = Rayleigh, and image 2 is converted to RGB color space, and the weights are calculated.

[0051] The calculation weights include Laplacian contrast weights, local contrast weights, and exposure weights of image 1 and image 2.

[0052] The Laplacian contrast weights of image 1 and image 2 are calculated as follows:

[0053] Use the Laplacian operator to filter image 1 and image 2, take the absolute value, and save it as the Laplacian contrast weight W L1 Compare the weights W with the Laplacian L2 .

[0054] The local contrast weights of image 1 and image 2 are calculated as follows:

[0055] Define the Gaussian filter h and normalize it, that is,

[0056] Set the high-frequency cutoff value whc=π÷2.75 to avoid over-amplifying low-contrast areas;

[0057] Use Gaussian filter to filter image 1 and image 2, calculate the difference between the pixel values ​​of image 1 and the filtered image, and the difference between the pixel values ​​of image 2 and the filtered image, and square them to enhance the contrast effect, D(x,y) = (I(x,y)-I s (x,y) 2 , where I(x,y) is the original image, I s (x, y) is the filtered image; the difference in pixel value between the original image and the filtered image of Image 1 is D1(x, y) = (I(x, y) - I s (x, y) 2 , the difference between the pixel values ​​of the original image and the filtered image of image 2 is D2(x, y) = (I(x, y) - I s (x, y) 2 ;

[0058] Apply the high-frequency cutoff value whc to D(x, y) to obtain the local contrast weight W LC (x, y) = min(D(x, y), whc); the local contrast weight of image 1 is W LC1 (x, y) = min(D1(x, y), whc), the local contrast weight of image 2 is W Lc2 (x,y)=min(D2(x,y),whc).

[0059] The exposure weights for image 1 and image 2 are calculated as follows:

[0060] Set average brightness and standard deviation σ;

[0061] according to The calculated exposure weights of image 1 and image 2 are W E1 , W E2 .

[0062] The Laplacian contrast weight, local contrast weight and exposure weight of image 1 and image 2 are integrated:

[0063] W1=(W L1 +W LC1 +W E1 ) / W total

[0064] W2=(W L2 +W LC2 +W E2 ) / W total

[0065] W total =W L1 +W LC1 +W E1 +W L2 +W LC2 +W E2

[0066] Set the number of pyramid layers 1evel to 5, and construct the weighted Gaussian pyramids of image 1 and image 2. Construct the Laplacian pyramid based on the weighted Gaussian pyramids of image 1 and image 2, and then perform image fusion to output the fused image, such as Figure 5 As shown, the pixel values ​​of the red R, green G, and blue B channels at different levels are:

[0067] C i =W 1i ·c 1i +W 2i ·c 2i

[0068] Among them, W 1i is the weight coefficient of image 1 in layer i, c 1i is the pixel value of image 1 in channel C of layer i, W 2i is the weight coefficient of image 2 in layer i, c 2i is the pixel value of channel C of image 2 in the i-th layer.

[0069] According to the fused image, the FAST dynamic threshold in ORB is calculated, including the grayscale standard deviation of the fused image. The grayscale standard deviation of the fused image is:

[0070]

[0071] Where W is the width of the image, H is the height of the image, and I(x,y) represents the pixel value of the xth row and yth column. is the mean value of the pixel values ​​of the fused image.

[0072] The fused image is divided into M×N cells, where M is a row and N is a column. In this embodiment, the number is 3×3.

[0073] Calculate the grayscale standard deviation for each cell:

[0074]

[0075] Among them, μ is the mean value of each cell pixel value.

[0076] The standard deviation is used to measure the richness of texture information. The larger the standard deviation, the greater the grayscale difference of the pixel points and the richer the texture information. According to the grayscale standard deviation of each cell, the standard deviation ratio coefficient of each cell and the fused image is calculated. The standard deviation ratio coefficient of each cell and the fused image is:

[0077]

[0078] Calculate the threshold for each cell based on the standard deviation proportionality factor:

[0079]

[0080] The calculated dynamic threshold is used as the adaptive threshold to extract ORB feature points for each cell.

[0081] An adaptive underwater image ORB feature point extraction method based on image fusion first improves the quality of underwater images, and then calculates the dynamic threshold to perform adaptive ORB feature extraction on the underwater images, effectively improving the quality of the extracted feature points. The traditional ORB feature extraction algorithm is developed by combining the BRIEF and FAST algorithms, and uses the same method of manually assigning fixed threshold parameters as the FAST algorithm. Underwater images often have problems such as low contrast and a large amount of noise. Therefore, if the traditional ORB algorithm is used, it will lead to the problem that feature points cannot be extracted or the extracted feature points are of poor quality, thereby affecting the effect of subsequent feature matching. The adaptive underwater image ORB feature point extraction method based on image fusion proposed in the present invention first improves the quality of underwater images, and then performs cell division and adaptive dynamic threshold calculation on the fused image. In cells with rich textures, the coefficient of the dynamic threshold is If it is greater than 1, the dynamic threshold is reduced, which is conducive to extracting more feature points; in cells with weak textures, the coefficient of the dynamic threshold is When it is less than 1, the dynamic threshold increases, which helps to avoid extracting feature points of poor quality, thereby improving the extraction quality of underwater image feature points.

[0082] The present invention is tested in a data set, and the underwater image is preprocessed by image fusion to obtain texture enhancement. Figure 4 As shown in Figure 1, the color-corrected image and the final fused image are evaluated using image quality evaluation indicators. The experimental results are shown in Table 1. The feature points extracted by this method are as follows: Figure 7 The results of feature point extraction using the original ORB algorithm are shown in Figure 6 As shown in the figure, the comparison results show that the quality of underwater images can be effectively improved, and the threshold can be adaptively adjusted based on the fused image to extract more quality and better feature points.

[0083] Table 1

[0084] Evaluation indicators AG IE UIQM UICM UISM UIConM UICQE Original Image 3.4872991 6.6182867 8.5943313 18.23697 0.382421 -0.176875 0.2511816 Fusion Image 11.208849 7.4298638 7.7046478 15.824008 1.279341 -0.20259 0.3711516

[0085] An adaptive underwater image ORB feature point extraction method based on image fusion, which first pre-processes the underwater image and then extracts the ORB feature points. Meanwhile, the original ORB feature point extraction uses a preset fixed threshold, and the present invention can realize adaptive threshold decision according to the richness of texture in each area of ​​the image. The advantage is that it solves the problem of serious degradation of underwater image quality leading to a reduction in the number of feature point detections or low quality of detected feature points. The method can extract more feature points in areas with rich image texture information and avoid extracting poor quality feature points in areas with less image texture information, thereby improving the feature point detection effect for underwater images.

[0086] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

Claims

1. An adaptive underwater image ORB feature point extraction method based on image fusion, characterized in that: include: Preprocess the original image to obtain a color correction image and save it as image 1; The color correction image is converted to Lab color space, filtered and contrast enhanced to obtain image 2; Convert image 2 to RGB color space, and calculate the Laplacian contrast weight, local contrast weight, and exposure weight of image 1 and image 2; Set the number of pyramid layers and construct weighted Gaussian pyramids for image 1 and image 2; Construct a Laplacian pyramid based on the weighted Gaussian pyramids of image 1 and image 2 to output a fused image; According to the fused image, calculate the FAST dynamic threshold in ORB; ORB feature point extraction is performed based on dynamic threshold.

2. The adaptive underwater image ORB feature point extraction method based on image fusion according to claim 1 is characterized in that: The step of calculating the FAST dynamic threshold in ORB according to the fused image includes: Calculate the grayscale standard deviation of the fused image; Divide the fused image into M×N cells, where M is the row and N is the column; Calculate the grayscale standard deviation of each cell; According to the grayscale standard deviation of each cell, the standard deviation ratio coefficient of each cell and the fused image is calculated; The threshold for each cell is calculated based on the standard deviation scaling factor.

3. The adaptive underwater image ORB feature point extraction method based on image fusion according to claim 2 is characterized in that: The grayscale standard deviation of the fused image is: Where W is the width of the image, H is the height of the image, and I(x,y) represents the pixel value of the xth row and yth column. is the mean value of the pixel value of the fused image; The grayscale standard deviation of each cell is: Among them, μ is the mean value of each cell pixel value; The standard deviation scaling factor for each cell and the fused image is: The threshold for each cell is: Among them, Th initial is the preset threshold.

4. The adaptive underwater image ORB feature point extraction method based on image fusion according to claim 1 is characterized in that , the original image is preprocessed, including: Grayscale world white balance processing; Calculate the saturation threshold of each channel C of the original image C ; Perform histogram contrast adjustment on each channel C.

5. The adaptive underwater image ORB feature point extraction method based on image fusion according to claim 4 is characterized in that: The calculation of the saturation threshold of each channel C of the original image is Threshold C ,include: Threshold C =0.002×μ C m C =Max / C Max=max{R,G,B} Among them, R, G, B are the average brightness of red, green, and blue channels, C∈{R,G,B}, μ C is the scaling factor of the corresponding channel.

6. The adaptive underwater image ORB feature point extraction method based on image fusion according to claim 4 is characterized by: The grayscale world white balance process also includes setting an initial FAST threshold Th initial ; The initial FAST threshold Th initial is 20.

7. The adaptive underwater image ORB feature point extraction method based on image fusion according to claim 1 is characterized in that: The step of calculating the Laplacian contrast weights of image 1 and image 2 includes: Use the Laplacian operator to filter image 1 and image 2 to obtain the Laplacian contrast weight W L1 Compare the weights W with the Laplacian L2 .

8. The adaptive underwater image ORB feature point extraction method based on image fusion according to claim 1 is characterized in that: The calculating of the local contrast weights of the image 1 and the image 2 includes: Set the Gaussian filter h and the high frequency cutoff value whc; Use Gaussian filter to filter image 1 and image 2, apply high-frequency cutoff value to D(x,y), and get local contrast weight W LC (x,y)=min(D(x,y),whc), the local contrast weight of image 1 and image 2 is W LC1 , W LC2 .

9. The adaptive underwater image ORB feature point extraction method based on image fusion according to claim 1 is characterized in that: The step of calculating the exposure weights of image 1 and image 2 includes: Set average brightness and standard deviation σ; according to The calculated exposure weights of image 1 and image 2 are W E1 , W E2 .

10. The adaptive underwater image ORB feature point extraction method based on image fusion according to claim 1, characterized in that: The pixel value C of the C channel of the i-th layer of the fused image i for: C i =W 1i ·c 1i +W 2i ·c 2i Among them, W 1i is the weight coefficient of image 1 in layer i, c 1i is the pixel value of image 1 in channel C of layer i, W 2i is the weight coefficient of image 2 in layer i, c 2i is the pixel value of channel C of image 2 in the i-th layer.

Citation Information

Patent Citations

  • ORB algorithm based on threshold adaptive threshold adjustment

    CN113191370A

  • Self-adaptive image matching algorithm fusing edge features

    CN114359591A

  • Multi-scale fusion underwater image enhancement method combined with adaptive gamma correction

    CN116630198A

  • Feature extraction method based on point-line feature fusion and adaptive threshold

    CN117011704A

  • Visual SLAM method based on adaptive uniform division feature point extraction

    CN117274620A