Forward-looking drilling panoramic image splicing method and device based on machine vision

Abstract

The invention relates to a machine vision-based foresight drilling panoramic image splicing method and device, and the method comprises the steps: carrying out the feature detection of each frame of foresight drilling unfolded image, and determining feature points; calculating a descriptor vector of each feature point; for two foresight borehole expanded images of adjacent frames, calculating the Euclidean distance between the descriptor vectors of any two feature points in different images, and determining feature point matching pairs based on the Euclidean distance; removing wrong feature point matching pairs in the feature point matching pairs to obtain final feature point matching pairs; determining a homography matrix between the two foresight borehole expanded images of the adjacent frames; and according to the homography matrix, realizing splicing of the two foresight borehole expanded images of the adjacent frames. According to the method, feature detection and matching are carried out on the adjacent frames of the drilling expanded image, a new feature point description, screening matching and model parameter estimation method is provided, the feature point matching accuracy is improved, and accurate splicing of the foresight drilling panoramic image is realized.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to a method and device for stitching panoramic images of forward-looking drilling based on machine vision. Background Art

[0002] The forward-looking panoramic drilling imaging technology captures the video and images of the hole wall in front of the hole through a CCD camera installed at the tip of the probe, and transmits the video signal to an external-hole monitor or controller for real-time monitoring or viewing by construction personnel, and stores it for subsequent analysis and retrieval. Since the forward-looking panoramic drilling camera is located in front of the probe, the video images are intuitive and have a strong three-dimensional sense, which conforms to the human eye observation habit. Using the forward-looking panoramic drilling imaging technology to observe the inside of the hole is an important means for geological exploration.

[0003] Currently, most construction personnel obtain information about the inner wall of the hole by directly observing the drilling videos or video screenshots obtained by forward-looking panoramic drilling imaging. Its disadvantage is that the interpretation of the drilling video images stays at manual operation and subjective recognition, which is time-consuming and laborious, and human differences will cause deviations in the analysis results. In addition, such drilling videos or video screenshots have not undergone image unfolding and image stitching, and it is impossible to obtain a panoramic unfolded image of the hole wall, and quantitative analysis of geological structures such as bedding planes, fracture zones, fissures, and joints inside the hole cannot be carried out. Summary of the Invention

[0004] To overcome at least one deficiency in the prior art, this application provides a method and device for stitching panoramic images of forward-looking drilling based on machine vision.

[0005] In a first aspect, a method for stitching panoramic images of forward-looking drilling based on machine vision is provided, including:

[0006] Performing feature detection on each frame of the forward-looking drilling unfolded image to determine feature points;

[0007] Calculating the descriptor vector of each feature point;

[0008] For two adjacent-frame forward-looking drilling unfolded images, calculating the Euclidean distance between the descriptor vectors of any two feature points in different images, and determining feature point matching pairs based on the Euclidean distance;

[0009] Removing the incorrect feature point matching pairs in the feature point matching pairs to obtain the final feature point matching pairs;

[0010] Based on the final feature point matching pairs, determining the homography matrix between two adjacent-frame forward-looking drilling unfolded images;

[0011] Implementing the stitching of two adjacent-frame forward-looking drilling unfolded images according to the homography matrix.

[0012] In one embodiment, feature detection is performed on each frame of the forward-looking borehole unfolding image to determine feature points, including:

[0013] Perform Laplacian sharpening filtering on the forward-looking borehole unfolding image to obtain a Laplacian transform image;

[0014] Perform convolution operations on the Laplacian transform image using 3 Sobel operators to obtain the Hessian matrix of each pixel point in the 3-layer scale space, and calculate the determinant value of the Hessian matrix. The determinant values of the Hessian matrices of all pixel points form a Hessian matrix determinant value image;

[0015] For any pixel point in the Hessian matrix determinant value image, if the determinant value of the Hessian matrix of the pixel point is greater than or less than the determinant values of the Hessian matrices of other pixel points in the neighborhood of the pixel point, then the pixel point is a potential feature point.

[0016] Use 3D linear interpolation method for all potential feature points to obtain interpolated feature points, and remove the feature points with determinant values of the Hessian matrix less than the first set threshold from the interpolated feature points to obtain the final feature points.

[0017] In one embodiment, the first set threshold is determined using the following formula:

[0018] r = E + 1.5×σ

[0019] where r is the first set threshold, E is the mean value of the determinant values of the Hessian matrices of all pixel points, and σ is the standard deviation of the determinant values of the Hessian matrices of all pixel points.

[0020] In one embodiment, calculating the descriptor vector of each feature point includes:

[0021] For each feature point, determine the dominant direction of the feature point;

[0022] Construct a square region centered on the feature point, and divide the square region into multiple sub-regions along the dominant direction of the feature point;

[0023] Use Haar wavelets to calculate the Haar wavelet response value of each sub-region;

[0024] Determine the descriptor vector corresponding to each sub-region according to the Haar wavelet response value;

[0025] The descriptor vectors corresponding to all sub-regions are combined to form the descriptor vector of the feature point.

[0026] In one embodiment, for each feature point, determining the dominant direction of the feature point includes:

[0027] Taking the feature point as the center, a circular area with a radius of multiple times the scale of the feature point is used as the circular neighborhood;

[0028] Within the circular neighborhood, a fan-shaped sliding window with a central angle of 60° centered on the feature point is determined. The fan-shaped sliding window rotates within the circular neighborhood, and the Haar wavelet response values within the fan-shaped sliding window at different rotation angles are calculated; the direction at the rotation angle corresponding to the maximum value of the Haar wavelet response value is selected as the dominant direction of the feature point.

[0029] In one embodiment, removing the incorrect feature point matching pairs in the feature point matching pairs to obtain the final feature point matching pairs includes:

[0030] All feature point matching pairs form a data set; multiple feature point matching pairs are randomly selected from all feature point matching pairs to form a minimum sample subset, and the feature points in the multiple feature point matching pairs are not collinear; the random selection process is repeated S times to obtain S minimum sample subsets;

[0031] For each minimum sample subset, the parameters of the homography matrix are estimated using the minimum sample subset to obtain the fitted homography matrix;

[0032] For each fitted homography matrix, the feature points of the front view borehole unfolded image in the previous frame of the feature point matching pair are transformed to the coordinate space of the front view borehole unfolded image in the next frame through the fitted homography matrix to obtain the transformed feature points;

[0033] Calculate the distance error between the transformed feature points and the feature points of the front view borehole unfolded image in the next frame of the feature point matching pair; calculate the likelihood value of the data set with respect to the fitted homography matrix based on the distance error;

[0034] Select the fitted homography matrix corresponding to the maximum value of the likelihood value as the final fitted homography matrix;

[0035] The feature points of the front view borehole unfolded image in the previous frame of the feature point matching pair are transformed to the coordinate space of the front view borehole unfolded image in the next frame through the final fitted homography matrix to obtain the transformed feature points; calculate the distance error between the transformed feature points and the feature points of the front view borehole unfolded image in the next frame of the feature point matching pair; determine whether the distance error is greater than the second set threshold. If so, delete the feature point matching pair corresponding to the distance error. If not, retain the feature point matching pair corresponding to the distance error.

[0036] In one embodiment, calculating the likelihood value of the data set with respect to the fitted homography matrix based on the distance error uses the following formula:

[0037]

[0038] Among them, L j is the likelihood value, j is the label of the minimum sample subset, i is the label of the feature point matching pair, N is the number of feature point matching pairs, σ is the standard deviation of the determinant values of the Hessian matrix of all pixel points, and e ij is the distance error determined by transforming the feature point of the forward-looking drilling unfolded image in the previous frame in the feature point matching pair i through the homography matrix fitted corresponding to the minimum sample subset j.

[0039] In one embodiment, the second set threshold is determined by the following formula:

[0040]

[0041] Among them, ε is the second set threshold, i is the label of the feature point matching pair, N is the number of feature point matching pairs, and e iK is the distance error determined by transforming the feature point of the forward-looking drilling unfolded image in the previous frame in the feature point matching pair i through the final fitted homography matrix, and K is the label of the minimum sample subset corresponding to the final fitted homography matrix.

[0042] In a second aspect, a forward-looking drilling panoramic image stitching device based on machine vision is provided, including:

[0043] A feature detection module, configured to perform feature detection on each frame of the forward-looking drilling unfolded image to determine feature points;

[0044] A feature description module, configured to calculate the descriptor vector of each feature point;

[0045] A feature point matching module, configured to calculate the Euclidean distance between the descriptor vectors of any two feature points in different images for two adjacent-frame forward-looking drilling unfolded images, and determine feature point matching pairs based on the Euclidean distance;

[0046] A feature point screening module, configured to remove the incorrect feature point matching pairs in the feature point matching pairs to obtain the final feature point matching pairs;

[0047] A homography matrix determination module, configured to determine the homography matrix between two adjacent-frame forward-looking drilling unfolded images based on the final feature point matching pairs;

[0048] An image stitching module, configured to stitch two adjacent-frame forward-looking drilling unfolded images according to the homography matrix.

[0049] In a third aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it realizes the above-mentioned forward-looking drilling panoramic image stitching method based on machine vision.

[0050] Compared with the prior art, the present application has the following beneficial effects:

[0051] 1. Due to the special working conditions environment, the quality of the forward-looking drilling hole image is poor. When using the traditional Scale-Invariant Feature Transform algorithm (SIFT algorithm) and Speeded-Up Robust Features SURF algorithm, there are problems of mis-extraction of feature points and low correct rate of feature point matching. The present application performs feature detection and matching on adjacent frames of the drilled hole unfolded image. By using bilateral filtering to replace Gaussian filtering to extract SURF key feature points in the image, a new descriptor method is proposed to solve the problem of mis-extraction of feature points in the drilling hole image and improve the correct rate of feature point matching.

[0052] 2. The present application screens the extracted feature points, effectively eliminates mis-matched points, and obtains accurate model parameters and matching point pairs. Compared with the commonly used Random Sample Consensus algorithm (RANSAC algorithm), the present application has stronger adaptability to the distribution of data points, more accurate model selection, and stronger algorithm robustness, which enables it to achieve better results in eliminating mis-matched points and estimating model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present application can be better understood by referring to the description given below in conjunction with the accompanying drawings. The drawings, together with the following detailed description, are included in this specification and form a part of this specification. In the drawings:

[0054] Figure 1 A flowchart showing a method for stitching panoramic images of forward-looking drilling holes based on machine vision is shown;

[0055] Figure 2 A structural block diagram showing a device for stitching panoramic images of forward-looking drilling holes based on machine vision is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] Hereinafter, exemplary embodiments of the present application will be described in conjunction with the accompanying drawings. For clarity and conciseness, not all features of the actual embodiments are described in the specification. However, it should be understood that many specific decisions specific to the embodiments can be made during the development of any such actual embodiment to achieve the specific goals of the developer, and these decisions may vary with different embodiments.

[0057] Here, it should also be noted that in order to avoid obscuring the present application due to unnecessary details, only the device structures closely related to the solution of the present application are shown in the drawings, and other details less related to the present application are omitted.

[0058] It should be understood that the present application is not limited to the described embodiments only due to the following description with reference to the accompanying drawings. In this document, where feasible, embodiments can be combined with each other, features can be replaced or borrowed between different embodiments, and one or more features can be omitted in one embodiment.

[0059] An embodiment of the present application provides a method for stitching panoramic images of forward-looking drilling based on machine vision. Figure 1 The flowchart of the method for stitching panoramic images of forward-looking drilling based on machine vision is shown. Refer to Figure 1 , the method includes:

[0060] Step S1, perform feature detection on each frame of the forward-looking drilling unfolded image to determine feature points.

[0061] Here, forward-looking drilling imaging is a common technical means for drilling geological exploration. The forward-looking drilling video image is processed by unfolding to obtain an effective rectangular unfolded image.

[0062] Step S2, calculate the descriptor vector of each feature point.

[0063] Step S3, for two adjacent frames of forward-looking drilling unfolded images, calculate the Euclidean distance between the descriptor vectors of any two feature points in different images, and determine the feature point matching pairs based on the Euclidean distance.

[0064] Step S4, remove the incorrect feature point matching pairs in the feature point matching pairs to obtain the final feature point matching pairs.

[0065] Step S5, based on the final feature point matching pairs, determine the homography matrix between two adjacent frames of forward-looking drilling unfolded images.

[0066] Step S6, implement the stitching of two adjacent frames of forward-looking drilling unfolded images according to the homography matrix.

[0067] In this embodiment, feature detection and matching are performed on adjacent frames of the forward-looking drilling unfolded images. By using bilateral filtering instead of Gaussian filtering to extract SURF key feature points in the image, a new descriptor method is proposed to solve the problem of mis-extraction of feature points in drilling images, and the correct rate of feature point matching is improved; the extracted feature points are screened to effectively remove mis-matched points, and accurate model parameters and matching point pairs are obtained. This embodiment has stronger adaptability to the distribution of data points, more accurate model selection, and stronger algorithm robustness, which enables it to achieve better results in removing mis-matched points and estimating model parameters.

[0068] In one embodiment, in step S1, performing feature detection on each frame of the forward-looking drilling unfolded image to determine feature points includes:

[0069] Step S11, perform Laplacian sharpening filtering on the forward-looking drilling expansion image to enhance the image edges and details, and obtain the Laplace transform image.

[0070] Step S12, perform convolution operations on the Laplace transform image using 3 Sobel operators to obtain the Hessian matrix of each pixel point in the 3-layer scale space, and calculate the determinant value of the Hessian matrix. The determinant values of the Hessian matrices of all pixel points form the Hessian matrix determinant value image.

[0071] Step S13, for any pixel point in the Hessian matrix determinant value image, if the determinant value of the Hessian matrix of the pixel point is greater than or less than the determinant values of the Hessian matrices of other pixel points in the 3×3×3 neighborhood of the pixel point, that is, a local extreme point, then the pixel point is a potential feature point. Here, the 3×3×3 neighborhood of the pixel point covers the same-layer two-dimensional neighborhood and the scale neighborhoods of adjacent layers.

[0072] Step S14, use the 3D linear interpolation method for all potential feature points to obtain the interpolated feature points, and remove the feature points with determinant values of the Hessian matrix less than the first set threshold from the interpolated feature points to obtain the final feature points. The first set threshold adopts an adaptive threshold method based on statistical analysis, performs statistical analysis on the determinant values of the Hessian matrices of all calculated pixel points, obtains its mean value E and standard deviation σ, and the first set threshold is determined by the following formula:

[0073] r = E + 1.5×σ

[0074] where r is the first set threshold, E is the mean value of the determinant values of the Hessian matrices of all pixel points, and σ is the standard deviation of the determinant values of the Hessian matrices of all pixel points.

[0075] In one embodiment, in step S2, calculating the descriptor vector of each feature point includes:

[0076] Step S21, for each feature point, determine the dominant direction of the feature point.

[0077] Specifically, with the feature point as the center, take a circular area with a radius of multiple times the feature point scale as the circular neighborhood; here, the multiple times the feature point scale can be 6 times the feature point scale, and the feature point scale refers to performing convolution operations on the Laplace transform image using 3 Sobel operators to obtain the scale space of the pixel point, such as 1, 2 or 3.

[0078] Within the circular neighborhood, a sector-shaped sliding window with a central angle of 60° centered at the feature point is determined. The sector-shaped sliding window rotates within the circular neighborhood, and the Haar wavelet response values within the sector-shaped sliding window at different rotation angles are calculated. The direction corresponding to the rotation angle at which the maximum Haar wavelet response value is selected is taken as the dominant direction of the feature point. Here, for example, when the rotation angle corresponding to the maximum Haar wavelet response value is 5°, the direction at the rotation angle refers to the direction after rotating 5° from the initial direction. The Haar wavelet response value within the sector-shaped sliding window refers to the sum of the Haar wavelet response values of all feature points within the sector-shaped sliding window.

[0079] Step S22: Construct a square region centered at the feature point and divide the square region into multiple sub-regions along the dominant direction of the feature point.

[0080] Here, the side length of the square region can be 20 times the scale of the feature point. The square region is divided into 4×4 sub-regions along the dominant direction of the feature point, and each sub-region has 5 times the scale of the feature point number of pixels.

[0081] Step S23: Use Haar wavelets to calculate the Haar wavelet response value of each sub-region.

[0082] Step S24: Determine the descriptor vector corresponding to each sub-region according to the Haar wavelet response value. Then, a 4-dimensional descriptor vector {∑dx ∑|dx| ∑dy ∑|dy|} is formed in each sub-region, where dx and dy are two components of the Haar wavelet response value.

[0083] Step S25: The descriptor vectors corresponding to all sub-regions are combined to form the descriptor vector of the feature point. The 4×4 sub-region descriptor vectors are combined into a 64-dimensional feature point descriptor vector and converted into a unit vector.

[0084] In one embodiment, step S4: Remove the incorrect feature point matching pairs in the feature point matching pairs to obtain the final feature point matching pairs, including:

[0085] Step S41: All feature point matching pairs form a data set; randomly select multiple feature point matching pairs from all feature point matching pairs. Here, 2 feature point matching pairs can be selected to form the minimum sample subset, and the feature points in the multiple feature point matching pairs are not collinear; repeat the random selection process S times to obtain S minimum sample subsets;

[0086] Step S42: For each minimum sample subset S j , j = 1, 2, …, S, j is the label of the minimum sample subset, and S is the number of minimum sample subsets. Use the minimum sample subset S jEstimate the homography matrix parameters h1, h2, h3, h4, h5, h6, h7, h8 to obtain the fitted homography matrix; the fitted homography matrix H j = [h1, h2, h3, h4, h5, h6, h7, h8, 1].

[0087] Step S43. For each fitted homography matrix H j , transform the feature points of the front view drilling expansion image in the previous frame in the feature point matching pair to the coordinate space of the front view drilling expansion image in the next frame through the fitted homography matrix to obtain the transformed feature points;

[0088] Step S44. Calculate the distance error between the transformed feature points and the feature points of the front view drilling expansion image in the next frame in the feature point matching pair; calculate the likelihood value of the data set with respect to the fitted homography matrix based on the distance error; here, each fitted homography matrix H j corresponds to a likelihood value L j , which is calculated using the following formula:

[0089]

[0090] where L j is the likelihood value, j is the label of the minimum sample subset, i is the label of the feature point matching pair, N is the number of feature point matching pairs, σ is the standard deviation of the determinant values of the Hessian matrix of all pixel points, and e ij is the distance error determined by transforming the feature points of the front view drilling expansion image in the previous frame in the feature point matching pair i through the fitted homography matrix corresponding to the minimum sample subset j.

[0091] Step S45. Select the fitted homography matrix corresponding to the maximum likelihood value as the final fitted homography matrix H K , K is the label of the minimum sample subset corresponding to the final fitted homography matrix.

[0092] Step S46. Transform the feature points of the front view drilling expansion image in the previous frame in the feature point matching pair to the coordinate space of the front view drilling expansion image in the next frame through the final fitted homography matrix to obtain the transformed feature points; calculate the distance error between the transformed feature points and the feature points of the front view drilling expansion image in the next frame in the feature point matching pair; determine whether the distance error is greater than the second set threshold. If so, delete the feature point matching pair corresponding to the distance error. If not, retain the feature point matching pair corresponding to the distance error.

[0093] The second set threshold is determined using the following formula:

[0094]

[0095] where ε is the second set threshold, i is the label of the feature point matching pair, N is the number of feature point matching pairs, and e iK is the distance error determined by transforming the feature point of the front view drilling expansion image of the previous frame in the feature point matching pair i through the final fitted homography matrix, and K is the label of the minimum sample subset corresponding to the final fitted homography matrix.

[0096] Specifically, in step S5, based on the final feature point matching pairs, the parameters h1, h2, h3, h4, h5, h6, h7, h8 of the homography matrix are estimated to determine the homography matrix between two front view drilling expansion images of adjacent frames.

[0097] Specifically, in step S6, the parameters h3 and h6 in the homography matrix between two front view drilling expansion images of adjacent frames respectively represent the horizontal offset and vertical offset in the matching of two adjacent frames of images. Based on h3 and h6 and the existing image fusion method (for example, due to the interference of factors such as brightness and gray scale changes, there may be obvious gaps in the directly stitched panoramic image, which may affect the observation. Weighted fusion and other methods can be used during the stitching process to eliminate these effects), the accurate stitching of the two frames of images can be completed. By repeatedly operating the above image stitching method, the stitching of the front view drilling panoramic image with a large viewing angle can be realized.

[0098] Adopting the same inventive concept as the method for stitching the front view drilling panoramic image based on machine vision, this embodiment also provides a corresponding device for stitching the front view drilling panoramic image based on machine vision. Figure 2 The structural block diagram of the device for stitching the front view drilling panoramic image based on machine vision is shown, including:

[0099] A feature detection module 21, configured to perform feature detection on each frame of the front view drilling expansion image to determine feature points;

[0100] A feature description module 22, configured to calculate the descriptor vector of each feature point;

[0101] A feature point matching module 23, configured to calculate the Euclidean distance between the descriptor vectors of any two feature points in different images for two front view drilling expansion images of adjacent frames, and determine feature point matching pairs based on the Euclidean distance;

[0102] A feature point screening module 24, configured to remove the incorrect feature point matching pairs in the feature point matching pairs to obtain the final feature point matching pairs;

[0103] A homography matrix determination module 25, configured to determine the homography matrix between two front view drilling expansion images of adjacent frames based on the final feature point matching pairs;

[0104] An image stitching module 26, configured to stitch two forward-looking drilled expanded images of adjacent frames according to a homography matrix.

[0105] The forward-looking drilled panoramic image stitching device based on machine vision in this embodiment has the same inventive concept as the above-mentioned forward-looking drilled panoramic image stitching method based on machine vision. Therefore, the specific implementation of this device can be seen in the embodiment part of the forward-looking drilled panoramic image stitching method based on machine vision in the foregoing, and its technical effects correspond to those of the above method, which will not be elaborated here.

[0106] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it is used to implement the above-mentioned forward-looking drilled panoramic image stitching method based on machine vision.

[0107] As described above, these are only various embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A forward-looking drilling panoramic image stitching method based on machine vision, characterized in that: include: Perform feature detection on each frame of forward-looking drilling expansion image to determine feature points; Calculate a descriptor vector for each of the feature points; For two forward-looking borehole expansion images of adjacent frames, calculating the Euclidean distance between the descriptor vectors of any two feature points in different images, and determining a feature point matching pair based on the Euclidean distance; Removing erroneous feature point matching pairs from the feature point matching pairs to obtain final feature point matching pairs; Determining a homography matrix between two forward-looking borehole expansion images of adjacent frames based on the final feature point matching pairs; The stitching of two forward-looking borehole expansion images of adjacent frames is achieved according to the homography matrix.

2. The method according to claim 1, characterized in that in, Perform feature detection on each frame of forward-looking drilling expansion image to determine feature points, including: Performing Laplace sharpening filtering on the forward-looking borehole expansion image to obtain a Laplace transform image; Using three sobel operators to perform convolution operation with the Laplace transform image, obtaining the Hessian matrix of each pixel point in the three-layer scale space, and calculating the Hessian matrix determinant value, the Hessian matrix determinant values ​​of all the pixels constitute a Hessian matrix determinant value image; For any pixel point in the Hessian matrix determinant value image, if the Hessian matrix determinant value of the pixel point is greater than or less than the Hessian matrix determinant values ​​of other pixel points in the neighborhood of the pixel point, then the pixel point is a potential feature point. A three-dimensional linear interpolation method is used for all potential feature points to obtain interpolated feature points, and feature points whose Hessian matrix determinant values ​​are less than a first set threshold are removed from the interpolated feature points to obtain final feature points.

3. The method according to claim 2, characterized in that The first set threshold is determined by the following formula: r=E+1.5×σ Wherein, r is the first set threshold, E is the mean value of the Hessian matrix determinant values ​​of all pixels, and σ is the standard deviation of the Hessian matrix determinant values ​​of all pixels.

4. The method according to claim 1, characterized in that in, Calculating the descriptor vector of each feature point includes: For each feature point, determine the dominant direction of the feature point; Constructing a square area with the feature point as the center, and dividing the square area into a plurality of sub-areas along the dominant direction of the feature point; Use Haar wavelet to calculate the Haar wavelet response value of each sub-region; Determine a descriptor vector corresponding to each sub-region according to the Haar wavelet response value; The descriptor vectors corresponding to all sub-regions are combined to form the descriptor vector of the feature point.

5. The method according to claim 4, characterized in that in, For each feature point, determine the dominant direction of the feature point, including: Taking the feature point as the center, a circular area with a radius that is multiple times the feature point scale is taken as the circular neighborhood; In the circular neighborhood, a fan-shaped sliding window with the feature point as the center and an opening angle of 60° is determined, the fan-shaped sliding window is rotated in the circular neighborhood, and the Haar wavelet response values ​​in the fan-shaped sliding window at different rotation angles are calculated; the direction at the rotation angle corresponding to the maximum value of the Haar wavelet response value is selected as the dominant direction of the feature point.

6. The method according to claim 1, characterized in that in, Removing erroneous feature point matching pairs from the feature point matching pairs to obtain final feature point matching pairs, including: All feature point matching pairs constitute a data set; randomly select multiple feature point matching pairs from all feature point matching pairs to constitute a minimum sample subset, where the feature points in the multiple feature point matching pairs are not collinear; repeat the random selection process S times to obtain S minimum sample subsets; For each minimum sample subset, the minimum sample subset is used to estimate homography matrix parameters to obtain a fitted homography matrix; For each fitted homography matrix, transform the feature points of the previous frame of the forward-looking drilling expansion image in the feature point matching pair to the coordinate space of the next frame of the forward-looking drilling expansion image through the fitted homography matrix to obtain the transformed feature points; Calculating the distance error between the transformed feature point and the feature point of the forward-looking borehole expansion image in a frame after the feature point matching; calculating the likelihood value of the data set with respect to the fitted homography matrix based on the distance error; Selecting the fitted homography matrix corresponding to the maximum value of the likelihood value as the final fitted homography matrix; The feature points of the previous frame of the forward-looking drilling expansion image in the feature point matching pair are transformed to the coordinate space of the next frame of the forward-looking drilling expansion image through the final fitting homography matrix to obtain the transformed feature points; the distance error between the transformed feature points and the feature points of the next frame of the forward-looking drilling expansion image in the feature point matching pair is calculated; and it is determined whether the distance error is greater than a second set threshold value, and if so, the feature point matching pair corresponding to the distance error is deleted, and if not, the feature point matching pair corresponding to the distance error is retained.

7. The method according to claim 6, characterized in that in, The likelihood value of the data set with respect to the fitted homography matrix is ​​calculated based on the distance error using the following formula: Among them, L j is the likelihood value, j is the label of the minimum sample subset, i is the label of the feature point matching pair, N is the number of feature point matching pairs, σ is the standard deviation of the Hessian matrix determinant value of all pixels, e ij It is the distance error determined by transforming the feature points of the previous frame forward-looking borehole expansion image in the feature point matching pair i through the fitted homography matrix corresponding to the minimum sample subset j.

8. The method according to claim 6, characterized in that The second set threshold is determined by the following formula: Where ε is the second set threshold, i is the number of feature point matching pairs, N is the number of feature point matching pairs, and e iK is the distance error determined by transforming the feature points of the previous frame forward-looking drilling expansion image in feature point matching pair i through the final fitting homography matrix, and K is the label of the minimum sample subset corresponding to the final fitting homography matrix.

9. A forward-looking drilling panoramic image stitching device based on machine vision, characterized in that: include: A feature detection module is used to perform feature detection on each frame of the forward-looking drilling expansion image and determine feature points; A feature description module, used for calculating a descriptor vector of each feature point; A feature point matching module, for calculating the Euclidean distance between the descriptor vectors of any two feature points in different images for two forward-looking borehole expansion images of adjacent frames, and determining a feature point matching pair based on the Euclidean distance; A feature point screening module is used to remove erroneous feature point matching pairs from the feature point matching pairs to obtain final feature point matching pairs; A homography matrix determination module, used for determining a homography matrix between two forward-looking borehole expansion images of adjacent frames based on the final feature point matching pair; The image stitching module is used to stitch two forward-looking borehole expansion images of adjacent frames according to the homography matrix.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the forward-looking drilling panoramic image stitching method based on machine vision according to any one of claims 1 to 8 is implemented.

Images