A virtual core imaging device and method based on forward-looking panoramic borehole video

Abstract

The application discloses a virtual core imaging device and method based on forward-looking panoramic drilling camera shooting. The virtual core imaging device is composed of a winding and unwinding frame hole support, a depth encoder, an electric capstan, a video transmission line, a data transmission line, a control host, a measuring probe, a computer and a supporting software system. The virtual core imaging method is to continuously shoot along the hole by using a modified forward-looking panoramic drilling camera shooting system to collect high-quality hole wall image sequences, and to process the hole wall image sequences based on an image preprocessing algorithm, a hole wall three-dimensional reconstruction algorithm, a hole wall image splicing algorithm and a real scene model generation algorithm to realize the construction of a real scene hole wall unfolding graph and a core real scene three-dimensional model. The application describes the stratum geology from two angles of the three-dimensional model and the unfolding graph, realizes full-dimension and high-precision drilling information analysis, and can more accurately monitor the deformation of the inclined pipe, thereby providing a guarantee for the safety and stability of the project.

Description

Technical Field

[0001] This invention relates to the field of engineering geological exploration, belonging to geological borehole analysis technology, and specifically to a virtual core modeling technology based on forward-looking borehole photography. Background Technology

[0002] During construction, it is often necessary to understand the mechanical properties and stability of the rock mass in the project area. The type, distribution, density, and development of structural planes within the rock mass are key factors affecting its structural stability, mechanical properties, and engineering characteristics. Many major engineering accidents occur because, in the early stages of construction, the internal structure of the rock mass is not clearly understood, and weak structural planes such as fissures, fracture surfaces, and weak interlayers are not accurately identified. Therefore, accurately understanding and describing the internal structure and mechanical properties of rock masses is an important and continuously evolving topic in engineering.

[0003] Forward-looking panoramic borehole camera systems are a commonly used method for borehole exploration. They acquire image data of the borehole wall using a CCD camera mounted on the probe, transmit this data to a monitor for real-time monitoring by engineers, and output it to a recorder for archiving and later analysis. Engineers can observe the borehole wall video recordings within the effectively illuminated area through the monitoring / recording end and analyze the rock mass by video playback or screenshots. This system has a wide range of applications, relatively low requirements for borehole quality, and is lightweight and inexpensive. However, its drawbacks include a large amount of image data and the inability to provide panoramic views of the borehole or "virtual core" models, resulting in limited quantitative analysis capabilities for the rock mass. Summary of the Invention

[0004] The technical problem to be solved by this invention is to optimize the quantitative analysis capabilities of existing equipment, realize the panoramic unfolding of borehole walls and the construction of "virtual cores", and realize the rapid stitching and fusion of panoramic unfolding images of cores from borehole wall image data by coupling image processing, visual reconstruction and point cloud modeling, and realize the "dimensional upscaling" inversion of the core three-dimensional model.

[0005] To solve the above technical problems, the present invention adopts the following technical solution:

[0006] First, this invention proposes a virtual core imaging device based on forward-looking panoramic borehole camera, comprising: a borehole support, a depth encoder, an electric winch, a video transmission line, a data transmission line, a control host, a measuring probe, and a computer system; wherein,

[0007] The orifice bracket is placed above the orifice of the drilled hole;

[0008] The electric winch includes a stepper motor and a cable disc. The stepper motor drives the cable disc to rotate, providing a smooth lowering or lifting speed. The cable disc is connected to the orifice bracket via a fixed structure.

[0009] The depth encoder is mounted on the orifice bracket and is used to record the real-time depth information of the measuring probe and the extension and retraction length of the video transmission line.

[0010] The video transmission line is used to transmit image data recorded by the measuring probe. One end of the line is connected to the measuring probe, and the other end is wound around the cable reel after passing through a depth encoder.

[0011] The data transmission line is used to transmit image data with depth values. One end of the line is connected to a depth encoder, and the other end is connected to the control host and the computer system in sequence via an electric winch.

[0012] The measuring probe is used to record borehole rock wall image data. It is lowered at a constant speed to the bottom of the borehole by an electric winch via a video transmission line and then pulled up to the borehole opening. The measuring probe includes a distortion-free camera and an illumination device. The distortion-free camera is used to collect borehole rock wall image data and is placed coaxially with the borehole during placement.

[0013] The control host extracts frames from the real-time received image data to form an image sequence, and performs time matching between the image sequence and the probe depth information to generate a borehole image sequence with probe depth information.

[0014] A computer system is used to perform three-dimensional reconstruction of the borehole rock wall and stitch together the borehole rock wall images based on a sequence of borehole images with probe depth information to construct a virtual rock core.

[0015] On the other hand, the present invention also proposes an imaging method based on the virtual core imaging device, the specific steps of which are as follows:

[0016] Step S1: Align the measuring probe with the borehole axis, start the equipment with the control host, zero the depth encoder, and check if the control host has real-time feedback of the drilling image.

[0017] Step S2: After successful debugging, start the stepper motor to make the measuring probe descend to the bottom of the hole at a constant speed; after stabilizing for a period of time, start the stepper motor to pull up the measuring probe and transmit the borehole rock wall image data in real time. The depth encoder transmits the real-time depth of the measuring probe in the borehole to the control host; the control host extracts frames from the image data to form an image sequence, and matches the image sequence with the probe depth in time to generate a borehole image sequence with probe depth information.

[0018] Step S3: For the borehole image sequence, the minimum radius R of the borehole center and rock ring is identified using the Hough circle detection method. min ;

[0019] Step S4: Divide the borehole image sequence using a mask sequence to generate a valid image sequence;

[0020] Step S5: Extract the DSP-SIFT features and SuperPoint features of the effective image sequence respectively, and perform sequential matching;

[0021] Step S6: Project all matched DSP-SIFT features and SuperPoint features onto the image respectively, and generate a feature association network for the image sequence through matching pairs. Where I is the valid image, T is the feature point trajectory, and L is the edge of the trajectory. At the same time, geodesic consistency is used to remove mismatches from the feature association network.

[0022] Step S7: Construct a normalized model of the core model through initial setup, depth calculation, and bundled optimization modules;

[0023] Step S8: Use a statistical denoising algorithm to remove noise from the model and restore the scale of the normalized model based on the depth values ​​of the effective image sequence.

[0024] Step S9: Use FPFH features for coarse matching, and then use the ICP algorithm to achieve precise coupling of the core 3D model;

[0025] Step S10: Perform cylindrical fitting on the core 3D model using a random consistency sampling algorithm to obtain the borehole axis and cylinder radius r. Calculate the vertical distance from the 3D point on the core 3D model to the cylinder axis and subtract it from the cylinder radius r to obtain the axial depth information of the 3D point.

[0026] Step S11: Thin and filter the effective image sequence to generate a circular image sequence. Construct the pixel mapping relationship between the Cartesian coordinate system of the rectangular unfolded image and the polar coordinate system of the circular image sequence through the coordinate transformation relationship between the two. And combine the bilinear interpolation algorithm to unfold the image.

[0027] Step S12: Perform template matching to stitch the rectangular unfolded image: First, set the previous frame image as the reference frame and the current frame image as the target frame, and select a template image in the reference frame image to perform template matching with the current frame; use the template matching algorithm to obtain the matching parameters of the two frames to perform template matching of the rectangular unfolded image; then, perform template matching and stitching with the next frame to generate the new image; finally, stitch them together to form a complete panoramic hole wall unfolded image.

[0028] Step S13: Associate the core 3D model and the panoramic borehole wall unfolded image by mutual assignment: Based on the axial depth information of the core 3D model, map each 3D point of the core 3D model onto the panoramic borehole wall unfolded image to form a point cloud model with elevation information. Then, assign colors to the 3D points according to the RGB information of the corresponding positions in the panoramic borehole wall unfolded image to generate a real-scene borehole wall unfolded image. Finally, assign the RGB information of the 3D points to the core 3D model to generate a real-scene core 3D model.

[0029] The technical effects of this invention, which adopts the above technical solution, compared with the prior art are as follows:

[0030] 1) This invention enables the assignment of axial depth to the panoramic hole wall unfolded diagram, which can quickly and effectively generate high-quality real-scene hole wall unfolded diagrams.

[0031] 2) This invention constructs a three-dimensional model of the rock core by relying on image sequences, making the stratigraphic data three-dimensional and visualized, which greatly reduces the deviation of two-dimensional interpretation results.

[0032] 3) This invention integrates technical challenges into the algorithm level, requiring lower supporting costs; and it is basically based on existing borehole camera systems, which can be quickly integrated into existing geological exploration and other fields of work. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the device of the present invention.

[0034] Figure 2 This is a system framework diagram of the present invention.

[0035] Figure 3 This is a partial flowchart of the algorithm of the present invention. Detailed Implementation

[0036] To make the objectives and technical advantages of this invention clearer, the invention will be further described in detail below. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0037] like Figure 1 As shown, this invention proposes a virtual core imaging device based on forward-looking panoramic drilling camera. The device is an improvement on the commonly used YTJ20 forward-looking panoramic drilling camera system and includes the following components: borehole support 1, depth encoder 2, electric winch 3, video transmission line 4, data transmission line 5, control host 6, measuring probe 7, computer and supporting software system 8, etc.

[0038] The borehole bracket 1 is placed above the borehole opening, and the depth encoder 2 is mounted on the borehole bracket 1. The electric winch 3 includes a stepper motor 3.1 and a cable disc 3.2. The stepper motor 3.1 drives the rotation of the cable disc 3.2 to provide a smooth lowering / lifting speed.

[0039] The video transmission line 4 is used to transmit image data recorded by the measuring probe. The depth encoder 2 records the extension and retraction length of the video transmission line 4. The data transmission line 5 connects the depth encoder 2, the electric winch 3, the control host 6, and the computer system 8, and is used to transmit image data with depth values. The control host 6 provides real-time monitoring on-site and stores the image data with depth values.

[0040] The measuring probe 7 is positioned as coaxially as possible with the borehole wall to obtain full-circle data of the borehole rock wall. Its main components are a distortion-free camera 7.1 and an illumination device 7.2. The distortion-free camera 7.1 is the core device for data acquisition, used to collect image data. The illumination device 7.2 provides necessary and effective lighting.

[0041] To ensure clear borehole images, the measuring probe 7, under the control of the electric winch 3, is lowered to the bottom of the borehole at a constant speed of 5 cm / s and then pulled back to the borehole opening to record borehole wall image data. The depth encoder 2 records the real-time depth information of the measuring probe. Subsequently, the control host 6 extracts frames from the image data at a rate of 10 frames per second to form an image sequence, and then performs time matching between the image sequence and the probe depth to generate a borehole image sequence with probe depth information.

[0042] Furthermore, to obtain a 3D model of the borehole core and a unfolded view of the borehole wall, a computer and supporting software system 8 performs 3D reconstruction of the borehole wall based on a sequence of borehole images containing probe depth information and stitches the borehole wall images together to construct a "virtual core." The flowchart of this method is shown below. Figure 2 As shown.

[0043] like Figure 2 As shown, this invention proposes a virtual core imaging method based on the aforementioned virtual core imaging device, the specific steps of which are as follows:

[0044] Step S1: Assemble the above equipment and place it at the borehole opening. Align the measuring probe with the borehole axis, start the equipment using the control host, zero the depth encoder, and check if the control host provides real-time feedback of the borehole image.

[0045] Step S2: After successful debugging, start the stepper motor to lower the measuring probe to the bottom of the hole at a constant speed; after stabilizing for 3-5 minutes, start the stepper motor to pull up the measuring probe and transmit drilling data in real time.

[0046] The data transmission process is as follows:

[0047] (1) The video transmission line transmits the borehole rock wall image data captured by the measuring probe to the control host for real-time monitoring and storage.

[0048] (2) The depth encoder transmits the real-time depth of the measuring probe in the borehole to the control host.

[0049] (3) The host extracts frames from the image data to form an image sequence, and matches the image sequence with the probe depth in time to generate a borehole image sequence with probe depth information.

[0050] Step S3: Due to the limited illumination area, the light becomes weaker the farther away from the light source, and the central area of ​​the borehole image sequence is a blind spot. For the borehole image sequence, the Hough circle detection method is used to identify the minimum radius R of the borehole center and the rock ring. min .

[0051] The Hough circle test method is as follows:

[0052] 1. The Canny edge detection algorithm is used to process the borehole images. This algorithm mainly consists of five parts: Gaussian filtering, pixel gradient calculation, non-maximum suppression, hysteresis thresholding, and isolated weak edge suppression.

[0053] Gaussian filtering: Gaussian filtering uses a two-dimensional Gaussian kernel with the same standard deviation and performs discrete convolution with the image. The calculation method is shown in formula (1).

[0054] (1)

[0055] in: For Gaussian kernel; This represents the pre-selected variance.

[0056] Pixel gradient calculation: The Sobel operator is used to perform pixel gradient transformation and generate the gradient intensity matrix of the image. The Sobel operator consists of two 3×3 matrices, respectively. and The former is used to calculate images. Pixel gradient matrix in direction The latter is used to calculate images. Pixel gradient matrix in direction The calculation method is shown in formulas (2)-(3). Gradient intensity matrix Then for and The gradient matrix takes the 2-norm, and the calculation method is shown in formula (4).

[0057] (2)

[0058] (3)

[0059] (4)

[0060] Non-maximum pixel gradient suppression: The non-maximum suppression method (NMS) is used to suppress the gradient intensity of the current pixel. The edge detection process compares the edge points with the adjacent intensities along the positive and negative gradient directions to suppress non-edge points and eliminate stray responses. Threshold hysteresis processing: A dual threshold is used to further classify edge points. Pixels below the lower threshold are re-suppressed; pixels above the higher threshold are considered strong edges; those in between are defined as weak edges and await further screening.

[0061] Weak edge isolation suppression: Weak edges are filtered out using neighborhood indexing, while strong edge points are retained if they are located in the neighborhood.

[0062] 2. Based on the gradient direction of the edge points, ellipse detection of the edge line segments is achieved using projection invariants (CNC); the optimal center (drill hole center) is solved through ellipse fitting; and the minimum rock ring radius is calculated based on the distance from the point to the center.

[0063] Step S4: To ensure the accuracy of the exploration, the borehole image sequence is effectively divided by a mask sequence to generate an effective image sequence, ensuring that only high-quality data areas are used to construct the core model and unfolded map.

[0064] The process of generating the mask sequence is as follows:

[0065] Based on the illumination capability of the lighting equipment and the accuracy of the camera, the maximum rock ring R is selected. max (By default, the outer boundary of the effective area is the 95% maximum inscribed circle centered at the borehole center); at the same time, to eliminate the influence of the central visual blind zone, the minimum rock ring R is selected. min This represents the inner boundary of the valid area. The mask is created using binary encoding, setting the valid area to white (selecting the desired portion of the image) and the remaining areas to black.

[0066] Step S5: Perform feature extraction and matching of the effective image sequence. Feature extraction: Extract DSP-SIFT features and SuperPoint features of the effective image sequence respectively. Image matching: Due to the temporal logic of the image sequence, an image is only related to images within its vicinity. Therefore, the matching method is sequential matching to reduce matching time.

[0067] The characteristics of DSP-SIFT are as follows:

[0068] The core of DSP-SIFT features is to weight the SIFT feature descriptors of a point at different scales; calculate the SIFT descriptor at each scale, average the descriptors at different scales and normalize the average descriptor to obtain the DSP-SIFT descriptor. The calculation method is shown in formula (5).

[0069] (5)

[0070] in, For point DSP-SIFT features, For weight parameters, for SIFT features of point p at scale The main direction corresponding to the feature point.

[0071] SuperPoint features are as follows:

[0072] SuperPoint features are extracted using a pre-learning model based on image features. This model is obtained through the operation of two networks: the MagicPoint network and the SuperPoint network. The MagicPoint network is a corner extraction network trained on virtual basic structures (polygons, polyhedra, etc.) to generate realistic features for unlabeled images. In the process of obtaining realistic features, each image is distorted at different angles. Features of the distorted images of each image are extracted and fused to obtain realistic features. The SuperPoint network, on the other hand, obtains a pre-learning model suitable for underground pipeline systems through joint training of feature locations and their descriptors.

[0073] Step S6: Project all matched DSP-SIFT features / SuperPoint features onto the image respectively, and generate feature association networks (DSP-SIFT feature network and SuperPoint feature network) for the image sequence through matching pairs. Where I represents the valid image, T represents the feature point trajectory, and L represents the edge of the trajectory. Simultaneously, geodesic consistency is used to remove mismatches from the feature association network.

[0074] The consistency check of geodesics is as follows:

[0075] Geodesic consistency means that if a feature can be extracted from an image, it should also be visible in its neighboring images. Performing geodesic consistency checks on features and using a scoring system to select the optimal solution can reduce computational ambiguity and improve the accuracy of core models. Scoring system The higher the value, the fewer mismatches the model will have. The evaluation metrics are as follows:

[0076] (6)

[0077] (7)

[0078] in, It is a quantitative indicator for the trajectory of feature points. There exists a matching feature point f in frame i and frame j.

[0079] Step S7: Construct a normalized model of the core model through modules such as initial setup, depth calculation, and bundled optimization.

[0080] The initial module setup is as follows:

[0081] Using the first pair of images, the initial frame of the normalized model can be reconstructed through epipolar geometry. Epipolar geometry describes the projection relationship between image pairs and can solve the estimation of 2D-2D trajectories; the calculation method is shown in formula (8). Due to the criticality of the initial pair, the image sequence is set to high-frequency acquisition to maintain the density of images. At the same time, the new image is registered into the current frame using the EPnP algorithm. The EPnP algorithm selects four control points to represent 3D points through principal component analysis (PCA); and based on the matching point pair relationship, it solves for the coordinates of the control points in the camera coordinate system and estimates the camera trajectory; the calculation method is shown in formulas (9), (10), and (11). The trajectory is described by rotation and translation matrices. Outliers in the trajectory estimation are removed using the RANSAC method and a Gaussian iterative solver.

[0082] (8)

[0083] in, For well-matched feature points; This is the intrinsic parameter matrix of the camera; Let be the translation vector of the camera; Let be the rotation matrix of the camera.

[0084] (9)

[0085] Formula (9) is the control point expression for a 3D point. 3D points; These are weight parameters; These are control points in world coordinates / camera coordinates.

[0086] Due to the camera coordinates of 3D points and world coordinates It is known that the camera trajectory can be solved using the Iterative Closest Point (ICP) algorithm. This algorithm is based on the camera coordinates. and world coordinates Find the centroid, and obtain the centroid-free coordinates of the two sets of coordinates. and The rotation matrix R is obtained by iterating according to formula (11), and the translation vector t is obtained by solving.

[0087] (10)

[0088] (11)

[0089] The depth computing module is as follows:

[0090] Based on the feature association network and camera trajectory, the IDWM algorithm (triangulation method) is used to recover 3D points. The calculation method is shown in formula (12). This method uses the reciprocal of the distance to control the influence of the camera trajectory on the point calculation, ensuring that the depth value of each point is positive. Its calculation error is smaller than other triangulation methods, making it more suitable for underground pipelines.

[0091] (12)

[0092] in, For image frames Scale factor; It is a translation vector; It is the rotation matrix that transforms image i from camera coordinates to image j from camera coordinates. The pixel positions for matching features.

[0093] The bundled optimization modules are as follows:

[0094] All 3D points were further refined using bundle adjustment (BA) to minimize nonlinear reprojection errors. The calculation method is described in formula (13). Then, normalized models (DSP-SIFT model and SuperPoint model) are established respectively.

[0095] (13)

[0096] in, For image frames Feature points; For image frames The corresponding matching pixels.

[0097] Step S8: Use a statistical denoising algorithm to remove noise from the model and restore the scale of the normalized model based on the depth values ​​of the effective image sequence.

[0098] The statistical denoising algorithm is as follows:

[0099] The algorithm calculates the average distance to each neighbor by using the distance distribution between each point and its neighbors. This average distance is assumed to follow a Gaussian distribution. If the average distance exceeds a certain threshold, the point can be defined as noise and removed.

[0100] The scale restoration method is as follows:

[0101] This method performs scale recovery using camera trajectory and corresponding probe depth information. The ratio of the depth difference between two borehole image sequences with a depth difference exceeding a predetermined distance to the Euclidean distance between the probe positions of the two corresponding frames is used as the scale recovery factor.

[0102] Step S9: Since FPFH features describe the spatial geometric relationship between feature points and their neighboring points, FPFH features are used for coarse matching to improve coupling speed; then the ICP algorithm is used to achieve precise coupling of the core 3D model.

[0103] The characteristics of FPFH are as follows:

[0104] The FPFH feature is represented by a weighted average of adjacent point pairs, calculated using formula (14). Specifically, the feature of each point pair is calculated using paired local coordinates, and significant intervals are found when simplifying the point feature histogram to obtain the FPFH feature. This feature combines the robustness of geometric distribution with the specificity of histogram statistics. It exhibits good detection repeatability and provides better initial values ​​for ICP.

[0105] (14)

[0106] in, The FPFH feature of point P; SPF features; Let k be the nearest neighbor of point p, and k be the number of nearest neighbors. For nearby points The weight.

[0107] The ICP algorithm is as follows:

[0108] The ICP method uses the least squares method to calculate the transformation matrix (including rotation and translation), and the calculation method is shown in formula (15). The error between the transformed model and the target model is then obtained. Through multiple iterations of the error threshold, a high-precision three-dimensional model of the core is constructed.

[0109] (15)

[0110] in, Let i be a point in the point cloud; Let be the corresponding point of point cloud s.

[0111] Step S10: Obtain the borehole axis by performing cylindrical fitting on the 3D model of the core using the Random Consistent Sampling (RANSAC) algorithm. , and the cylinder radius r, where Let (l,m,n) be a point on the borehole axis, and let (l,m,n) be the direction vector of the borehole axis. RANSAC can estimate the parameters of the mathematical model iteratively from a set of observation datasets containing "outside points". The mathematical model for the cylinder is given in formula (16).

[0112] (16)

[0113] The vertical distance from a 3D point on the core 3D model to the cylinder axis is calculated and subtracted from the cylinder radius r to obtain the axial depth information of the 3D point.

[0114] Step S11: Thin and filter the effective image sequence to generate a circular image sequence. The pixel mapping relationship between the Cartesian coordinate system of the rectangular unfolded image and the polar coordinate system of the circular image sequence is constructed through coordinate transformation; and the image is unfolded using a bilinear interpolation algorithm.

[0115] The unfolding process is as follows:

[0116] First, the theoretical value of the corresponding point P in the effective image sequence is obtained by back-mapping the pixel Q of the rectangular unfolded image. Then, the theoretical value is rounded up by the bilinear interpolation algorithm to ensure the rationality of the algorithm. The back-mapping relationship is shown in formula (17), and the bilinear interpolation algorithm is shown in formula (18).

[0117] (17)

[0118] in, The angle is in polar coordinates. The coordinates of the borehole center; Let P be the coordinates of any point P in the valid image sequence; Let Q be the coordinates of the point Q in the rectangular unfolded diagram.

[0119] (18)

[0120] in .

[0121] Step S12: Perform template matching to stitch the rectangular unfolded image. First, set the previous frame as the reference frame and the current frame as the target frame; then, select a template image from the reference frame and perform template matching with the current frame. Use the template matching algorithm to obtain the matching parameters of the two frames for stitching the rectangular unfolded image. Then, the newly generated image is template matched and stitched with the next frame, finally stitching together a complete panoramic hole wall unfolded image.

[0122] The specific details of the template matching method are as follows:

[0123] Template selection: To balance the diversity of template features and sizes, the central area of ​​the rectangular unfolded diagram is selected as the preferred template for matching. This ensures both sufficient margin and adequate template resolution.

[0124] Preset matching area: To reduce the template matching range and speed up the matching process, a pre-set search area is provided, and the module will only perform matching within the pre-defined search area.

[0125] Similarity evaluation: Considering robustness to grayscale, scale, and illumination, the Normalized Cross-Correlation (NCC) algorithm is used to determine template similarity. The core of this algorithm is to traverse the search region and calculate template similarity. With candidate images The normalized cross-correlation function is given. The maximum value of the function corresponds to the solution that is most similar to the template among all candidate images. The calculation method is given in formula (19).

[0126] (19)

[0127] in, For the cross-correlation value, please refer to formula (20); For the autocorrelation value of the template, see formula (21); The autocorrelation value of the candidate image is given in formula (22).

[0128] (20)

[0129] (twenty one)

[0130] (twenty two)

[0131] Rectangular unfolded diagram splicing: using the center coordinates of the template and optimal candidate image Calculate the matching parameters (horizontal offset) based on the center coordinates. and vertical offset ( ) and splicing, see formula (23).

[0132] (twenty three)

[0133] The relationship between step S13, the three-dimensional model of the rock core, and the panoramic borehole wall unfolding diagram.

[0134] Since each 3D point in the core 3D model is associated with a pixel in the effective image sequence, and each point in the panoramic borehole wall unfolded image is also associated with a pixel in the effective image sequence, a mutual assignment method is used for association: based on the axial depth information of the core 3D model, each 3D point of the core 3D model is mapped onto the panoramic borehole wall unfolded image to form a point cloud model with elevation information. Then, based on the RGB information of the corresponding position in the panoramic borehole wall unfolded image, the 3D points are assigned colors to generate a realistic borehole wall unfolded image. Finally, the RGB information of the 3D points is assigned to the core 3D model to generate a realistic 3D model of the core.

[0135] The virtual core imaging method proposed in this invention can be summarized as follows: image preprocessing algorithm (S3-S4), borehole wall three-dimensional reconstruction algorithm (S5-S9), borehole wall image stitching algorithm (S11-S12), and real scene model generation algorithm (S10, S13).

[0136] 1) Image preprocessing algorithm (steps S3-S4) is used to remove invalid image information from the borehole image sequence and improve the quality of the initial data. This invention preferably uses the OpenCV library to implement Canny edge detection and ellipse fitting. By comparing the average values ​​of the major and minor axes of the ellipse, the center of the ellipse with the minimum average value is taken as the borehole center, and the major axis of the ellipse is taken as the minimum rock ring radius R. min Draw the minimum torus. Based on the borehole center, draw the maximum inscribed circle of the borehole image, and take 95% of the radius of the maximum inscribed circle as the maximum rock ring R. max The image information inside the double ring is valid information, and it is marked by a mask to generate a valid image sequence.

[0137] 2) The three-dimensional reconstruction algorithm of the borehole rock wall (steps S5-S9) mainly includes: feature extraction and matching (step S5) → removal of mismatches (step S6) → normalization model construction (step S7) → scale restoration (step S8) → dual model coupling (step S9).

[0138] To ensure the quality of the core 3D model, this invention specifically extracts manual features DSP-SIFT and deep learning-based features SuperPoint, and constructs normalized models based on the two types of features, which helps to enrich the number of feature points.

[0139] As a specific embodiment of the present invention, constructing normalized three-dimensional core models for the two types of feature points can be divided into four steps, as shown in the flowchart below. Figure 3 :

[0140] The first step is to extract the first two frames of the effective image sequence, and use epipolar geometry to estimate the pose transformation R,t (normalized) of the two frames, treating the camera coordinate system of the first frame as the world coordinate system.

[0141] The second step is to extract the next frame image from the valid image sequence, use the EPnP algorithm to estimate the camera trajectory, and use the IDWM algorithm to recover the 3D points of the feature points.

[0142] The third step is to determine whether the camera trajectory increment Δd, i.e., the Euclidean distance between the camera position in the last globally optimized image frame and the camera position in the current image frame, is greater than a threshold. If greater than Δd > Then, the bundle adjustment method is used to locally optimize the output of the frame and its neighborhood.

[0143] The fourth step is to determine whether there are any unreconstructed image frames in the valid image sequence. If so, return to the second step; if reconstruction has been completed, perform global optimization on the normalized model.

[0144] Furthermore, statistical denoising is performed on the two normalized models. The scale restoration factor is used by taking the ratio of the depth difference between two borehole image sequences whose depth difference exceeds a predetermined distance threshold to the Euclidean distance between the probe positions in the corresponding two frames. Preferably, the predetermined distance threshold is 500 mm.

[0145] 3) The borehole wall image stitching algorithm mainly consists of two parts: constructing a rectangular unfolded image by combining coordinate mapping with a bilinear interpolation algorithm (step S11), and stitching the panoramic borehole wall unfolded image based on NCC template matching (step S12). The flowchart is shown below. Figure 3 (a) in the middle.

[0146] In practice, the measuring probe may jitter, rotate, and stall within the borehole. Jitter and rotation within the borehole cause the borehole center to deviate from the geometric center of the image, resulting in severe geometric distortion during the unfolding of the effective image sequence into a rectangular image. Probe stalling generates a large number of invalid duplicate images, thus affecting the matching and stitching efficiency of the rectangular image sequence. To ensure the quality and efficiency of the panoramic borehole wall unfolding image stitching, it is necessary to screen the effective image sequence:

[0147] Step 1: Thin out the effective image sequence, retaining images whose depth difference with the previous frame is greater than a threshold. (Preferably 1cm) effective image.

[0148] Step 2: Calculate the horizontal and vertical offsets between the geometric center of the image and the borehole center. Valid images with an offset greater than 10% of the image size are removed, and a circular image sequence is generated.

[0149] Furthermore, take the circular image sequence Minimum value ,and Maximum value As the effective area due to the unfolding, the size of the rectangular unfolded graph is length 2. ,Width .

[0150] Furthermore, a rectangular unfolded image is generated from the annular image sequence by using coordinate mapping combined with a bilinear interpolation algorithm.

[0151] Furthermore, the first frame of the circular image sequence is used as the reference frame, and the second frame as the target frame. A matching template is extracted from the center of the reference frame. To ensure that the template has sufficient features, it is preferable that the template width is 40% of the rectangular unfolded image, and the length is twice the width. Considering that the offset between adjacent frames of the rectangular unfolded image is relatively small, the search area for the target frame is set to 200% of the area centered on the template.

[0152] Furthermore, the region with the highest similarity within the matching area is obtained based on the NCC algorithm, and the horizontal offset is calculated. and vertical offset Image stitching is achieved through image cropping. This process is repeated until a panoramic view of the hole wall is constructed. (See...) Figure 3 (b) in the middle.

[0153] As a further optimization, only when the offset between the current frame image and the previous frame involved in the stitching is... Only when the cumulative width of the rectangular unfolded image is greater than 20% of the width of the rectangular unfolded image will the rectangular unfolded image of the current frame participate in the stitching of the panoramic hole wall unfolded image.

[0154] 4) Real-scene model generation algorithm (step S13): mainly includes two aspects: obtaining axial depth information of 3D points based on cylinder fitting and constructing a real-scene model of rock core.

[0155] The preferred cylindrical fitting algorithm uses the publicly available MATLAB function `pcfitcylinder`. The camera position in the 3D core model is fitted as a straight line using the least squares method, with its direction vector serving as the reference direction constraint for `pcfitcylinder`. The borehole diameter `r` and borehole axis are obtained through cylindrical fitting, and the axial depth information of 3D points in the 3D core model is calculated.

[0156] Furthermore, 3D points in the core 3D model are mapped to a panoramic borehole wall unfolded image using their corresponding pixel coordinates in the effective image sequence. A real-world borehole wall unfolded image is then constructed using axial depth information and the RGB information of the panoramic borehole wall unfolded image. (See...) Figure 3 (c) in the middle.

[0157] Furthermore, the RGB information of the panoramic borehole wall unfolding diagram is applied to the 3D points in the core 3D model to form a real-world 3D model of the core.

[0158] In summary, the three-dimensional core model and the unfolded borehole wall diagram provided by this invention can more realistically reflect the actual rock strata conditions of the borehole. They complement each other, describing the geological strata from two perspectives: the three-dimensional model and the unfolded diagram. This enables full-dimensional and high-precision borehole information analysis, making it easier to control the safety level of the project more simply and accurately.

[0159] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. All components not explicitly stated in this embodiment can be implemented using existing technology.

Claims

1. A virtual core imaging device based on forward looking panoramic borehole video, characterized by, include: The components include: orifice support (1), depth encoder (2), electric winch (3), video transmission line (4), data transmission line (5), control host (6), measuring probe (7), and computer system (8); among which, The orifice bracket (1) is placed above the orifice of the drilled hole; the electric winch (3) includes a stepper motor and a cable disc, the stepper motor is used to drive the cable disc to rotate, providing a smooth lowering or lifting speed; wherein the cable disc is connected to the orifice bracket (1) through a fixed structure; The depth encoder (2) is mounted on the orifice bracket (1) and is used to record the depth information of the real-time measuring probe (7) and the extension and retraction length of the video transmission line (4). The video transmission line (4) is used to transmit the image data recorded by the measuring probe. One end of the line is connected to the measuring probe (7), and the other end is wound around the cable disc after passing through the depth encoder (2). The data transmission line (5) is used to transmit image data with depth values. One end of the line is connected to the depth encoder (2), and the other end is connected to the control host (6) and the computer system (8) in sequence via the electric winch (3). The measuring probe (7) is used to record borehole rock wall image data. It is controlled by the electric winch (3) to be lowered at a constant speed in the borehole to the bottom of the hole and then pulled up to the opening of the hole through the video transmission line (4). The control host (6) extracts frames from the real-time received image data to form an image sequence, and performs time matching between the image sequence and the probe depth information to generate a borehole image sequence with probe depth information. The computer system (8) is used to perform three-dimensional reconstruction of the borehole rock wall and stitch together the borehole rock wall images based on the borehole image sequence with probe depth information to construct a virtual rock core; including: obtaining the borehole center and minimum rock ring radius, constructing an effective image sequence; extracting DSP-SIFT features and SuperPoint features of the effective image sequence respectively, and performing feature matching; generating a feature association network, eliminating mismatches, and constructing a borehole normalization model based on DSP-SIFT features and SuperPoint features respectively; removing model noise, and realizing scale recovery of the normalization model based on depth information of the depth encoder; constructing a three-dimensional model of a high-precision three-dimensional rock core, performing cylindrical fitting on the three-dimensional model of the rock core, and extracting axial depth information; thinning and filtering the effective image sequence, generating a rectangular unfolded map through coordinate transformation, and constructing a panoramic borehole wall unfolded map; associating the three-dimensional model of the rock core and the panoramic borehole wall unfolded map to generate a real-world three-dimensional model of the rock core.

2. An imaging method based on the virtual core imaging device of claim 1, characterized by, The specific steps are as follows: Step S1: Align the measuring probe with the borehole axis, start the equipment with the control host, zero the depth encoder, and check if the control host has real-time feedback of the drilling image. Step S2: After successful debugging, start the stepper motor to make the measuring probe descend to the bottom of the hole at a constant speed; after stabilizing for a period of time, start the stepper motor to pull up the measuring probe and transmit the borehole rock wall image data in real time. The depth encoder transmits the real-time depth of the measuring probe in the borehole to the control host; the control host extracts frames from the image data to form an image sequence, and matches the image sequence with the probe depth in time to generate a borehole image sequence with probe depth information. Step S3, for the borehole image sequence, the Hough circle detection method is used to identify the borehole center and the minimum radius R of the rock ring min ; Step S4: Divide the borehole image sequence using a mask sequence to generate a valid image sequence; Step S5: Extract the DSP-SIFT features and SuperPoint features of the effective image sequence respectively, and perform sequential matching; Step S6, respectively project all matched DSP-SIFT features and SuperPoint features onto the image, and generate a feature association network of the image sequence through matching pairs wherein I is an effective image, T is a feature point track, L is an edge of the track, and geodesic consistency is used for false matching elimination of the feature association network; Step S7: Construct a normalized model of the core model through initial setup, depth calculation, and bundled optimization modules; Step S8: Use a statistical denoising algorithm to remove noise from the model and restore the scale of the normalized model based on the depth values ​​of the effective image sequence. Step S9: Use FPFH features for coarse matching, and then use the ICP algorithm to achieve precise coupling of the core 3D model; Step S10: Perform cylindrical fitting on the core 3D model using a random consistency sampling algorithm to obtain the borehole axis and cylinder radius r. Calculate the vertical distance from the 3D point on the core 3D model to the cylinder axis and subtract it from the cylinder radius r to obtain the axial depth information of the 3D point. Step S11: Thin and filter the effective image sequence to generate a circular image sequence. Construct the pixel mapping relationship between the Cartesian coordinate system of the rectangular unfolded image and the polar coordinate system of the circular image sequence through the coordinate transformation relationship between the two. And combine the bilinear interpolation algorithm to unfold the image. Step S12: Perform template matching to stitch the rectangular unfolded image: First, set the previous frame image as the reference frame and the current frame image as the target frame, and select a template image in the reference frame image to perform template matching with the current frame; use the template matching algorithm to obtain the matching parameters of the two frames to perform template matching of the rectangular unfolded image; then, perform template matching and stitching with the next frame to generate the new image; finally, stitch them together to form a complete panoramic hole wall unfolded image. Step S13: Associate the core 3D model and the panoramic borehole wall unfolded image by mutual assignment: Based on the axial depth information of the core 3D model, map each 3D point of the core 3D model onto the panoramic borehole wall unfolded image to form a point cloud model with elevation information. Then, assign colors to the 3D points according to the RGB information of the corresponding positions in the panoramic borehole wall unfolded image to generate a real-scene borehole wall unfolded image. Finally, assign the RGB information of the 3D points to the core 3D model to generate a real-scene core 3D model.

3. The imaging method of claim 2, wherein, The Hough circle detection method used in step S3 specifically employs the Canny edge detection algorithm to process the borehole image, including: (1) Gaussian filtering: A two-dimensional Gaussian kernel with the same standard deviation is used, and it is discretely convolved with the image: （1） wherein: is a Gaussian kernel; is a preselected variance; (2) Pixel gradient operation: The Sobel operator is used to perform pixel gradient transformation and generate the gradient intensity matrix of the image; the Sobel operator consists of two 3×3 matrices, respectively and ,in Used to calculate images Pixel gradient matrix in direction , Used to calculate images Pixel gradient matrix in direction Specifically: （2） （3） （4） wherein respectively represent two different pixel points; (3) Non-maximum pixel gradient suppression: The non-maximum suppression method (NMS) is used to suppress the gradient intensity of the current pixel. By comparing the intensity with adjacent intensities in the positive and negative gradient directions, non-edge points are suppressed, and stray responses in edge detection are eliminated. (4) Threshold hysteresis processing: Double thresholds are used to further classify edge points. Pixels below the low threshold are re-suppressed; pixels above the high threshold are considered strong edge points; those in between are defined as weak edges and await further screening. (5) Suppression of isolated weak edges: Weak edges are screened out by neighborhood indexing, and strong edge points are retained if they are in the neighborhood; ellipse detection of edge segments is achieved according to the gradient direction of the edge points and the projection invariant; the optimal circle center is solved by ellipse fitting, and the minimum rock ring radius is calculated according to the distance from the point to the circle center.

4. The imaging method of claim 2, wherein, In step S5, the DSP-SIFT feature calculation method is as follows: （5） wherein, is a point of a DSP-SIFT feature, is a weight parameter, is a SIFT feature of a point p at scale s, a dominant orientation of the feature point. SuperPoint features are extracted by a pre-learning model based on image features. This model is obtained through the MagicPoint network and the SuperPoint network. The MagicPoint network is a corner extraction network trained with a virtual basic structure to generate real features of unlabeled images. It extracts features from the distorted images of each image and fuses them to obtain real features. The SuperPoint network is a pre-learning model suitable for underground pipeline systems obtained by jointly training feature locations and their descriptors.

5. The imaging method of claim 2, wherein, In step S6, geodesic consistency is used to remove false matches from the feature association network, and an evaluation index is used to evaluate the new association network after removing false matches Evaluation is performed, and the evaluation index is as follows: （6） （7） in, As a quantitative indicator of feature point trajectories, For a matching feature point p in frame i and frame j, the scoring system... The larger the value, the more it is considered... The more edges are removed, the cleaner the removal of mismatches.

6. The imaging method of claim 2, wherein, In step S7, a normalized model of the core model is constructed, wherein... (1) The initial module setup is as follows: Using the first pair of images, an initial framework for a normalized model based on polar geometry reconstruction is established: （8） wherein, , is a matched feature point; is an intrinsic matrix of the camera; is a translation vector of the camera; is a rotation matrix of the camera; The new image is registered into the current frame using the EPnP algorithm. The EPnP algorithm selects four control points to represent 3D points through principal component analysis, and solves for the coordinates of the control points in the camera coordinate system and estimates the camera trajectory based on the matching point pairs. The calculation method is as follows: （9） （10） （11） The trajectory is described by rotation and translation matrices. Outliers in the trajectory estimation are removed using the RANSAC method and a Gaussian iterative solver. Equation (9) is the expression for the control points of the 3D points. 3D points; These are weight parameters; The control points are defined in world coordinates / camera coordinates; the camera trajectory is solved using an iterative nearest-point algorithm, based on the camera coordinates. and world coordinates Find the centroid, and obtain the centroid-free coordinates of the two sets of coordinates. and The rotation matrix R is obtained by iterating according to formula (11), and the translation vector t is solved. (2) The depth calculation module is as follows: Based on feature association networks and camera trajectories, 3D points are recovered using triangulation methods. The calculation method is as follows: （12） wherein are the scale factors of the image frames are the scale factors of the image frames are the pixel positions of the matched features, is the rotation matrix from image i camera coordinates to image j camera coordinates; (3) The bundled optimization modules are as follows: All 3D points are further refined by bundle adjustment to minimize the non-linear re-projection error The calculation method is as follows （13） in, For image frames Feature points; For image frames The corresponding matching pixels; Then, normalized models, namely the DSP-SIFT model and the SuperPoint model, are established respectively.

7. The imaging method of claim 2, wherein, In step S9, FPFH features are represented by a weighted average of adjacent point pairs, calculated as follows: （14） in, For the FPFH features of point P, SPF characteristics Let k be the nearest neighbor of point p, and k be the number of nearest neighbors. For nearby points The weights are calculated by pairwise local coordinates to obtain the features of each point pair, and significant intervals are found when simplifying the point feature histogram to obtain FPFH features. The ICP method uses the least squares method to calculate the transformation matrix, including rotation and translation. The calculation method is as follows: （15） wherein, is a point of the point cloud i; is a corresponding point of the point cloud s; This yields the error between the transformed model and the target model. Through multiple iterations using the error threshold, a high-precision three-dimensional model of the rock core is constructed.

8. The imaging method of claim 2, wherein, In step S10, the mathematical model of the cylinder is as follows: （16） Drilling axis: wherein, is a point on the drilling axis and (l,m,n) is a direction vector of the drilling axis.

9. The imaging method of claim 2, wherein, In step S11, the image is unfolded using a bilinear interpolation algorithm. The specific process is as follows: First, the theoretical value of the corresponding point P in the effective image sequence is obtained by back-mapping the pixel Q of the rectangular unfolded image. Then, the theoretical value is rounded up using a bilinear interpolation algorithm. The calculation method for the back-mapping relationship is as follows: （17） in, The angle is in polar coordinates. The coordinates of the borehole center; Let P be the coordinates of any point P in the valid image sequence; R represents the coordinates of point Q in the rectangular unfolded graph; min R max These are the minimum and maximum radii of the borehole center and the rock ring, respectively; known effective image coordinates adjacent RGB values , , , Then the bilinear difference algorithm is as follows: （18） wherein .

10. The imaging method of claim 9, wherein, In step S12, the template matching method is as follows: (1) Template selection: Select the central area of ​​the rectangular unfolded diagram as the matching template; (2) Preset matching area: Preset the search area and match only in the predefined search area; (3) Matching similarity evaluation: the normalized cross-correlation measure algorithm is used to determine the similarity of the template, specifically: traversing the search area, calculating the normalized cross-correlation function of the template and the candidate image , and the maximum value of the function corresponds to the solution most similar to the template among all candidate images, and the calculation method is as follows: （19） wherein, is a cross-correlation value, is an autocorrelation value of the template, is an autocorrelation value of the candidate image, calculated as follows: （20） （21） （22） (4) Rectangular development map splicing: using the center coordinates of the template and the center coordinates of the optimal candidate image to calculate the matching parameters and splicing, and the calculation method is: （23） wherein is a horizontal offset, is a vertical offset.

Images