A method, system and device for image stitching for borehole television

Abstract

The present application relates to the technical field of image stitching, and particularly relates to a drilling television-oriented image stitching method, system and device, wherein suspected feature points in each drilling image in drilling video frame images to be stitched are acquired; the possibility of each suspected feature point in each drilling image appearing in a previous drilling image is analyzed, and each suspected feature point in each drilling image is screened according to the possibility to determine stable feature points; the drilling video frame images to be stitched are stitched based on the stable feature points in each drilling image to obtain a stitched image. The present application effectively improves the accuracy of image stitching by screening stable feature points in each drilling image in drilling video frame images to be stitched.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image stitching, in particular to an image stitching method, system and device for borehole television. BACKGROUND

[0002] Borehole television is a tool for checking the internal conditions of a borehole, which provides visual information about the underground geological conditions by placing a camera into the borehole and shooting through the camera, making underground exploration and monitoring more efficient and accurate, and significantly improving the safety and effectiveness in mineral exploration and engineering construction. Since the borehole is usually long, a single image cannot cover the entire view of the borehole, and image stitching technology is often needed to provide more comprehensive visual data for borehole exploration and geological survey.

[0003] When performing borehole image stitching, it is necessary to first determine the feature points in the image, and then match the feature points to achieve image stitching. SIFT algorithm (Scale-Invariant Feature Transform) is a commonly used algorithm for determining feature points in a single borehole image. However, due to the complex texture of the borehole wall, there are a large number of pits and cracks, etc., and in the process of feature extraction using the SIFT algorithm, there may be some unstable feature points in the image, such as those formed by shadows. These feature points may disappear during the descent and rotation of the camera and cannot provide useful information for image matching, and may even interfere with the accuracy of image matching, ultimately resulting in poor image stitching accuracy. SUMMARY

[0004] The present application aims to provide an image stitching method, system and device for borehole television, which solves the problem of poor image stitching accuracy caused by unstable feature points.

[0005] To solve the above technical problems, in a first aspect, the present application provides an image stitching method for borehole television, comprising the following steps:

[0006] Obtaining suspected feature points in each borehole image of the borehole video frame image to be stitched;

[0007] Analyzing the possibility of each suspected feature point in each borehole image appearing in the previous borehole image, and screening each suspected feature point in each borehole image according to the possibility to determine each stable feature point;

[0008] Based on each stable feature point in each borehole image, the borehole video frame image to be stitched is stitched to obtain a stitched image.

[0009] In some possible implementation manners of the first aspect, the analysis on the possibility of each of the suspected feature points in each of the drilling images appearing in the previous drilling image includes:

[0010] determining a region where each of the suspected feature points in each of the drilling images is located, and a magnification of the region in the previous drilling image;

[0011] screening all suspected feature points in the previous drilling image based on the magnification and the location of the region where the suspected feature point is located in the image, to determine each of the suspected feature points in each of the drilling images as each of the to-be-analyzed feature points in the previous drilling image;

[0012] determining, based on the magnification, a coincidence degree between the region where each of the suspected feature points in each of the drilling images is located and the region where each of the to-be-analyzed feature points is located;

[0013] determining, according to the coincidence degree between each of the suspected feature points in each of the drilling images and all the to-be-analyzed feature points, the possibility of each of the suspected feature points in each of the drilling images appearing in the previous drilling image.

[0014] In some possible implementation manners of the first aspect, the determination of each of the suspected feature points in each of the drilling images as each of the to-be-analyzed feature points in the previous drilling image includes:

[0015] determining a distance from the region where each of the suspected feature points in each of the drilling images is located to a camera center position in the corresponding drilling image;

[0016] determining, according to the distance and the magnification, a reference distance of the region in the previous drilling image relative to the camera center position;

[0017] screening all suspected feature points in the previous drilling image according to the reference distance, to determine each of the to-be-analyzed feature points.

[0018] In some possible implementation manners of the first aspect, the determination of the coincidence degree between the region where each of the suspected feature points in each of the drilling images is located and the region where each of the to-be-analyzed feature points is located includes:

[0019] performing a reduction processing on the region where each of the suspected feature points in each of the drilling images is located according to the magnification, to obtain a reduction processing region;

[0020] determining a number of coincident pixel points between the reduction processing region of each of the suspected feature points in each of the drilling images and the region where each of the to-be-analyzed feature points is located;

[0021] According to a difference between the number of pixel points of the reduced processing region of each of the suspected feature points in each of the drilling images and the corresponding number of overlapping pixel points, a degree of overlap between the region where each of the suspected feature points is located in each of the drilling images and the region where each of the to-be-analyzed feature points is located is determined; the greater the difference, the smaller the corresponding degree of overlap.

[0022] In combination with the first aspect, in some possible implementation manners, determining the degree of overlap between the region where each of the suspected feature points is located in each of the drilling images and the region where each of the to-be-analyzed feature points is located further includes:

[0023] determining a first connecting line from a center point of the reduced processing region of each of the suspected feature points in each of the drilling images to a camera center position in the corresponding drilling image;

[0024] determining a second connecting line from a center point of the region where each of the to-be-analyzed feature points is located in the previous drilling image to the camera center position in the previous drilling image;

[0025] According to positions of the first connecting line and the second connecting line, a rotation included angle is determined, the reduced processing region of each of the suspected feature points in each of the drilling images is subjected to rotation processing according to the rotation included angle, and the number of overlapping pixel points between the reduced processing region of each of the suspected feature points in each of the drilling images and the region where each of the to-be-analyzed feature points is located is determined based on the reduced processing region after the rotation processing.

[0026] In combination with the first aspect, in some possible implementation manners, determining the degree of overlap between the region where each of the suspected feature points is located in each of the drilling images and the region where each of the to-be-analyzed feature points is located includes:

[0027] determining a difference absolute value between the number of pixel points of the reduced processing region of each of the suspected feature points in each of the drilling images after the rotation processing and the corresponding number of overlapping pixel points;

[0028] performing negative correlation normalization processing on the difference absolute value, so as to obtain the degree of overlap between the region where each of the suspected feature points is located in each of the drilling images and the region where each of the to-be-analyzed feature points is located.

[0029] In combination with the first aspect, in some possible implementation manners, determining the possibility of each of the suspected feature points in each of the drilling images appearing in the previous drilling image includes:

[0030] determine the maximum value in the coincidence degree between each of the suspected feature points in each of the borehole images and all of the feature points to be analyzed, and determine the maximum value as the possibility of each of the suspected feature points in each of the borehole images appearing in the previous borehole image.

[0031] In combination with the first aspect, in some possible implementation manners, determining the stable feature points comprises:

[0032] determining whether the possibility of each of the suspected feature points in each of the borehole images appearing in the previous borehole image is greater than or equal to the possibility threshold value, and determining the suspected feature points with the possibility greater than or equal to the possibility threshold value as the stable feature points.

[0033] To solve the above technical problem, the second aspect of the present application further provides an image stitching system for borehole television, comprising:

[0034] a suspected feature point determination module, configured to: acquire suspected feature points in each of the borehole images in the borehole video frame images to be stitched;

[0035] a stable feature point determination module, configured to: analyze the possibility of each of the suspected feature points in each of the borehole images appearing in the previous borehole image, and screen each of the suspected feature points in each of the borehole images according to the possibility, to determine the stable feature points;

[0036] an image stitching module, configured to: stitch the borehole video frame images to be stitched based on the stable feature points in each of the borehole images, to obtain a stitched image.

[0037] To solve the above technical problem, the third aspect of the present application further provides an image stitching device for borehole television, comprising a memory and a processor; the memory is configured to store executable program codes; the processor is configured to call and run the executable program codes from the memory, to implement the image stitching method for borehole television in the first aspect or one of the possible implementation manners of the first aspect.

[0038] The present application has the following beneficial effects: by acquiring suspected feature points in each of the borehole images in the borehole video frame images to be stitched, analyzing the possibility of each of the suspected feature points in each of the borehole images appearing in the previous borehole image, and screening the suspected feature points in each of the borehole images in the borehole video frame images to be stitched according to the possibility, unstable feature points that mistake shadow areas as feature points and do not appear in previous borehole images are removed, so that real stable feature points are obtained, and the borehole video frame images are stitched based on the real stable feature points, to obtain a stitched image, and the accuracy of image stitching is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, and the advantages thereof, simple introduction to the drawings needed to be used in the embodiments or the prior art description will be given below. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative effort based on these drawings.

[0040] Figure 1 A basic flowchart of an image stitching method for borehole television of an embodiment of the present application;

[0041] Figure 2 A gray scale image corresponding to a frame of borehole image of an embodiment of the present application;

[0042] Figure 3 A schematic diagram of the distance between the center point of the region where a suspected feature point in a frame of borehole image of an embodiment of the present application is located and the position of the center point of the camera region in the current image;

[0043] Figure 4 A schematic diagram of the reference distance of the region where a suspected feature point in the image stitching result of an embodiment of the present application is located in the previous frame of borehole image relative to the center position of the camera; Figure 3

[0044] Figure 5 A schematic diagram of the rotation angle of an embodiment of the present application;

[0045] Figure 6 A schematic diagram of the stitched image of an embodiment of the present application;

[0046] Figure 7 A structural schematic diagram of an image stitching device for borehole television of an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to clearly illustrate the technical features of the present application, the present application will be described in detail below by specific implementation manners and in combination with the drawings.

[0048] Embodiments of the present application will be described in more detail below with reference to the drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms, and should not be interpreted as being limited to the embodiments described herein, on the contrary, these embodiments are provided to make the present application more thorough and complete. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes, and are not intended to limit the protection scope of the present application.

[0049] ​It should be understood that each of the steps recited in the method embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present application is not limited in this regard.

[0050] The term "comprises" and variations thereof such as "comprising" and "comprises" as used herein are open-ended, that is, "comprising but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Related definitions will be given in the description below.

[0051] It should be noted that the "first", "second", and the like concepts mentioned in the present application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0052] Although the operations or steps in the embodiments of the present application are described in a specific order in the accompanying drawings, they should not be understood as requiring the specific order or serial order shown, or requiring all of the operations or steps to be performed to obtain the desired results. In the embodiments of the present application, these operations or steps can be performed in series; they can also be performed in parallel; and a part of them can be performed.

[0053] At the same time, it can be understood that the data involved in the technical solutions of the present application (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and relevant provisions. Unless otherwise defined, all technical and scientific terms used in the present application have the same meaning as understood by those skilled in the art to which the present application belongs, and all parameters or indicators in the formulas involved in the present application are normalized values that eliminate the dimension influence.

[0054] First, the application scenario of the present application is described. When performing borehole image stitching, due to the complex texture of the inner wall of the borehole, there are a large number of pits, cracks and the like, and many feature obvious shadow regions will appear. In the process of matching the borehole image using the SIFT algorithm, the pixels in these shadow regions and the like may be misidentified as feature points. The stability of these feature points in the shadow part is poor, and with the movement and rotation of the camera, these suspected feature points may disappear, thereby affecting the accuracy of the borehole image matching, and ultimately affecting the accuracy of the image stitching.

[0055] In order to solve the problem of poor image splicing accuracy caused by the influence of unstable feature points, the embodiment of the present application provides a drilling television-oriented image splicing method, system and device, by analyzing the possibility of each suspected feature point in the current drilling image appearing in the previous drilling image, and then eliminating the feature points that do not appear in the previous drilling image, the stability of the determined feature points is improved, and finally the accuracy of the drilling image matching and splicing result is improved.

[0056] The drilling television-oriented image splicing method, system and device provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0057] Figure 1 The basic flow diagram of the drilling television-oriented image splicing method provided by the embodiment of the present application is shown as Figure 1 The method specifically comprises the following steps:

[0058] Step S100: acquiring suspected feature points in each drilling image in the drilling video frame image to be spliced.

[0059] Since the embodiment of the present application needs to splice multiple drilling images, the drilling internal video captured by the drilling television camera during the gradual descent process is acquired, wherein the camera focal length is set to 6mm, and the camera descent speed is 5m / min. According to the moving speed of the camera in the drilling, in order to reduce the repetition of the drilling image, the video image is intercepted once every 3 frames, thereby obtaining all the drilling images to be spliced, that is, the drilling video frame image to be spliced.

[0060] In order to facilitate subsequent image splicing, for each drilling image in the above-mentioned acquired video frame image to be spliced, first, gray-scale processing is performed to obtain the corresponding gray-scale image, and a feature point extraction algorithm is used to extract feature points from the gray-scale image, and the extracted feature points are regarded as suspected features, so as to acquire all suspected feature points in the drilling image. In some possible implementation manners, the SIFT algorithm can be used to extract feature points from the gray-scale image. Since the specific process of using the SIFT algorithm to extract feature points from the image belongs to the prior art, it will not be described here. In addition, through the gray-scale image, the gray-scale value of each pixel point in the corresponding drilling image can also be determined. Figure 2 A gray-scale image corresponding to a certain drilling image is shown.

[0061] Step S200: analyzing the possibility of each suspected feature point in each drilling image appearing in the previous drilling image, and screening each suspected feature point in each drilling image according to the possibility to determine each stable feature point.

[0062] After extracting the suspected feature points in each borehole image in the video frame image to be spliced, because the borehole interior texture is relatively complex and is affected by light, some shadows with obvious features may be generated, and these shadow points may also be misidentified as feature points. However, these shadow feature points have poor stability, and therefore further screening of the suspected feature points is required. Since the borehole internal structure only undergoes scaling and position translation during lens movement, and does not undergo deformation, the possibility of each suspected feature point in the current borehole image appearing in the previous borehole image can be determined by analyzing the matching degree of the structure at the position of each suspected feature point in the current borehole image and the position of the suspected feature point in the previous borehole image, to reflect the stability of the suspected feature point, and to further obtain stable feature points.

[0063] In some possible implementations, the analysis of the possibility of each suspected feature point in each borehole image appearing in the previous borehole image includes:

[0064] Step S201: determining the region where each suspected feature point in each borehole image is located, and the magnification of the region in the previous borehole image.

[0065] Since there are a large number of concave and convex structures on the borehole inner wall, the gray value at the same position may change greatly under different shooting angles and is affected by light, but the gray value in the same convex structure or pit structure in the same borehole image is relatively consistent. Therefore, for each suspected feature point in each borehole image, a connected domain is generated according to the condition that the gray difference is less than a gray threshold value, and the connected domain where the suspected feature point is located is determined as the region where the suspected feature point is located. The gray threshold value can be reasonably set as needed, and in some possible implementations, the value of the gray threshold value can be set to 2.

[0066] As the borehole television camera descends, the far part (close to the lens position) in the borehole image is enlarged and moves to the edge of the picture, and therefore, according to the speed of the camera descent, the time interval between adjacent frames, and the camera focal length, the magnification of the region where each suspected feature point in the current borehole image is located relative to the previous borehole image can be calculated.

[0067] For the above step S201, in some possible implementations, the magnification of the region where each suspected feature point in each borehole image is located in the previous borehole image is determined, and the implementation includes:

[0068] (1) determining the relationship between the size of the object in each borehole image and the distance of the camera.

[0069] The relationship between the size of the object in each borehole image and the distance of the camera can be represented by the following formula:

[0070] ;

[0071] wherein, represents the size of the object in the borehole image; represents the focal length of the camera; represents the distance between the object and the camera; represents the size of the borehole image. represents that the proportion of the object in the borehole image is proportional to the focal length, that is, the longer the focal length, the greater the magnification of the object, and is inversely proportional to the distance between the actual object and the camera, that is, the closer the distance, the larger the object in the borehole image.

[0072] (2) Determine the relationship between the relative distance of each borehole inner wall position in each borehole image and the camera and the relative distance in the previous borehole image.

[0073] The relationship between the relative distance of each borehole inner wall position in each borehole image and the camera and the relative distance in the previous borehole image can be represented as:

[0074] ;

[0075] wherein, represents the relative distance of the borehole inner wall position in the current borehole image and the camera; represents the relative distance of the borehole inner wall position in the previous borehole image of the current borehole image and the camera; represents the speed of the camera descending; represents the time interval between the two borehole images.

[0076] (3) Determine the magnification of the object in the current borehole image relative to the previous borehole image.

[0077] The magnification of the object in the current borehole image relative to the previous borehole image can be represented by the following formula:

[0078] ;

[0079] wherein, represents the magnification of the object in the current borehole image relative to the previous borehole image; represents the size of the object in the current borehole image; represents the size of the object in the previous borehole image of the current borehole image.

[0080] (4) According to the calculation formula in steps (1)-(3), determine the magnification of the region where each suspected feature point in the current borehole image is located in the previous borehole image.

[0081] The magnification of the region where each suspected feature point in the current borehole image is located in the previous frame borehole image can be represented by the following formula:

[0082] ;

[0083] ;

[0084] wherein, represents the number of pixel points contained in the region where the u-th suspected feature point in the current borehole image is located in the previous frame borehole image; represents the magnification of the region where the u-th suspected feature point in the current borehole image is located in the previous frame borehole image; represents the number of pixel points contained in the region where the u-th suspected feature point in the current borehole image is located; represents the speed of the camera descending; represents the time interval between adjacent two frames of borehole images; represents the focal length of the camera; represents the total number of pixel points in the borehole image.

[0085] Step S202: based on the magnification and the position of the region where the suspected feature point is located in the image, screening all suspected feature points in the previous frame borehole image to determine each suspected feature point in each frame borehole image as a to-be-analyzed feature point in the previous frame borehole image.

[0086] Considering that the moving distance of the borehole inner wall in the borehole image to the image edge is proportional to the image magnification when the borehole inner wall moves to the image edge during the downward movement of the camera, according to the magnification and the position of the region where the suspected feature point is located in the image, the screening of all suspected feature points in the previous frame borehole image can be realized, and the suspected feature point in the current borehole image is most likely to correspond to the feature point in the previous frame borehole image, thereby obtaining the to-be-analyzed feature point.

[0087] For the above step S202, in some possible implementation manners, determining each suspected feature point in each frame borehole image as a to-be-analyzed feature point in the previous frame borehole image includes:

[0088] Step S2021: determining the distance from the region where each suspected feature point in each frame borehole image is located to the center position of the camera in the corresponding borehole image.

[0089] Since the inner wall of the borehole in the borehole image moves to the image edge during the downward movement of the camera, the moving distance is proportional to the image magnification, therefore, in order to determine each suspected feature point in each frame of the borehole image from each to-be-analyzed feature point in the previous frame of the borehole image, first, the distance between the center point of the region where the suspected feature point is located and the camera region center point position in the image is obtained , and the camera region center point position in the image refers to the center point position in the image. Figure 3 The distance between the center point of the region where the u-th suspected feature point in a certain frame of the borehole image is located and the camera region center point position in the current image is shown as .

[0090] Step S2022: According to the distance and the magnification, the reference distance of the region in the previous frame of the borehole image relative to the camera center position is determined.

[0091] According to the distance of the region where each suspected feature point in each frame of the borehole image is located to the camera center position in the current frame of the borehole image and the magnification, the reference distance of the region where each suspected feature point in each frame of the borehole image is located in the previous frame of the borehole image relative to the camera center position is determined:

[0092] ;

[0093] wherein, represents the reference distance of the region where the u-th suspected feature point in each frame of the borehole image is located in the previous frame of the borehole image relative to the camera center point; represents the distance between the center point of the region where the u-th suspected feature point in each frame of the borehole image is located and the camera region center point position in the current image, that is, the distance of the region where the u-th suspected feature point is located in the current image relative to the camera center point; represents the magnification of the region where the u-th suspected feature point in the current frame of the borehole image is located in the previous frame of the borehole image.

[0094] For Figure 3 The reference distance of the region where the u-th suspected feature point in a certain frame of the borehole image is located in the previous frame of the borehole image relative to the camera center position is shown as as Figure 4 indicated.

[0095] Step S2023: According to the reference distance, all suspected feature points in the previous frame of the borehole image are screened to determine each to-be-analyzed feature point.

[0096] According to the reference distance, each suspected feature point in each frame of the drilling image is screened from all suspected feature points in the previous frame of the drilling image, suspected feature points whose distance between the center point of the region where the suspected feature point is located and the position of the center point of the camera region in the previous frame of the drilling image is equal to the reference distance are screened out, and the suspected feature points are determined as the feature points to be analyzed. It should be understood that, considering the existence of errors, when the absolute value of the difference between the distance between the center point of the region where the suspected feature point is located in the previous frame of the drilling image and the position of the center point of the camera region in the previous frame of the drilling image and the reference distance is less than the set distance threshold, the distance is considered to be equal to the reference distance. The set distance threshold can be reasonably set according to the actual error size, which is not limited here.

[0097] Step S203: Based on the magnification, the degree of coincidence between the region where each suspected feature point in each frame of the drilling image is located and the region where each feature point to be analyzed is located is determined.

[0098] In order to facilitate the matching of the region where each suspected feature point in the current drilling image is located in the previous frame of the drilling image, it is necessary to reduce the region where each suspected feature point is located to the size it should have in the previous frame of the drilling image according to the magnification. Further, the reduced region where each suspected feature point in the current drilling image is located is sequentially analyzed with the region where the corresponding feature point to be analyzed is located in the previous frame of the drilling image, so that the degree of coincidence between the region where each suspected feature point in the current drilling image is located and the region where each feature point to be analyzed is located can be obtained.

[0099] For the above step S203, in some possible implementation manners, based on the magnification, the degree of coincidence between the region where each suspected feature point in each frame of the drilling image is located and the region where each feature point to be analyzed is located is analyzed, including:

[0100] Step S2031: According to the magnification, the region where each suspected feature point in each frame of the drilling image is located is reduced to obtain a reduced region.

[0101] The maximum width of the region where the u-th suspected feature point in the current frame of the drilling image is located in the horizontal direction is obtained and the maximum height in the vertical direction , the current frame of the drilling image refers to any frame of the drilling image except the first frame of the drilling image, then the width and height of the region after reduction should be:

[0102] ;

[0103] ;

[0104] wherein, and respectively represent the maximum width in horizontal direction and the maximum height in vertical direction of the region of the u-th suspected feature point in the current frame drilling image after the region is reduced in size; represents the magnification of the region of the u-th suspected feature point in the current frame drilling image in the previous frame drilling image.

[0105] After the maximum width in horizontal direction and the maximum height in vertical direction of the region of the u-th suspected feature point in the current frame drilling image are determined by the above-mentioned manner and then, by using the bicubic interpolation method, the region of the u-th suspected feature point in the current frame drilling image is reduced to and in size, thereby obtaining the reduced processing region of the region of the u-th suspected feature point in the current frame drilling image.

[0106] Step S2032: determining the number of overlapping pixel points between the reduced processing region of each suspected feature point in each frame drilling image and the region of each feature point to be analyzed.

[0107] Because the camera may rotate during movement, the region of the same feature point cannot be directly matched, and the reduced processing region of the suspected feature point needs to be rotated first, and based on the rotated reduced processing region, the matching of the region of the same feature point is performed, thereby determining the number of overlapping pixel points between the reduced processing region of each suspected feature point in each frame drilling image and the region of each feature point to be analyzed.

[0108] For the above-mentioned step S2032, in some possible implementation manners, determining the number of overlapping pixel points between the reduced processing region of each suspected feature point in each frame drilling image and the region of each feature point to be analyzed comprises:

[0109] Step S20321: determining a first connecting line of a connecting line between the center point of the reduced processing region of each suspected feature point in each frame drilling image and the center position of the camera in the corresponding drilling image.

[0110] determining a connecting line between the center point of the region of each suspected feature point in the current frame drilling image after the region is reduced in size and the center position of the camera in the current drilling image, and recording the connecting line as a first connecting line.

[0111] Step S20322: Determine the line connecting the center point of the region where each of the suspected feature points in the current frame of the drilling image is located to the position of the camera center in the previous frame of the drilling image, to obtain a second line.

[0112] Determine the line connecting the center point of the region where each of the suspected feature points in the current frame of the drilling image is located to the position of the camera center in the previous frame of the drilling image, to obtain a second line.

[0113] Step S20323: According to the positions of the first line and the second line, determine a rotation angle, and perform rotation processing on the reduced processing region of each of the suspected feature points in each frame of the drilling image according to the rotation angle, and determine the number of overlapping pixel points between the reduced processing region of each of the suspected feature points in each frame of the drilling image and the region where each of the analyzed feature points is located based on the reduced processing region after rotation processing.

[0114] Map the first line and the second line to the same image, for example, in the case of keeping the position of the first line in the two images the same, the first line can be mapped to the previous frame of the drilling image, and the rotation angle between the mapped first line and the second line in the previous frame of the drilling image is determined, which is an angle size with a positive or negative sign, the angle size refers to the angle formed by the mapped first line and the second line, which is less than or equal to 180°, and the positive or negative sign is used to represent the rotation direction, when the mapped first line reaches the second line by rotating the angle size clockwise, the sign of the angle size is positive, otherwise when the mapped first line reaches the second line by rotating the angle size counterclockwise, the sign of the angle size is negative. As shown in Figure 5 For the u-th suspected feature point in a frame of the drilling image as shown in Figure 3 The mapped first line corresponding to reaches the second line corresponding to by rotating the angle size clockwise , then the rotation angle between the mapped first line and the second line in the previous frame of the drilling image is .

[0115] After determining the rotation angle in the above manner, the reduced processing region of each suspected feature point in each frame of the drilling image is rotated according to the rotation angle, and the number of overlapping pixel points between the reduced processing region after rotation processing and the region where each of the analyzed feature points is located is determined.

[0116] Step S2033: determining the coincidence degree between the region where each of the suspected feature points in each frame of the drilling image is located and the region where each of the feature points to be analyzed is located according to the difference between the number of pixel points in the reduced processing region of each of the suspected feature points in each frame of the drilling image and the corresponding number of coincident pixel points.

[0117] The difference between the number of pixel points in the reduced processing region of each suspected feature point in each frame of the drilling image and the corresponding number of coincident pixel points is analyzed, so as to determine the coincidence degree between the region where each suspected feature point in each frame of the drilling image is located and the region where each feature point to be analyzed in the previous frame of the drilling image is located. When the difference is smaller, it indicates that the coincidence degree between the two regions is higher, and the value of the corresponding coincidence degree should be larger.

[0118] In some possible implementation manners, the pixel coincidence degree between the region where each of the suspected feature points in each frame of the drilling image is located and the region where each of the feature points to be analyzed is located is determined, including: determining the absolute value of the difference between the number of pixel points in the reduced processing region of each of the suspected feature points in each frame of the drilling image and the corresponding number of coincident pixel points; and performing negative correlation normalization processing on the absolute value of the difference, so as to obtain the coincidence degree between the region where each of the suspected feature points in each frame of the drilling image is located and the region where each of the feature points to be analyzed is located, and the corresponding calculation formula is:

[0119] ;

[0120] wherein, represents the coincidence degree between the region where the u th suspected feature point in the current drilling image is located and the region where the v th feature point to be analyzed in the previous frame of the drilling image is located; represents the number of pixel points contained in the region where the u th suspected feature point in the current drilling image is located after being reduced, that is, the number of pixel points in the reduced processing region of the u th suspected feature point in the current drilling image; represents the number of coincident pixel points between the reduced processing region of the u th suspected feature point in the current drilling image after being rotated and the region where the v th feature point to be analyzed in the previous frame of the drilling image is located; represents a normalization function.

[0121] In the above calculation formula, the number of pixel points contained in the region where the u th suspected feature point in the current drilling image is located after being reduced , and the number of coincident pixel points between the reduced processing region of the u th suspected feature point in the current drilling image after being rotated and the region where the v th feature point to be analyzed in the previous frame of the drilling image is located the absolute value of the difference, and taking the reciprocal of the absolute value of the difference plus 1 to perform negative correlation processing. The purpose of adding 1 to the absolute value of the difference is to prevent the denominator of the reciprocal from being zero, and then the reciprocal is normalized by using the norm function, thereby obtaining the coincidence degree. When the absolute value of the difference is smaller, it indicates that the region where the u-th suspected feature point in the current borehole image coincides with the region where the v-th feature point to be analyzed in the previous borehole image is higher, and the v-th feature point to be analyzed is more likely to be the feature point corresponding to the u-th suspected feature point in the previous borehole image. The value of the corresponding coincidence degree is larger.

[0122] Step S204: According to the coincidence degree between each suspected feature point in each borehole image and all the feature points to be analyzed, the possibility of each suspected feature point in each borehole image appearing in the previous borehole image is determined.

[0123] Since the larger the coincidence degree between each suspected feature point in each borehole image and each feature point to be analyzed is, it indicates that the two corresponding feature points are most likely to be the same feature point. Therefore, the possibility of each suspected feature point in the current borehole image appearing in the previous borehole image can be determined. The implementation steps include: determining the maximum value in the coincidence degree between each suspected feature point in each borehole image and all the feature points to be analyzed, and determining the maximum value as the possibility of each suspected feature point in each borehole image appearing in the previous borehole image.

[0124] After determining the possibility of each suspected feature point in each borehole image appearing in the previous borehole image in the above manner, each suspected feature point in each borehole image can be screened according to the possibility, thereby determining each stable feature point.

[0125] In some possible implementation manners, a possibility threshold is set in advance, and it is determined whether the possibility of each suspected feature point in each borehole image appearing in the previous borehole image is greater than or equal to the possibility threshold. The suspected feature point whose possibility is greater than or equal to the possibility threshold is determined as a stable feature point. In some possible implementation manners, the value of the possibility threshold is set to 0.7. When the possibility of the u-th suspected feature point in the current borehole image is greater than or equal to 0.7, the suspected feature point is considered as a stable feature point; when the possibility of the u-th suspected feature point in the current borehole image is less than 0.7, the suspected feature point is considered as an unstable feature point and is discarded. Therefore, each stable feature point in each borehole image can be determined.

[0126] ​​Step S300: based on the respective stable feature points in each borehole image of the borehole video frame images to be stitched, stitching the borehole video frame images to be stitched to obtain a stitched image.

[0127] After determining the respective stable feature points in each borehole image of the borehole video frame images to be stitched through the above steps, a feature description, that is, a feature vector, is generated, and the borehole video frame images to be stitched are stitched according to the feature vector, and finally a stitched image is obtained.

[0128] In some possible implementation manners, for each borehole image of the borehole video frame images to be stitched, the specific process of determining the feature vector of the stable feature point is as follows:

[0129] a. Calculate the main direction of the feature point.

[0130] The calculation of the main direction of the feature point is to make the descriptor of the feature point invariant to rotation. First, the gradient amplitude and gradient direction of each pixel point in the current image are calculated by using the Sobel operator. Then, a 16x16 (empirical value) window is set with each feature point as the center. The gradient directions of the pixel points in the window corresponding to each feature point are generated into a direction histogram with every 10° as an interval. Finally, the middle value of each interval is taken as the direction corresponding to each column. The direction corresponding to the peak value in the direction histogram is the main direction of the feature point.

[0131] b. Rotate the x-axis of the current borehole image to the main direction of the feature point.

[0132] c. Generate the feature description of the feature point.

[0133] First, the 16x16 window corresponding to each feature point is divided into 4x4 sub-windows of 4x4. Then, the gradient amplitude and gradient direction of the pixel points in each sub-window are calculated by using the Sobel operator. A direction histogram is generated with every 45° as an interval, that is, the direction histogram corresponds to 8 directions. The direction histograms of these sub-windows are combined to obtain the descriptor of the feature point. Finally, the feature vectors of each sub-window are obtained according to the direction histogram of each sub-window, and the feature vectors are normalized to improve the adaptability of the feature point to different illuminations.

[0134] Thus, the feature vectors of all the stable feature points in each borehole image of the borehole video frame images to be stitched are obtained. It should be understood that the above merely gives one specific implementation manner of obtaining the feature vectors of the stable feature points, and other means in the prior art can also be used to obtain the feature vectors of the stable feature points, which are not limited herein.

[0135] Considering that the inner wall of the borehole is cylindrical and the camera is vertically downward when it goes into the borehole, the shooting angle of the camera is downward, so the image seen in the camera is conical, but it is actually the inside of the cylinder. When stitching, each frame of the borehole image needs to be unfolded. In some possible implementations, when implemented, a circular ring with an inner radius r and an outer radius R is set for each frame of the shot borehole image, and the two circular rings have the same center. The area inside the circular ring is the area to be unfolded. At this time, the unfolding process includes:

[0136] d. Establish a rectangular coordinate system with the lower left corner of the borehole image as the origin, the horizontal right as the positive direction of the x-axis, and the vertical up as the positive direction of the y-axis, to obtain the coordinates of the center of the borehole image in the borehole image and the coordinates of each pixel point .

[0137] e. Establish a polar coordinate system with the center of the borehole image as the origin and the positive direction of the x-axis as the positive direction, to obtain the polar coordinate angle of each pixel point of the borehole image .

[0138] f. Establish the coordinate system of the unfolded image. Since only the circular ring area is unfolded, the unfolded image should be rectangular.

[0139] g. According to the coordinates of the pixel points in the borehole image, calculate the corresponding coordinates in the unfolded image coordinate system. The corresponding relationship between the coordinate values of the pixel points in the unfolded image and their coordinates in the original image is:

[0140] ;

[0141] wherein, represents the coordinates of the i-th pixel point in the shot borehole image; represents the center coordinates of the shot borehole image, represents the coordinates of the i-th pixel point in the unfolded image, represents the inner radius of the circular ring, represents the outer radius of the circular ring, represents the polar coordinate angle of the i-th pixel point in the shot borehole image.

[0142] Thus, the unfolded image of each frame of the borehole image to be stitched can be obtained. After unfolding all the borehole images, the borehole images of adjacent frames are stitched, and the specific process is as follows:

[0143] First, match the stable feature points in the borehole images of adjacent frames. If the feature vectors of two stable feature points are consistent, it is considered that the two stable feature points are matched.

[0144] Then, taking the coordinates of the previous frame of the drilling image as a standard, the coordinates of the drilling image of the next frame are adjusted according to the matched stable feature points, so that the pixel point coordinates satisfy the coordinate system of the previous frame of the drilling image.

[0145] Secondly, because the brightness of the two drilling images may be different due to light changes during shooting, it is necessary to adjust the two drilling images to the same brightness level. Here, the brightness of the next drilling image is adjusted to be consistent with the brightness level of the previous drilling image by using a histogram matching method.

[0146] Finally, for the overlapping part of the two drilling images, the average value of the gray values in the two images at the same position is taken as the gray value of the pixel point at the position.

[0147] In this way, the spliced drilling television shooting image, that is, the spliced image, can be obtained. Figure 6 A schematic diagram of a spliced image is shown.

[0148] Based on the same inventive concept, the embodiments of the present application also provide an image splicing system for drilling television, comprising:

[0149] A suspected feature point determination module is configured to: acquire suspected feature points in each drilling image of the drilling video frame images to be spliced;

[0150] A stable feature point determination module is configured to: analyze the possibility of each suspected feature point in each drilling image appearing in the previous frame of drilling image, and screen each suspected feature point in each drilling image according to the possibility to determine each stable feature point.

[0151] An image splicing module is configured to: splice the drilling video frame images to be spliced based on the stable feature points in each drilling image, to obtain a spliced image.

[0152] It should be noted that: the system provided by the above embodiments is only exemplified by the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above.

[0153] Based on the same inventive concept, the embodiments of the present application also provide an image splicing device for drilling television, as shown in Figure 7 The device comprises a memory 701 and a processor 702; the memory 701 is used to store executable program code 703; the processor 702 is used to call and run the executable program code 703 from the memory 701 to realize any one of the foregoing image splicing methods for drilling television.

[0154] The embodiments of the present application can divide the functional modules of the device according to the above method examples, for example, each functional module can be corresponded, or two or more functions can be integrated in one processing module, and the integrated module can be realized in the form of hardware. It should be noted that the division of the modules in the embodiments is illustrative, and is only a logical function division, and another division mode can be used in actual implementation.

[0155] It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of image mosaicking for borehole television, characterized by, The method comprises the following steps: obtaining a suspected feature point in each drilling image frame in a drilling video frame image to be spliced; analyzing the possibility of each suspected feature point in each drilling image frame appearing in the previous drilling image frame, and screening each suspected feature point in each drilling image frame according to the possibility to determine each stable feature point; splicing the drilling video frame image to be spliced based on each stable feature point in each drilling image frame to obtain a spliced image; wherein analyzing the possibility of each suspected feature point in each drilling image frame appearing in the previous drilling image frame comprises: determining the region where each suspected feature point in each drilling image frame is located, and the magnification of the region in the previous drilling image frame; screening all suspected feature points in the previous drilling image frame based on the magnification and the location of the region where the suspected feature point is located in the image to determine the distance from the region where each suspected feature point in each drilling image frame is located to the center position of the camera in the corresponding drilling image; determining the reference distance of the region where each suspected feature point in each drilling image frame is located relative to the center position of the camera in the previous drilling image frame according to the distance and the magnification; screening all suspected feature points in the previous drilling image frame according to the reference distance to determine each feature point to be analyzed; According to the magnification, the region where each suspected feature point in each drilling image frame is located is reduced to obtain a reduced region; the number of overlapping pixel points between the reduced region of each suspected feature point in each drilling image frame and the region where each corresponding feature point to be analyzed is located is determined; the coincidence degree between the region where each suspected feature point in each drilling image frame is located and the region where each corresponding feature point to be analyzed is located is determined according to the difference between the number of pixel points in the reduced region of each suspected feature point in each drilling image frame and the corresponding number of overlapping pixel points; the greater the difference, the smaller the corresponding coincidence degree; According to the coincidence degree between each suspected feature point in each drilling image frame and all corresponding feature points to be analyzed, the possibility of each suspected feature point in each drilling image frame appearing in the previous drilling image frame is determined.

2. The image stitching method for borehole television according to claim 1, wherein, Determination of the coincidence degree between the region where each suspected feature point in each drilling image frame is located and the region where each corresponding feature point to be analyzed is located also includes: determining the first connecting line from the center point of the reduced region of each suspected feature point in each drilling image frame to the center position of the camera in the corresponding drilling image to obtain the first connecting line; determining the second connecting line from the center point of the region where each feature point to be analyzed in each drilling image frame is located in the previous drilling image to the center position of the camera in the previous drilling image to obtain the second connecting line; According to positions of the first connecting line and the second connecting line, a rotation included angle is determined, a rotation processing is performed on a reduced processing region of each of the suspected feature points in each of the drilling image frames according to the rotation included angle, and a number of overlapping pixel points between the reduced processing region of each of the suspected feature points in each of the drilling image frames and a region where each of the to-be-analyzed feature points is located is determined based on the reduced processing region after the rotation processing.

3. The image stitching method for borehole television according to claim 2, wherein, The determination of the coincidence degree between the region where each of the suspected feature points in each of the drilling image frames is located and the region where each of the to-be-analyzed feature points is located comprises: determining an absolute value of a difference between a number of pixel points of the reduced processing region of each of the suspected feature points in each of the drilling image frames after the rotation processing and the number of overlapping pixel points; and performing a negative correlation normalization processing on the absolute value of the difference, so as to obtain the coincidence degree between the region where each of the suspected feature points in each of the drilling image frames is located and the region where each of the to-be-analyzed feature points is located.

4. The image stitching method for borehole television according to any one of claims 1-3, characterized in that, The determination of the possibility of each of the suspected feature points in each of the drilling image frames appearing in the previous drilling image comprises: determining a maximum value in the coincidence degrees between each of the suspected feature points in each of the drilling image frames and all of the to-be-analyzed feature points, and determining the maximum value as the possibility of each of the suspected feature points in each of the drilling image frames appearing in the previous drilling image.

5. The image stitching method for borehole television according to claim 4, wherein, The determination of the stable feature points comprises: judging whether the possibility of each of the suspected feature points in each of the drilling image frames appearing in the previous drilling image is greater than or equal to a possibility threshold value, and determining the suspected feature points corresponding to the possibility greater than or equal to the possibility threshold value as the stable feature points.

6. A borehole television oriented image mosaicking system characterized by, The system is configured to perform the drilling television image splicing method in any one of claims 1 to 5, and the system comprises: a suspected feature point determination module configured to acquire suspected feature points in each of drilling image frames in drilling video frame images to be spliced; a stable feature point determination module configured to analyze the possibility of each of the suspected feature points in each of the drilling image frames appearing in the previous drilling image, and screen each of the suspected feature points in each of the drilling image frames according to the possibility, so as to determine the stable feature points; an image splicing module configured to splice the drilling video frame images to be spliced based on the stable feature points in each of the drilling image frames, so as to obtain a spliced image.

7. A bore television oriented image mosaicking apparatus, characterized by, The system comprises a memory and a processor; the memory is configured to store executable program codes; and the processor is configured to call and run the executable program codes from the memory, so as to implement the drilling television image splicing method in any one of claims 1 to 5.

Images