Image splicing method for tunnel inspection area-array camera

Abstract

The invention relates to a tunnel inspection area-array camera image splicing method, which comprises the following steps that distortion correction is carried out on images to unify the physical resolution of the images acquired by each area-array camera, and the distortion correction comprises transverse distortion correction and longitudinal distortion correction on the images; wherein the transverse distortion correction refers to the direction parallel to the axial direction of the tunnel, and the longitudinal distortion correction refers to the circumferential direction of the tunnel; zooming the images subjected to distortion correction so as to further unify the physical resolution of each camera image; and feature points of adjacent images are calculated based on an SURF algorithm and are matched, then the pixel offset of each matched feature point is calculated, the offset of the images is obtained after averaging, and image splicing is completed. The system has the advantages that the acquisition equipment is simple in structure and easy to maintain, and does not contain additional auxiliary equipment for laser ranging, positioning and the like; the collection operation is convenient, and the collection platform is high in universality; the image splicing algorithm is efficient; the original image data utilization rate is high.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel vision detection, and in particular to a tunnel inspection area array camera image stitching method. Background Art

[0002] Tunnel mobile visual scanning can obtain panoramic images of the tunnel wall through the tunnel inspection area array camera mounted on the self-moving platform. Compared with 3D laser scanning, the area array camera has lower equipment cost and higher image resolution. High-resolution panoramic images of the tunnel wall can be used for crack detection, which greatly improves the efficiency of daily tunnel monitoring.

[0003] The panoramic image of the inner wall of the tunnel needs to be obtained by stitching images taken by area array cameras. The basic principle of image stitching is the matching of image feature points. However, due to the curvature of the tunnel and the shooting angle of the camera, the captured image will be distorted. The distortion of the image will greatly affect the matching of image feature points and thus the effect of image stitching.

[0004] Invention patent CN116128734A discloses an image stitching method based on deep learning, which uses deep learning to match feature points. However, this method requires a large amount of data for model training in advance. In the face of different tunnels, it is necessary to collect new data sets to retrain and improve the model.

[0005] Invention patent CN113256714A discloses a tunnel surface image processing method, which requires the addition of additional distance measuring instruments to assist in image distortion correction, and the collection process is more complicated.

[0006] Invention patent CN114820311A discloses a method for cylindrical panoramic stitching of tunnel inspection area array camera images. Although it does not require additional information and complex algorithms, it requires the camera to collect images co-centered with the tunnel. Tunnels of different specifications require adjustable collection equipment, and on-site debugging by operators is required. In addition, it is not possible to achieve co-centering with the collection equipment in some large circular tunnels.

[0007] Invention patent CN116703720 A discloses a method for stitching photos from a tunnel array camera. The method does not perform distortion correction on the photos, but instead increases the overlap rate of adjacent photos from a single camera and identifies tunnel annular joints based on a deep learning algorithm. The physical resolution of images between multiple cameras is unified based on the tunnel annular joints, and a panoramic image of the tunnel is finally obtained. The high data redundancy and deep learning algorithm increase the complexity of internal data processing. The method is only applicable to shield tunnels with annular joints and cannot be applied to mining tunnels without annular joints. Summary of the invention

[0008] The purpose of the present invention is to provide a tunnel inspection area array camera image stitching method based on the above-mentioned deficiencies in the prior art, which performs distortion correction by calculating the difference in physical resolution of each pixel of the image, and then calculates the feature points of adjacent images after correction based on the SURF algorithm, and stitches the images according to the feature point matching, so as to finally obtain a panoramic image of the tunnel, thereby improving the stitching efficiency and accuracy of the photos.

[0009] The purpose of the present invention is achieved by the following technical solutions:

[0010] A tunnel inspection area array camera image stitching method is used to stitch images collected by a plurality of area array cameras, and is characterized in that the image stitching method comprises the following steps:

[0011] Performing distortion correction on the image to unify the physical resolution of the image collected by each array camera, the distortion correction includes lateral distortion correction and longitudinal distortion correction on the image, wherein the lateral distortion correction refers to the direction parallel to the axial direction of the tunnel, and the longitudinal distortion correction refers to the circumferential direction of the tunnel;

[0012] Scaling the distortion-corrected image to further unify the physical resolution of each camera image;

[0013] Based on the SURF algorithm, the feature points of adjacent images are calculated and matched, and then the pixel offset of each matching feature point is calculated. The image offset is obtained after averaging to complete the image stitching.

[0014] The lateral distortion correction is performed by calculating the distance l from each row of pixels in the image to the camera and selecting the farthest distance l max , in l / l max The horizontal pixels in each row of the image are scaled by the coefficient to achieve normalization of the horizontal physical resolution.

[0015] The longitudinal distortion correction is performed by dividing the longitudinal pixels in the image into several sub-intervals at equal intervals, calculating the longitudinal length h of each sub-interval projected on the tunnel wall, and selecting the longest projection length h. max , in h / h max The vertical pixels of each sub-interval of the image are scaled by the coefficient to achieve normalization of the vertical physical resolution.

[0016] The distortion-corrected image is scaled based on the distance s from the camera along the axis to the tunnel ring, and the scaling factor is s max is the maximum distance from the camera to the tunnel ring along the axis.

[0017] The advantages of the present invention are:

[0018] 1) The acquisition equipment has a simple structure and is easy to maintain, and does not include additional auxiliary equipment such as laser ranging and positioning;

[0019] 2) The acquisition operation is convenient and the acquisition platform is highly versatile: the acquisition operation does not require the center of the equipment to be at the center of the tunnel ring, and can be easily mounted on various tunnel inspection mobile platforms. During the acquisition operation, you only need to install the fixed equipment and turn on the power to start image acquisition;

[0020] 3) Efficient image stitching algorithm: The distortion correction algorithm is based on the geometric relationship between the area array camera and the tunnel ring piece. This geometric relationship usually does not change during an operation. Therefore, the correction coefficient only needs to be calculated once, and it does not involve algorithms with high time complexity and does not require high performance of the data processing platform.

[0021] 4) High utilization rate of original image data: Since the distortion correction of the image has been completed before image stitching, the overlap rate of adjacent images is not required to be high. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the structure of the tunnel inspection area array camera involved in the present invention;

[0023] Figure 2 It is a schematic diagram of the acquisition operation and image distortion principle of the tunnel area array camera in the present invention;

[0024] Figure 3 It is a schematic diagram of the principle of image lateral correction in the present invention;

[0025] Figure 4 This is the image horizontal correction contrast effect diagram;

[0026] Figure 5 It is a schematic diagram of the principle of image longitudinal correction in the present invention;

[0027] Figure 6 This is the image longitudinal correction contrast effect diagram;

[0028] Figure 7 This is the image effect after the physical resolution of each camera is unified;

[0029] Figure 8 This is the effect of stitching the panoramic image of the tunnel inner wall. DETAILED DESCRIPTION

[0030] The features of the present invention and other related features are further described in detail below through embodiments in conjunction with the accompanying drawings to facilitate understanding by those skilled in the art:

[0031] Embodiment: In this embodiment, the tunnel inspection area array camera image stitching method is applied to the tunnel mobile visual scanning operation. Based on the geometric relationship between the area array camera and the tunnel ring piece, this embodiment realizes the distortion correction of the original tunnel image and the unification of the physical resolution of each camera image. The processed image data can obtain a high-quality panoramic image of the tunnel inner wall through SURF feature point matching and the existing image fusion method.

[0032] like Figure 1 As shown, the area array camera array used in this embodiment includes multiple area array cameras. When collecting image data in the field, the area array cameras are mounted on the inspection vehicle. Each area array camera faces a different direction. The area array cameras are used to obtain a panoramic image of the inner wall of the tunnel, so as to judge the current situation of the inner wall of the tunnel, such as detecting cracks, damage and other defects on the inner wall of the tunnel. As the inspection vehicle moves, the area array camera continues to collect images of the inner wall of the tunnel. In this embodiment, there is no need to ensure that the center of the area array camera coincides with the center of the tunnel ring. Image acquisition can be started by turning on the power and opening the acquisition control software.

[0033] like Figure 2 As shown in the figure, since the center of the tunnel inspection area array camera cannot coincide with the center of the tunnel ring piece, the distance from each camera to each part of the tunnel ring piece is different, resulting in different physical resolutions of the image, that is, the actual physical distance corresponding to each pixel point is different, resulting in distortion of the projection of the image obtained by the area array camera on the tunnel ring wall.

[0034] In this embodiment, the images acquired by the array camera are subjected to distortion correction, that is, the physical resolution of the images is unified, thereby improving the stitching accuracy, and further improving the accuracy of subsequent image-based tunnel detection.

[0035] Specifically, the distortion correction in this embodiment includes lateral distortion correction of the image and longitudinal distortion correction of the image. The lateral direction of the image is a direction parallel to the axial direction of the tunnel, and the longitudinal direction of the image is the circumferential direction of the tunnel.

[0036] In this embodiment, the lateral distortion correction of the image includes the following schemes:

[0037] The physical resolution of the horizontal pixels of the image is calculated by calculating the distance l from each row of pixels to the camera and selecting the farthest distance l max , in l / l max The coefficient is used to scale the horizontal pixels of each row of the image to achieve the normalization of the horizontal physical resolution. The distance l from the pixel to the camera is calculated as follows:

[0038] like Figure 3As shown, O is the center of the tunnel ring, C is the center of the array camera, B is the position of a camera, A is the intersection of the i-th row of pixels in the image collected by the camera and the tunnel ring, and D is the intersection of the camera axis and the tunnel ring. Figure 3 In the figure, the distance between the center of the array camera and the center of the tunnel ring is OC, the radius of the tunnel ring is OA, the device radius of the array camera is BC, and the angle ∠OCB between the camera axis and the vertical are all known quantities. The line segment AB is the distance from the i-th row of pixels to the camera.

[0039]

[0040] ∠OBA=2π-∠OBC-∠ABD

[0041]

[0042] Among them, i is the current pixel row number, n is the total number of pixel rows, L c is the length of the photosensitive element, and f is the focal length of the camera. If OA, OB, and ∠OBA are known, the line segment AB can be obtained.

[0043]

[0044] AB is the distance l from the row of pixels to the corresponding camera. Select the farthest distance l max , in l / l max Scale the horizontal pixels of each image by a factor.

[0045] like Figure 4 As shown, (a) is the original state of the image captured by the area array camera, and (b) is the state after lateral distortion correction.

[0046] In this embodiment, the longitudinal distortion correction of the image includes the following schemes:

[0047] like Figure 5 As shown in the figure, the vertical physical resolution of the image is obtained by dividing the vertical pixels into equal intervals and calculating the vertical length h of each sub-interval projected on the ring wall, that is, AA′, and selecting the longest projection length h max , in h / h max The coefficient is used to scale the vertical pixels of each sub-interval of the image. The calculation method of hmax is as follows:

[0048] ∠ABA′=∠ABD∠A′BD

[0049] AA′=AB 2 +A′B 2 +2AB·A′B·∠ABA′

[0050] Among them, AB, A′B, ∠ABD, and ∠A′BD are all calculated during the lateral distortion correction of the image, that is, the longitudinal distortion correction in this embodiment can be performed after the lateral distortion correction to make full use of various parameters calculated during the lateral distortion correction, thereby improving the calculation efficiency.

[0051] Thus, AA′ is calculated, that is, the longitudinal length h of each sub-interval projected on the ring wall, and the longest projection length h is selected. max , in h / h max The coefficient is used to scale the vertical pixels of each sub-interval of the image.

[0052] like Figure 6 As shown, (a) is the original state of the image captured by the area array camera, and (b) is the state after longitudinal distortion correction.

[0053] After the distortion correction of each camera is completed, the physical resolution of the images taken by different cameras is still different, which can be calculated by the distance s from the camera to the tunnel ring along the axis (i.e. Figure 3 The corrected image is scaled by the midline segment BD) as the reference, and the scaling factor is s max is the maximum distance from the camera to the tunnel ring along the axis. The calculation method of BD is the same as that of line segment AB.

[0054] like Figure 7 As shown in the figure, the physical resolution of each camera is unified to facilitate image stitching.

[0055] like Figure 8 As shown, based on the SURF algorithm, feature points of adjacent images are calculated and matched, and then the pixel offset of each matching feature point is calculated, and the image offset is obtained after averaging, and the image splicing is completed. In the splicing process, the existing image fusion method can be used to eliminate the splicing seam.

[0056] Although the above embodiments have described the concepts and embodiments of the present invention in detail with reference to the accompanying drawings, ordinary technicians in this field can recognize that various improvements and changes can still be made to the present invention without departing from the scope of the claims, so they are not described one by one here.

Claims

1. A tunnel inspection area array camera image stitching method, used for stitching images collected by a plurality of area array cameras, characterized in that: The image stitching method comprises the following steps: Performing distortion correction on the image to unify the physical resolution of the image collected by each array camera, the distortion correction includes lateral distortion correction and longitudinal distortion correction on the image, wherein the lateral distortion correction refers to the direction parallel to the axial direction of the tunnel, and the longitudinal distortion correction refers to the circumferential direction of the tunnel; Scaling the distortion-corrected image to further unify the physical resolution of each camera image; Based on the SURF algorithm, the feature points of adjacent images are calculated and matched, and then the pixel offset of each matching feature point is calculated. The image offset is obtained after averaging to complete the image stitching.

2. The method for stitching images of a tunnel inspection area array camera according to claim 1, characterized in that: The lateral distortion correction is performed by calculating the distance l from each row of pixels in the image to the camera and selecting the farthest distance l max , in l / l max The horizontal pixels in each row of the image are scaled by the coefficient to achieve normalization of the horizontal physical resolution.

3. The method for stitching images of a tunnel inspection area array camera according to claim 1, characterized in that: The longitudinal distortion correction is performed by dividing the longitudinal pixels in the image into several sub-intervals at equal intervals, calculating the longitudinal length h of each sub-interval projected on the tunnel wall, and selecting the longest projection length h. max , in h / h max The vertical pixels of each sub-interval of the image are scaled by the coefficient to achieve normalization of the vertical physical resolution.

4. The method for stitching images of a tunnel inspection area array camera according to claim 1, characterized in that: The distortion-corrected image is scaled based on the distance s from the camera along the axis to the tunnel ring, and the scaling factor is s max is the maximum distance from the camera to the tunnel ring along the axis.

Images