A multi-scale lining image stitching method in a tunnel detection parallax scene and a computer program product
A multi-scale lining image stitching method combining CNN feature pyramid network and TPS interpolation algorithm solves the problems of image stitching accuracy and adaptability in tunnel detection parallax scenarios, generating high-quality panoramic images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UESTC (SHENZHEN) ADVANCED RES INST
- Filing Date
- 2026-04-23
- Publication Date
- 2026-06-23
AI Technical Summary
Existing methods for stitching lining images in parallax scenarios during tunnel inspection suffer from low multi-scale feature alignment accuracy, insufficient non-rigid registration, difficulty in completely eliminating stitching artifacts, poor adaptability, and difficulty in ensuring the quality of lining image stitching.
A feature pyramid network based on convolutional neural network (CNN) is used for multi-scale feature extraction. Global and local registration is performed by combining multi-scale matrix iterative optimization and TPS interpolation algorithm. Content masking is used to hide artifacts and generate high-quality panoramic images.
It achieves high-precision global and local registration in tunnel inspection parallax scenarios, effectively eliminates stitching artifacts, improves the stitching quality of lining images, adapts to the complex needs of different parallax levels and tunnel spaces, and outputs wide-view, high-resolution panoramic images.
Smart Images
Figure CN122089564B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a multi-scale lining image stitching method and computer program product for tunnel detection parallax scenarios. Background Technology
[0002] As a critical infrastructure for land transportation, highway tunnels are prone to defects such as cracks, water leakage, and spalling in their lining structures (permanent support layers constructed in tunnel engineering to maintain the stability of the surrounding rock and ensure structural safety) during long-term operation. Automated detection of highway tunnel defects based on visual images has become a mainstream technology in the industry. In actual inspection operations, inspection vehicles / drones continuously take pictures along the longitudinal direction of the tunnel. The camera's angle, distance, and posture constantly change, resulting in a sequence of images that generally suffer from problems such as large parallax, local deformation, inconsistent scale, and complex overlapping areas.
[0003] Image stitching technology is the core step in synthesizing a complete panoramic image from multiple frames of local tunnel lining images, enabling the full-area display, localization, and quantification of defects. However, highway tunnel scenes are characterized by strong parallax, long distances, monotonous textures, uneven lighting, and large local deformations. Traditional stitching methods are prone to problems such as low multi-scale feature alignment accuracy, insufficient non-rigid registration, and difficulty in completely eliminating stitching artifacts. These issues directly affect the accuracy of subsequent defect detection, identification, and measurement, and cannot meet the engineering requirements for high-quality panoramic imaging and defect detection in highway tunnels.
[0004] In the process of realizing this invention, the inventors discovered at least the following problems in the prior art:
[0005] Existing methods for stitching lining images in parallax scenarios during tunnel inspection suffer from low multi-scale feature alignment accuracy, insufficient non-rigid registration, difficulty in completely eliminating stitching artifacts, poor adaptability, and difficulty in ensuring the quality of lining image stitching. Summary of the Invention
[0006] The purpose of this invention is to provide a multi-scale lining image stitching method and computer program product for tunnel inspection parallax scenarios, to solve the technical problems of low multi-scale feature alignment accuracy, insufficient non-rigid registration, difficulty in completely eliminating stitching artifacts, poor adaptability, and difficulty in ensuring the quality of lining image stitching in existing tunnel inspection parallax scenarios. The various technical effects of the preferred solutions provided by this invention are detailed below.
[0007] To achieve the above objectives, the present invention provides the following technical solution:
[0008] This invention provides a multi-scale lining image stitching method for tunnel detection parallax scenarios, comprising the following steps: S100: Inputting two overlapping images, a first image and a second image, as tunnel lining images to be stitched in a parallax scenario; using a feature pyramid network based on a convolutional neural network (CNN) to simultaneously extract features from the processed first and second images, generating multiple layers of feature maps at different scales; performing cross-image feature matching on each feature map layer; simultaneously extracting features from the first and second images and performing feature map matching; obtaining an initial global homography matrix based on the coordinates of the matched feature points; S200: performing multi-scale matrix iterative optimization and generating globally aligned feature maps based on the initial global homography matrix to obtain the optimal global homography matrix. S300: Apply the optimal global homography matrix to the original image of the second image for global rigid alignment to obtain the globally rigid aligned second image. Perform local non-rigid adjustment on the globally rigid aligned second image using the TPS interpolation algorithm, and output the registered first and second images. S400: Extract the mask boundaries from the registered first and second images to obtain the initial mask, and generate the seam mask. S500: Find the optimal seam within the seam mask, and perform pixel-level expansion on the optimal seam. Perform a bitwise AND operation between the expanded seam mask and the initial mask to generate the content mask. S600: Perform fusion processing on the registered first and second images based on the content mask to hide artifacts in the stitching and output the panoramic image in the tunnel detection parallax scene.
[0009] Preferably, step S100 specifically includes: S110: performing grayscale normalization and pixel value normalization processing on the input first image and second image; S120: aligning the feature maps of the first image and second image at each scale, and outputting the aligned feature pyramid; S130: using the improved RANSAC algorithm to perform feature point matching and outlier removal on the coarsest scale aligned feature map, and obtaining the initial global homography matrix by solving the projection transformation formula based on the coordinates of the matched feature points.
[0010] Preferably, the feature pyramid network contains four layers of feature maps, which are, from coarse to fine, 1 / 32 scale feature maps, 1 / 16 scale feature maps, 1 / 8 scale feature maps, and 1 / 4 scale feature maps.
[0011] Preferably, step S200 specifically includes: S210: pre-distorting the 1 / 16 scale feature map of the second image using the initial global homography matrix to generate a preliminary globally aligned feature map; S220: performing secondary feature matching between the pre-distorted 1 / 16 scale feature map and the 1 / 16 scale feature map of the first image, and fine-tuning the initial global homography matrix based on the matching result to obtain an optimized global homography matrix; S230: repeating the above steps sequentially for the 1 / 8 scale feature map and the 1 / 4 scale feature map, pre-distorting the corresponding scale feature map of the second image based on the optimized matrix of the previous scale to generate a globally aligned feature map, and then fine-tuning the matrix through feature matching to finally obtain the optimal global homography matrix in the tunnel detection parallax scenario.
[0012] Preferably, in step S300, the process of performing local non-rigid adjustment on the globally rigidly aligned second image using the TPS interpolation algorithm and outputting the registered first and second images specifically includes: S310: uniformly selecting 100 feature control points in the overlapping area of the first and second images; S320: calculating the pixel offset between the control points of the globally aligned second image and the corresponding control points of the first image; S330: fitting the pixel offset using the TPS interpolation algorithm to generate a local distortion transformation field; S340: applying the local distortion transformation field to the globally rigidly aligned second image for local non-rigid alignment, completing the global and local registration of the first and second images, and outputting the registered first and second images.
[0013] Preferably, step S400 specifically includes: S410: performing binarization processing on the two registered images, with the overlapping area as the foreground and the non-overlapping area as the background, to generate an initial mask; S420: extracting the mask boundary of the initial mask using the Canny edge detection algorithm to obtain the mask boundary contours of the two images; S430: taking the mask boundary contours of the intersecting area of the two images to generate a seam mask that marks the core seam area of the stitching, used to adapt to the natural boundary of the image content.
[0014] Preferably, step S500 specifically includes: S510: Based on the seam mask, with the objective function of minimizing the pixel grayscale difference and gradient difference in the splicing area, a dynamic programming algorithm is used to find the optimal seam line within the seam mask, and the optimal seam line extends along the natural edge of the image content; S520: The optimal seam line is expanded pixel-level to both sides to generate an expanded seam mask; S530: The expanded seam mask is bitwise ANDed with the initial mask to remove invalid areas in the mask and generate the final content mask.
[0015] Preferably, in step S600, the fusion processing of the registered first and second images based on the content mask to hide artifacts during the stitching process includes: S610: fusing the registered first and second images based on the content mask to hide artifacts during stitching; S620: using a linear fusion algorithm to smooth the pixel grayscale transition in the fusion area marked by the content mask; S630: directly retaining the pixel information of the original image in the retained area outside the fusion area, and weakening and hiding ghosting and edge blurring generated during the stitching process through pixel weighting of the content mask.
[0016] Preferably, in step S600, outputting the panoramic image in the tunnel detection parallax scene specifically includes: stitching and cropping the edges of the fused image, eliminating the black borders of the stitched image, completing the multi-scale lining image stitching in the tunnel detection parallax scene, and outputting the panoramic image in the tunnel detection parallax scene.
[0017] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of a multi-scale lining image stitching method for tunnel detection parallax scenarios as described above. The method includes an image input module that is sequentially electrically connected and transmits data, a multi-scale feature extraction module, a homography matrix estimation and optimization module, a non-rigid alignment module, a mask processing module, an artifact hiding module, and a panoramic image output module. The image input module is used to input a first image and a second image with overlapping regions. The multi-scale feature extraction module uses a feature pyramid network to perform multi-scale feature extraction on the first image and the second image. The homography matrix estimation and optimization module performs feature point matching and outlier removal on the coarsest-scale aligned feature map, and obtains the initial global homography matrix by solving the projection transformation formula. The non-rigid alignment module applies the optimal global homography matrix to the original image of the second image for global rigid alignment, and performs local non-rigid alignment on the globally rigidly aligned second image using the TPS interpolation algorithm. The mask processing module is used to generate seam masks. The artifact hiding module performs fusion processing on the registered first and second images based on the content mask to hide artifacts in the stitching. The panoramic image output module outputs panoramic images in tunnel detection parallax scenarios.
[0018] Implementing one of the above-described technical solutions of the present invention has the following advantages or beneficial effects:
[0019] In this application, cross-scale feature matching of two lining images in tunnel detection parallax scenarios is achieved through multi-scale feature extraction and alignment from coarse to fine. By pre-distorting to generate a global alignment feature map and gradually fine-tuning the matrix, the optimal homography matrix is finally obtained, significantly improving the global registration accuracy. The combination of global rigid alignment and non-rigid alignment using the TPS interpolation algorithm achieves accurate global and local registration in tunnel detection parallax scenarios, effectively solving the problems of local image deformation and pixel offset. The registration effect is adapted to the complex requirements of parallax scenarios. Based on the content mask artifact hiding method, through linear fusion and pixel weighting, artifacts such as ghosting, brightness inconsistency, and edge blurring during the stitching process are effectively eliminated, and the stitched image has no obvious stitching seam. Therefore, this application has good adaptability to scenarios with different parallax levels, can adapt to tunnel spaces with different diameters and shapes, and scenarios with abrupt changes in tunnel cross sections, greatly improving the stitching quality of lining images and facilitating the output of wide-view, high-resolution naturally stitched panoramic images. Attached Figure Description
[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:
[0021] Figure 1 This is a flowchart of a multi-scale lining image stitching method in a tunnel detection parallax scenario according to an embodiment of the present invention;
[0022] Figure 2 This is a flowchart of step S100 in a multi-scale lining image stitching method for tunnel detection parallax scenario according to an embodiment of the present invention;
[0023] Figure 3 This is a flowchart of step S200 in a multi-scale lining image stitching method for tunnel detection parallax scenario according to an embodiment of the present invention;
[0024] Figure 4 This is a flowchart of step S300 in a multi-scale lining image stitching method for tunnel detection parallax scenario according to Embodiment 1 of the present invention;
[0025] Figure 5 This is a flowchart of step S400 in a multi-scale lining image stitching method for tunnel detection parallax scenario according to Embodiment 1 of the present invention;
[0026] Figure 6 This is a flowchart of step S500 in a multi-scale lining image stitching method for tunnel detection parallax scenario according to Embodiment 1 of the present invention;
[0027] Figure 7 This is a flowchart of step S600 in a multi-scale lining image stitching method for tunnel detection parallax scenarios according to an embodiment of the present invention. Detailed Implementation
[0028] To make the objectives, technical solutions, and advantages of the present invention clearer, various exemplary embodiments described below will be referenced to the accompanying drawings, which form part of the exemplary embodiments, illustrating various exemplary embodiments that may be used to implement the present invention. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. It should be understood that they are merely examples of processes, methods, and apparatuses consistent with some aspects of the present invention disclosed as detailed in the appended claims, and other embodiments may be used, or structural and functional modifications may be made to the embodiments listed herein without departing from the scope and spirit of the present invention.
[0029] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," etc., indicate the orientation or positional relationship based on the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the referred element must have a specific orientation, or be constructed and operated in a specific orientation. The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. The term "multiple" means two or more. The terms "connected" and "linked" should be interpreted broadly, for example, they can refer to fixed connections, detachable connections, integral connections, mechanical connections, electrical connections, communication connections, direct connections, indirect connections through an intermediate medium, and can refer to the internal communication of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more of the related listed items. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0030] To illustrate the technical solution described in this invention, specific embodiments are described below, showing only the parts related to the embodiments of this invention.
[0031] Example 1:
[0032] like Figure 1As shown, this invention provides a multi-scale lining image stitching method for tunnel detection parallax scenarios, including the following steps: S100: Input two images, a first image and a second image, with overlapping regions, as the tunnel lining images to be stitched in the parallax scenario. Perform simultaneous multi-scale feature extraction and feature map matching on the first and second images. Obtain an initial global homography matrix based on the coordinates of the matched feature points; S200: Perform multi-scale matrix iterative optimization and global alignment feature map generation based on the initial global homography matrix to obtain the optimal global homography matrix; S300: Apply the optimal global homography matrix to the original image of the second image for global rigid alignment, that is, find an optimal rigid transformation (rotation + translation) to make the two sets of data as geometrically coincident as possible to obtain the optimal global homography matrix. The second image after local alignment is locally non-rigidly adjusted using the TPS interpolation algorithm. The TPS interpolation algorithm allows for local bending, stretching, and distortion of the image while maintaining the smoothness of the transformation, thus achieving local non-rigid adjustment, and outputting the registered first and second images; S400: The mask boundaries of the registered first and second images are extracted to obtain the initial mask, and a seam mask is generated; S500: The optimal seam is found within the seam mask, and the optimal seam is expanded at the pixel level. The expanded seam mask is bitwise ANDed with the initial mask to generate a content mask; S600: The registered first and second images are fused based on the content mask to hide artifacts in the stitching and output a panoramic image in the tunnel detection parallax scene.In this embodiment, multi-scale feature extraction and alignment from coarse to fine scales achieves cross-scale feature matching between two lining images in tunnel detection parallax scenarios. This addresses the problems of inaccurate multi-scale feature matching and coarse-to-fine alignment failure caused by large parallax and monotonous texture in highway tunnel detection images, thus achieving stable feature alignment of tunnel lining sequence images. By pre-distorting to generate a global alignment feature map and progressively fine-tuning the matrix, the final optimal homography matrix is obtained, significantly improving global registration accuracy. This solves the problem of registration error accumulation and low global alignment accuracy in long sequence images of highway tunnels when using only a single global homography matrix without multi-scale iterative optimization, improving the overall consistency of the panoramic image. The combination of global rigid alignment and non-rigid alignment using the TPS interpolation algorithm achieves global + local accuracy in tunnel detection parallax scenarios. Registration addresses issues such as localized deformation, lens distortion, insufficient rigid alignment due to road surface undulations, and misalignment / ghosting in damaged areas in highway tunnel images. It achieves high-precision, non-rigid, fine registration, with results adaptable to the complex requirements of parallax scenarios. Based on a content masking artifact hiding method, through linear fusion and pixel weighting, it effectively eliminates artifacts such as ghosting, inconsistent brightness, and blurred edges during the stitching process. The stitched image has no obvious seams, ensuring a clear and usable panoramic view that meets the accuracy requirements for subsequent defect detection, segmentation, and measurement. Therefore, this embodiment demonstrates good adaptability to scenarios with varying parallax levels, adapting to tunnel spaces of different diameters and shapes, as well as scenarios with abrupt changes in tunnel cross-sections. The stitching quality of the lining image is significantly improved, facilitating the output of wide-view, high-resolution, naturally stitched panoramic images.
[0033] As an optional implementation method, such as Figure 2As shown, step S100 specifically includes: S110: performing grayscale normalization (mapping the pixel values of the image to a specific range, usually [0, 1] or [0, 255], to eliminate the effects of uneven illumination, contrast differences, or different sensor gains, thereby accelerating model convergence or improving the robustness of the image analysis algorithm) and pixel value normalization (linearly mapping the image pixel values to [0, 1], thereby eliminating the influence of dimensions, accelerating model convergence, and improving numerical stability) on the input first image and second image. The sizes of the first image and second image are selected as needed and are not specifically limited. S120: A Feature Pyramid Network (FPN) based on a Convolutional Neural Network (CNN) is used to synchronously extract features from the processed first and second images by constructing an additional top-down path and lateral connections to build a feature pyramid with strong semantic information at all scales at extremely low computational cost. In this embodiment, the Feature Pyramid Network (FPN) is replaced with an improved PANet / HRNet, which also achieves multi-scale feature extraction from coarse to fine and cross-image feature alignment. There are only slight differences in the speed and number of parameters of feature extraction, which do not affect the feature matching and alignment accuracy. Multi-scale feature extraction facilitates the simultaneous identification and processing of targets of different sizes in the first and second images, generating multiple feature maps of different scales. Cross-image feature matching is performed on each feature map (cosine similarity matching is preferred in this embodiment), thereby establishing a pixel-level or region-level correspondence between the first and second images. Alignment of feature maps at various scales in the first and second images is then performed, transforming all feature maps involved in the fusion to the same spatial size and channel dimension for information fusion. The aligned feature pyramid is then output. Through spatial alignment and channel alignment, the extracted original multi-scale features are transformed into a series of semantically rich new feature layers with different resolutions.S130: An improved RANSAC (Random Sample Consensus) algorithm is used to perform feature point matching and outlier removal on the coarsest-scale aligned feature map (1 / 32 scale feature map in this embodiment). The improved RANSAC algorithm is used to address the shortcomings of the traditional RANSAC algorithm. Specific improvements include guided sampling strategy to improve convergence speed, local optimization strategy to improve accuracy, and multi-model fitting strategy to handle complex scenarios. In the coarsest-scale aligned feature map, image details are blurred, but the global semantic context and large-scale geometric structure are the clearest, which facilitates the establishment of global correspondence and the significant removal of outliers. Outlier removal eliminates noise points caused by feature mismatch. In this embodiment, the improved RANSAC algorithm can also be replaced by the MAGSAC++ algorithm or the MSAC algorithm. Both are robust feature point matching and outlier removal algorithms that can achieve the same initial homography matrix estimation accuracy and meet the basic requirements of multi-scale iterative optimization. Based on the coordinates of the matched feature points, an initial global homography matrix (3×3 matrix) is obtained by solving the projection transformation formula. That is, using known matching point pairs, a 3×3 projection transformation matrix is solved so that points in one image coincide with corresponding points in another image after transformation. The initial global homography matrix provides a coarse but topologically correct geometric transformation model, mapping points in the source image to the target image or global coordinate system, thus initially aligning the two images geometrically. In this embodiment, the feature pyramid network contains four layers of feature maps, from coarse to fine: a 1 / 32 scale feature map (maximum receptive field, covering the entire object or even the scene, achieving the coarsest-grained matching), a 1 / 16 scale feature map (maximum receptive field, covering the main part of the object, achieving coarse-to-medium-grained matching), a 1 / 8 scale feature map (medium receptive field, local texture blocks, achieving standard feature matching), and a 1 / 4 scale feature map (minimal receptive field, detailed textures, achieving fine-grained matching). This allows for both clear global contours (low resolution, high semantics) and recognition of local details (high resolution, low semantics). Multi-scale feature extraction and alignment, from coarse to fine, enables cross-scale feature matching between two images in tunnel detection parallax scenarios. Compared with independent feature extraction, the accuracy of feature matching is improved by 40%, providing a high-precision feature foundation for subsequent matrix estimation.
[0034] As an optional implementation method, such as Figure 3As shown, step S200 specifically includes: S210: Pre-distorting (projection transformation) the 1 / 16 scale feature map of the second image using the initial global homography matrix to generate a preliminary globally aligned feature map; S220: Performing secondary feature matching between the pre-distorted 1 / 16 scale feature map and the 1 / 16 scale feature map of the first image, and fine-tuning the initial global homography matrix based on the matching result to obtain an optimized global homography matrix; S230: Repeating the above steps sequentially for the 1 / 8 scale feature map and the 1 / 4 scale feature map, pre-distorting the corresponding scale feature map of the second image based on the optimized matrix of the previous scale (the order of the 1 / 16 scale feature map, the 1 / 8 scale feature map, and the 1 / 4 scale feature map) to generate a globally aligned feature map, and then fine-tuning the matrix through feature matching to finally obtain a highly accurate 3×3 transformation matrix, which is the optimal global homography matrix in the tunnel detection parallax scenario. The iterative optimization method for multi-scale homography matrices reduces the projection transformation error of the final optimal homography matrix by 55% through pre-distorting to generate a global alignment feature map and gradually fine-tuning the matrix, thereby controlling the pixel deviation of global registration to within 3 pixels and significantly improving the global registration accuracy.
[0035] As an optional implementation method, such as Figure 4 As shown, step S300 specifically includes: S310: uniformly selecting 100 feature control points in the overlapping area of the first image and the second image, or other numbers of feature control points as needed; S320: calculating the pixel offset between the control points of the second image and the corresponding control points of the first image after global alignment; S330: fitting the pixel offset using the TPS interpolation algorithm. The TPS interpolation algorithm not only ensures accurate matching at the control points, but also ensures that the regional changes between control points are continuous and smooth, avoiding wrinkles or unnatural abrupt changes that may be generated by local deformation algorithms. The TPS interpolation algorithm is convenient for local bending, stretching and twisting operations, while maintaining the smoothness of the transformation, generating a local twist transformation field. The local twist transformation field is highly flexible and can adapt to arbitrarily complex geometric deformations. In this embodiment, the TPS interpolation algorithm can also be replaced by Bézier curve interpolation or radial basis function interpolation algorithm, which can also achieve local non-rigid adjustment of multi-point twisting, with similar local registration effects in tunnel detection parallax scenarios, differing only in the complexity of interpolation calculation. S340: Applying a local distortion transform field to the globally rigidly aligned second image for local non-rigid alignment, completing the global and local registration of the first and second images, and outputting the registered first and second images. The combination of global rigid alignment and TPS interpolation algorithm non-rigid alignment achieves accurate global and local registration in tunnel detection parallax scenarios, effectively solving the problems of local image deformation and pixel offset. The ghosting elimination rate in the stitching area reaches over 90%, and the registration effect is adapted to the complex requirements of parallax scenarios.
[0036] As an optional implementation method, such as Figure 5 As shown, step S400 specifically includes: S410: Binarizing the two registered images respectively, with the overlapping area as the foreground and the non-overlapping area as the background, to generate an initial mask. S420: Extracting the mask boundaries of the initial mask using the Canny edge detection algorithm to obtain the mask boundary contours of the two images. The Canny edge detection algorithm finds the best balance between low error rate, high positioning accuracy, and single-pixel response, facilitating accurate extraction of mask boundary contours. In this embodiment, the Canny edge detection algorithm can also be replaced by the Sobel operator or the Laplacian operator, both of which can achieve the same mask boundary extraction effect, meeting the requirements for seam mask generation. S430: Taking the mask boundary contours of the intersecting area of the two images to generate a seam mask that marks the core seam area of the stitching. The seam refers to the boundary line used for segmentation in the overlapping area of the two images. The core seam area usually refers to a strip-shaped neighborhood around this optimal stitching line. The seam mask is used to clearly indicate which input source image each pixel in the panoramic image originates from, and is used to adapt the natural boundaries of the image content. Based on Canny edge detection, mask boundary extraction and intersecting region seam mask generation are achieved, enabling adaptive acquisition of seam lines. Compared to fixed seam lines, the matching accuracy between the seam mask and image content is improved by 60%, avoiding the limitations of manually setting seam lines. This solves the problems of mismatch between traditional fixed seam lines and tunnel lining structures, and the cutting of defective areas by splicing seams, by adaptively generating seam masks that fit the lining structure based on mask boundaries.
[0037] As an optional implementation method, such as Figure 6As shown, step S500 specifically includes: S510: Based on the seam mask, with the minimum pixel grayscale difference and gradient difference of the splicing area as the objective function, a dynamic programming algorithm is used to find the optimal seam line within the seam mask. The optimal seam line extends along the natural edge of the image content, avoiding cutting the image target. In this embodiment, the dynamic programming algorithm can be replaced by a greedy algorithm or a graph cut algorithm, with the minimum pixel grayscale difference and gradient difference as the objective, which can also find the optimal seam line matching the image content and achieve the same seam optimization effect. S520: The optimal seam line is expanded to both sides at the pixel level. In this embodiment, it is preferred to expand to both sides by 5-10 pixels to generate an expanded seam mask, improving the smoothness of the fusion area. S530: The expanded seam mask is bitwise ANDed with the initial mask to remove invalid areas in the mask and generate the final content mask. The content mask accurately marks the fusion area and the reserved area of the spliced image. The optimal seam found by the dynamic programming algorithm extends along the natural edge of the image, effectively avoiding cutting the image target and reducing the visual abruptness of the seam by 70%. The content mask generated by seam expansion and bitwise AND operation accurately marks the fusion area, improving the feature protection rate by 50%. This solves the problems of defect breakage, abrupt edges, and inaccurate measurement after splicing caused by unoptimized seams and lack of content mask protection, achieving smooth seams and region fidelity.
[0038] As an optional implementation method, such as Figure 7 As shown, in step S600, the fusion processing of the registered first and second images based on the content mask to hide artifacts during the stitching process includes: S610: fusing the registered first and second images based on the content mask to hide artifacts during stitching; S620: using a linear fusion algorithm to smooth the pixel grayscale transition in the fusion area marked by the content mask to solve the problem of inconsistent brightness; in this embodiment, the linear fusion algorithm can be replaced by Poisson fusion or multi-resolution fusion, combined with the content mask for fusion processing. Poisson fusion can achieve a smooth transition of image gradients, and multi-resolution fusion can achieve layer-by-layer fusion at different scales, eliminating stitching artifacts and improving image naturalness. S630: for the retained areas outside the fusion area, the pixel information of the original image is directly retained, and the ghosting and edge blurring generated during the stitching process are weakened and hidden by pixel weighting of the content mask. Based on the content masking artifact hiding method, through linear fusion and pixel weighting, artifacts such as ghosting, inconsistent brightness, and blurred edges in the stitching process are effectively eliminated. The stitched image has no obvious stitching seams and the naturalness is improved by 80%.
[0039] As an optional implementation, step S600, outputting a panoramic image in the tunnel detection parallax scenario specifically includes: stitching and cropping the edges of the fused image, eliminating the black borders of the stitched image, and outputting a wide-view, high-resolution (set according to the usage scenario) naturally stitched panoramic image through the panoramic image output module, thereby completing the multi-scale lining image stitching in the tunnel detection parallax scenario and outputting a panoramic image in the tunnel detection parallax scenario.
[0040] The embodiment is merely a specific example and does not indicate that this is the only way to implement the present invention.
[0041] Example 2:
[0042] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of a multi-scale lining image stitching method for tunnel detection parallax scenarios as described in Embodiment 1. It includes an image input module, a multi-scale feature extraction module, a homography matrix estimation and optimization module, a non-rigid alignment module, a mask processing module, an artifact hiding module, and a panoramic image output module, all sequentially electrically connected and transmitting data. The image input module acquires and inputs a first image and a second image with overlapping regions through a tunnel appearance image acquisition device. The appearance image acquisition device is an automated device used to acquire visual information such as the external shape, surface texture, color, and three-dimensional structure of an object. By integrating a camera, light source, motion control, and data processing unit, it aims to achieve high-precision and high-efficiency acquisition of highway tunnel image data. The first image and the second image are tunnel lining images to be stitched. The multi-scale feature extraction module uses a feature pyramid network to extract features from the first image and the second image. Multi-scale feature extraction is performed on the two images; the homography matrix estimation and optimization module performs feature point matching and outlier removal on the coarsest-scale aligned feature map, and obtains the initial global homography matrix by solving the projection transformation formula; the non-rigid alignment module applies the optimal global homography matrix to the original image of the second image for global rigid alignment, and performs local non-rigid alignment on the globally rigidly aligned second image by using the TPS interpolation algorithm; the mask processing module extracts the mask boundaries of the registered first and second images to obtain the initial mask, and generates the seam mask; the artifact hiding module performs fusion processing on the registered first and second images based on the content mask to hide artifacts in the stitching; the panoramic image output module outputs the panoramic image under the tunnel detection parallax scene.
[0043] The above description is merely a preferred embodiment of the present invention. Those skilled in the art will understand that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.
Claims
1. A method for multi-scale lining image stitching in tunnel detection parallax scenarios, characterized in that, Includes the following steps: S100: Input two images, the first and the second, which have overlapping areas, as images of the tunnel lining to be stitched in the parallax scene. Use a feature pyramid network based on a convolutional neural network (CNN) to extract features from the processed first and second images simultaneously at multiple scales to generate feature maps of different scales. Perform cross-image feature matching on each feature map and obtain the initial global homography matrix based on the coordinates of the matched feature points. S200: Based on the initial global homography matrix, perform multi-scale matrix iterative optimization and global alignment feature map generation to obtain the optimal global homography matrix; S300: Apply the optimal global homography matrix to the original image of the second image for global rigid alignment to obtain the globally rigid aligned second image. Then, perform local non-rigid alignment on the globally rigid aligned second image using the TPS interpolation algorithm, and output the registered first and second images. S400: Extract the mask boundaries from the registered first and second images to obtain the initial mask, and generate the seam mask; S500: Find the optimal seam within the seam mask, expand the optimal seam pixel by pixel, and perform a bitwise AND operation between the expanded seam mask and the initial mask to generate the content mask. S600: Based on the content mask, the first and second images after registration are fused to hide artifacts in the stitching and output a panoramic image in the tunnel detection parallax scene.
2. The method for multi-scale lining image stitching in a tunnel detection parallax scenario according to claim 1, characterized in that, The S100 steps specifically include: S110: Perform grayscale normalization and pixel value normalization on the input first and second images; S120: Align the feature maps of the first and second images at each scale, and output the aligned feature pyramid; S130: The improved RANSAC algorithm is used to match feature points and remove outliers in the coarsest-scale aligned feature map. The initial global homography matrix is obtained by solving the projection transformation formula based on the coordinates of the matched feature points.
3. The method for multi-scale lining image stitching in a tunnel detection parallax scenario according to claim 2, characterized in that, The feature pyramid network contains four layers of feature maps, ranging from coarse to fine: 1 / 32 scale feature map, 1 / 16 scale feature map, 1 / 8 scale feature map, and 1 / 4 scale feature map.
4. The multi-scale lining image stitching method for tunnel detection parallax scenarios according to claim 3, characterized in that, The S200 steps specifically include: S210: The 1 / 16 scale feature map of the second image is pre-distorted using the initial global homography matrix to generate a preliminary globally aligned feature map; S220: Perform secondary feature matching between the pre-distorted 1 / 16 scale feature map and the 1 / 16 scale feature map of the first image, and fine-tune the initial global homography matrix based on the matching result to obtain the optimized global homography matrix; S230: Repeat the above steps for the 1 / 8 scale feature map and the 1 / 4 scale feature map in sequence. Based on the optimization matrix of the previous scale, pre-distort the corresponding scale feature map of the second image to generate a globally aligned feature map. Then, fine-tune the matrix through feature matching to finally obtain the optimal global homography matrix in the tunnel detection parallax scenario.
5. The method for multi-scale lining image stitching in a tunnel detection parallax scenario according to claim 1, characterized in that, In step S300, the globally rigidly aligned second image is locally non-rigidly adjusted using the TPS interpolation algorithm, and the registered first and second images are output. Specifically, these include: S310: Select 100 feature control points evenly in the overlapping area of the first image and the second image; S320: Calculate the pixel offset between the control points of the second image and the corresponding control points of the first image after global alignment; S330: The pixel offset is fitted by the TPS interpolation algorithm to generate a local distortion transformation field; S340: Apply the local distortion transform field to the globally rigidly aligned second image to perform local non-rigid alignment, complete the global and local registration of the first and second images, and output the registered first and second images.
6. The method for multi-scale lining image stitching in a tunnel detection parallax scenario according to claim 1, characterized in that, The S400 steps specifically include: S410: Perform binarization processing on the two registered images, with the overlapping area as the foreground and the non-overlapping area as the background, to generate an initial mask; S420: Extract the mask boundaries of the initial mask using the Canny edge detection algorithm to obtain the mask boundary contours of the two images; S430: Take the mask boundary contour of the intersection area of two images and generate a seam mask that marks the core seam area of the stitching, which is used to adapt to the natural boundaries of the image content.
7. The method for multi-scale lining image stitching in a tunnel detection parallax scenario according to claim 1, characterized in that, The S500 steps specifically include: S510: Based on the seam mask, with the objective function of minimizing the pixel grayscale difference and gradient difference in the splicing area, a dynamic programming algorithm is used to find the optimal seam line within the seam mask. The optimal seam line extends along the natural edge of the image content. S520: The optimal suture is expanded pixel-level to both sides to generate the expanded suture mask; S530: Perform a bitwise AND operation between the expanded suture mask and the initial mask to remove invalid regions in the mask and generate the final content mask.
8. The method for multi-scale lining image stitching in a tunnel detection parallax scenario according to claim 1, characterized in that, In step S600, the first and second images registered based on the content mask are fused to hide artifacts during the stitching process, including: S610: Based on the content mask, the registered first and second images are fused to hide artifacts in the stitching; S620: For the fusion area marked by the content mask, a linear fusion algorithm is used to perform smooth transition of pixel grayscale; S630: For the areas outside the fusion region, the pixel information of the original image is directly retained. Ghosting and edge blurring caused during the stitching process are weakened and hidden by pixel weighting of the content mask.
9. The method for multi-scale lining image stitching in a tunnel detection parallax scenario according to claim 1, characterized in that, In step S600, outputting a panoramic image in the tunnel detection parallax scenario specifically includes: stitching and cropping the edges of the fused image, eliminating the black borders of the stitched image, completing the multi-scale lining image stitching in the tunnel detection parallax scenario, and outputting a panoramic image in the tunnel detection parallax scenario.
10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the multi-scale lining image stitching method in the parallax detection scenario of any one of claims 1-9, including an image input module that is electrically connected in sequence and performs data transmission, a multi-scale feature extraction module, a homography matrix estimation and optimization module, a non-rigid alignment module, a mask processing module, an artifact hiding module, and a panoramic image output module. The image input module acquires and inputs a first image and a second image with overlapping regions through a tunnel appearance image acquisition device. The first and second images are tunnel lining images to be stitched together. The multi-scale feature extraction module uses a feature pyramid network to extract multi-scale features from the first and second images. The homography matrix estimation and optimization module performs feature point matching and outlier removal on the coarsest-scale aligned feature map, and obtains the initial global homography matrix by solving the projection transformation formula. The non-rigid alignment module applies the optimal global homography matrix to the original image of the second image for global rigid alignment, and performs local non-rigid alignment on the globally rigidly aligned second image using the TPS interpolation algorithm. The mask processing module is used to generate seam masks. The artifact hiding module fuses the registered first and second images based on the content mask to hide artifacts in the stitching. The panoramic image output module outputs a panoramic image under the tunnel detection parallax scene.