An image stitching method, an electronic device, a storage medium and a program product

Abstract

The embodiment of the application provides an image splicing method, an electronic device, a storage medium and a program product, through structure type identification, non-uniform overlap ratio dynamic estimation, global light joint correction, multi-frequency band differentiation fusion and an encoder assisted longitudinal positioning, the tunnel sections of two different structure types of shield and non-shield can be adaptively processed, complex working conditions that the overlap ratios of each linear array camera are different and the light conditions are significantly different are processed, and high-quality tunnel inspection images covering the full section and full length and without obvious misplacement and light change boundaries are output, so that reliable image data is provided for tunnel disease detection.

Description

Technical Field

[0001] This application relates to the field of tunnel detection technology, and more specifically, to an image stitching method, electronic device, storage medium, and program product. Background Technology

[0002] With the rapid development of transportation infrastructure construction in my country, the number of underground projects such as urban subways and highway tunnels is increasing daily. During long-term operation, tunnel linings are prone to defects such as cracks, water leakage, spalling, and exposed rebar due to factors such as geological conditions, groundwater seepage, vehicle loads, and environmental erosion. These defects seriously threaten traffic safety and structural stability. Therefore, regular, full-coverage, high-precision inspections of tunnel lining surfaces are of great significance.

[0003] In related technologies, manual inspection methods are inefficient, dangerous, have limited coverage, and are highly subjective. In recent years, image acquisition-based tunnel inspection systems have gradually replaced manual inspections. Linear scan cameras (i.e., line-scan cameras) have advantages such as high resolution, high line frequency, and suitability for continuous motion acquisition, and are widely used for image acquisition of tunnel lining surfaces. In actual engineering, multiple line scan cameras are usually arranged along the circumferential direction of the tunnel (i.e., the cross-sectional circumference), and a vehicle is used to travel at a constant speed along the longitudinal direction of the tunnel (i.e., the tunnel extension direction, also known as the horizontal direction) to achieve image coverage of the entire tunnel cross-section.

[0004] For U-shaped tunnel cross-sections, due to the irregular cross-sectional shape (flat bottom, upward-curving sides, and arched top), the shooting angles and distances from the lining surface of each camera differ when the nine line-scan cameras are arranged circumferentially. This results in inconsistent circumferential overlap ratios between adjacent cameras, with each overlap ratio being different. Simultaneously, the complex lighting conditions within the tunnel, with variations in the illumination angle and intensity at each camera's location, coupled with the uneven reflectivity of the tunnel lining surface, lead to significant differences in brightness, contrast, and hue in the images captured by each camera. Furthermore, the tunnel structure employs both shield tunneling and non-shield tunneling methods (such as mining and cut-and-cover methods). Shield tunnels feature segmented, regularly curved lining structures, while non-shield tunnels have relatively rough and unevenly textured lining surfaces. These two structures exhibit significant differences in cross-sectional shape, surface reflectivity, and illumination distribution, resulting in varying overlap ratios and illumination conditions across different tunnel sections.

[0005] In relevant tunnel image stitching techniques, a combination of feature point matching and weighted fusion is commonly used. Specifically, firstly, SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust Features) feature points are extracted from the overlapping regions of adjacent camera images. These points are then matched using the Nearest Neighbor Distance Ratio (NNDR) method. Next, the RANSAC (Random Sample Consensus) algorithm is used to eliminate mismatched points, and geometric transformation parameters are estimated. Finally, stitching is completed through weighted averaging or linear fusion. Some schemes introduce illumination preprocessing such as histogram equalization or gamma correction before stitching to mitigate illumination differences.

[0006] However, the relevant tunnel image stitching technology cannot solve the problem of non-uniform circumferential overlap ratio under U-shaped cross-section, cannot eliminate the stitching seam caused by the difference in lighting of multiple cameras, cannot adapt to the structural differences between shield and non-shield tunnels, cannot simultaneously take into account the combined working condition of non-uniform circumferential overlap + variable lighting + dual structure type + horizontal non-overlap, and cannot solve the problem of stable stitching under the condition of no overlap in the horizontal direction (i.e., longitudinal direction). Summary of the Invention

[0007] The purpose of this application is to provide an image stitching method, electronic device, storage medium, and program product to achieve the technical effect of generating high-quality tunnel inspection images that cover the entire cross-section and length without obvious misalignment or boundary changes in illumination.

[0008] A first aspect of this application provides an image stitching method, the method comprising: Acquire tunnel lining images and encoder output pulse data simultaneously from multiple linear array cameras arranged circumferentially along the tunnel. Based on the pulse data, calculate the longitudinal physical location of the tunnel corresponding to each row of the tunnel lining image; The tunnel lining image is used to identify the tunnel structure type and determine the tunnel structure type to which the current image segment belongs; Based on the tunnel structure type, the overlap ratio between circumferentially adjacent line scan cameras is determined; Based on the overlap ratio, the overlapping area between the images acquired by adjacent circumferential line scan cameras is determined; The overlapping areas are image registered, and the images acquired by each line scan camera are aligned to a unified global coordinate system to obtain an aligned image; Based on the tunnel structure type, determine the lighting parameter set corresponding to the tunnel structure type; The aligned image is then illuminated using the illumination parameter set to obtain the corrected image. The corrected image is fused in multiple frequency bands within the overlapping area to obtain a seamless circumferential stitched image. Based on the pulse data and the longitudinal physical location of the tunnel, the multiple segments of the seamless circumferential image collected along the longitudinal direction of the tunnel are longitudinally stitched together according to their physical locations to obtain a complete tunnel inspection image.

[0009] In the above implementation process, through structural type identification, non-uniform overlap ratio dynamic estimation, global illumination joint correction, multi-band differential fusion, and encoder-assisted longitudinal positioning, it is possible to adapt to tunnel cross sections with two different structural types, shield tunneling and non-shield tunneling, and handle complex working conditions with different overlap ratios of linear array cameras in the circumferential direction and significant differences in illumination conditions. It outputs high-quality tunnel inspection images that cover the entire cross section and length without obvious misalignment and illumination change boundaries, providing reliable image data for tunnel defect detection.

[0010] Further, the step of identifying the tunnel structure type of the tunnel lining image and determining the tunnel structure type of the current image segment includes: Multiple image features are extracted from the current image segment and concatenated into a feature vector. The current feature vector is then normalized using preset normalization parameters. These multiple image features include gray-level statistical features, local texture features, gray-level co-occurrence matrix texture features, and frequency domain features. The gray-level statistical features include gray-level mean, standard deviation, skewness, and kurtosis. The local texture features are extracted using a local binary mode, with a radius of a first preset value and a sampling number of a second preset value. The gray-level co-occurrence matrix texture features are used to calculate contrast, energy, homogeneity, and correlation in different directions. The frequency domain features are used to extract the energy distribution and dominant frequency information of each frequency band after performing a Fourier transform on the current image segment. The normalized feature vector is input into the classifier model to obtain the structure type label of the current image segment. The classifier model is trained through the following steps: acquiring tunnel lining image samples with known structure types and labeling them with structure type labels; extracting multi-class image features from each tunnel lining image sample and concatenating them into a feature vector; normalizing the feature vectors of all samples in the training set; using the normalized feature vector as input and the labeled structure type label as output to train the initial classifier model, thereby obtaining the trained classifier model and normalization parameters.

[0011] In the above implementation process, feature vectors were constructed from four dimensions: gray-level distribution, local texture, gray-level co-occurrence relationship, and frequency domain characteristics. These features comprehensively characterize the differences between the lining surfaces of shield tunnels and non-shield tunnels from different perspectives, ensuring that the classifier model can effectively distinguish between the two structural types. The feature extraction parameters are consistent between the training and inference phases, and normalized parameters are reused to guarantee the classification accuracy and stability of the classifier model during online inference.

[0012] Further, determining the overlap ratio between circumferentially adjacent linear scan cameras based on the tunnel structure type includes: Based on the installation calibration parameters of each linear array camera and the tunnel cross-sectional profile data, the theoretical overlap ratio of each adjacent linear array camera pair is calculated by geometric projection. Within the neighborhood of the theoretical overlap ratio, the edge stripes of adjacent linear scan camera images are coarsely located to obtain the initial overlap width; Traverse candidate overlap widths around the initial overlap width and calculate the normalized cross-correlation value of the overlapping region of adjacent linear array cameras under each candidate overlap width; The overlap width corresponding to the maximum normalized cross-correlation value is taken as the target overlap width, and the overlap ratio is calculated based on the ratio of the target overlap width to the image width.

[0013] In the above implementation process, accurate estimation of the overlap ratio of adjacent linear array cameras in the circumferential direction under the U-shaped tunnel cross-section was achieved, solving the shortcomings of related technologies that cannot adapt to non-uniform overlap distributions based on fixed overlap ratio assumptions or unified models. Specifically, geometric coarse estimation limits the search range to the vicinity of the theoretical value, which improves search efficiency; coarse positioning provides a pixel-level initial estimate within the neighborhood of the theoretical value, which narrows the search range of fine matching; normalized cross-correlation fine matching performs a pixel-by-pixel fine search near the initial value, and its normalized cross-correlation value is robust to linear changes in grayscale, providing accurate matching results under inconsistent lighting conditions. The combination of these three methods keeps the overlap ratio estimation error within a small range, providing accurate overlap region boundaries for subsequent image registration and eliminating splicing misalignment and ghosting problems caused by overlap ratio estimation deviations.

[0014] Further, the step of image registration of the overlapping region, aligning the images acquired by each line scan camera to a unified global coordinate system to obtain an aligned image, includes: Using the line scan camera at the arch position as a reference, the transformation matrix of the line scan camera at the arch position is used as the identity matrix. For each line scan camera on one side of the line scan camera at the arch position, the global transformation matrix of each line scan camera is obtained by multiplying it by the inverse of the transformation matrix of the adjacent line scan camera in turn. For each line scan camera on the other side of the line scan camera at the arch position, the global transformation matrix of each line scan camera is obtained by multiplying the transformation matrix of the adjacent line scan cameras by the right in sequence. Based on the global transformation matrix of each line scan camera, the images acquired by each line scan camera are transformed into the global coordinate system to obtain the aligned image.

[0015] In the above implementation process, a global coordinate system was established using the linear array camera at the arch location as a reference. A strategy of transferring transformation matrices from both sides was employed to achieve high-precision spatial alignment of multi-camera images. The arch linear array camera has the most accurate viewpoint and minimal distortion. Using this camera as a reference, the registration error was propagated symmetrically to both sides, effectively avoiding the problem of excessive cumulative error on one side. Left and right multiplication operations were used on both sides respectively to ensure the correct synthesis of coordinate transformations in different circumferential directions.

[0016] Further, the step of performing illumination correction on the aligned image using the illumination parameter set to obtain the corrected image includes: The alignment images of each line scan camera are corrected using a correction model. The pixel values ​​of the corrected image of each line scan camera are obtained by linearly transforming the pixel values ​​of the alignment image based on the gain coefficient and bias coefficient of the line scan camera. Using the line scan camera at the arch position as a reference, the gain coefficient and offset coefficient of the line scan camera at the arch position are fixed. The optimization objective is to minimize the sum of squared grayscale differences of all adjacent linear scan camera pairs after correction in the overlapping region. Under the constraint of gain coefficient normalization, the gain coefficient and bias coefficient of all linear scan cameras are solved jointly. The aligned images of each line scan camera are corrected based on the obtained gain coefficients and bias coefficients of each line scan camera to obtain the corrected images.

[0017] In the above implementation process, the dome-shaped linear array camera is used as a reference benchmark to ensure that the correction direction is clear and the image with the best quality is used as a reference. The objective function is to minimize the sum of squared gray-level differences after correction of all adjacent linear array cameras in the overlapping area, thus elevating the illumination correction from independent single-point processing to global consistency optimization. The introduction of gain normalization constraints ensures the uniqueness and numerical stability of the solution to the optimization problem. After the above correction, the gray-level jump amplitude between the circumferential linear array cameras is significantly reduced, and the circumferential gray-level consistency is improved, providing a uniformly distributed input image for subsequent multi-band fusion and eliminating visual stitching seams caused by illumination differences.

[0018] Further, the step of performing multi-band fusion of the corrected image within the overlapping region to obtain a seamless circumferential stitched image includes: The corrected images from each linear array camera are used to construct multi-layer pyramids, and each corrected image is decomposed into components of different frequency bands. For low-frequency components, a fusion weight is determined within the overlapping region based on the distance from each pixel position to the center position of the overlapping region. The fusion weight is then used to perform weighted fusion of the pixel values ​​of the corrected images of two adjacent line scan cameras within the overlapping region. For high-frequency components, the gradient energy in the neighborhood of each pixel is calculated within the overlapping area, and the pixel value of the corrected image on the side with larger gradient energy is determined as the fusion result at the pixel location. The fused frequency band components are reconstructed layer by layer to obtain the seamless circumferential stitched image.

[0019] In the above implementation process, the contradiction between eliminating stitching seams and maintaining sharpness in the stitched image was resolved through multi-band decomposition and differential fusion. Low-frequency components employ distance-based weighted fusion, ensuring a smooth transition of overall illumination within the overlapping area and effectively eliminating residual illumination jumps. High-frequency components utilize selective fusion based on local gradient energy, preserving information from the side with clearer texture and sharper edges at each pixel location, avoiding high-frequency detail blurring caused by weighted averaging. The reconstructed circumferential stitched image achieves both a visually seamless transition and maximizes the preservation of subtle texture features of the tunnel lining surface.

[0020] Further, based on the pulse data and the longitudinal physical location of the tunnel, the multiple segments of the seamlessly stitched circumferential images acquired along the longitudinal direction of the tunnel are longitudinally stitched together according to their physical locations to obtain a complete tunnel inspection image, including: Based on the pulse data and calibration coefficients, the starting position of each segment of the seamless circumferential stitched image on the vertical canvas is determined. At the vertical seam of two adjacent seamlessly stitched circumferential images, extract the first row of grayscale mean sequence at the end of the previous segment (preset number of rows) and the second row of grayscale mean sequence at the beginning of the next segment (preset number of rows). Calculate the normalized cross-correlation values ​​of the first row grayscale mean sequence and the second row grayscale mean sequence at different offsets; The offset corresponding to the maximum normalized cross-correlation value is used as the vertical offset correction amount; Based on the offset correction amount, the starting row position of the next segment of the circumferential seamless stitched image is finely adjusted; At the longitudinal joint, the preset number of rows at the end of the previous segment and the preset number of rows at the beginning of the next segment are linearly weighted and fused to obtain the complete tunnel inspection image.

[0021] In the above implementation process, encoder position mapping provides preliminary positioning based on physical distance, ensuring that each image segment is arranged in the correct order on the canvas; one-dimensional cross-correlation motion compensation, by comparing the row grayscale distribution on both sides of the seam, finely corrects the positioning error caused by vehicle speed fluctuations based on encoder positioning, eliminating row misalignment; linear gradient weighted fusion eliminates grayscale jumps at the seam through smooth transition, making the stitching result visually natural and continuous. Stable and reliable longitudinal stitching is achieved without relying on image feature matching, ensuring the row alignment accuracy and visual continuity of the complete tunnel inspection image in the longitudinal direction.

[0022] A second aspect of this application provides an electronic device, the electronic device comprising: processor; Memory used to store processor-executable instructions; Wherein, when the processor invokes the executable instructions, it implements any of the methods described in the first aspect.

[0023] A third aspect of this application provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement any of the methods described in the first aspect.

[0024] A fourth aspect of this application provides a computer program product, the computer program product including a computer program, which, when executed by a processor, implements any of the methods described in the first aspect. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A flowchart illustrating an image stitching method provided in an embodiment of this application; Figure 2 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0028] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] In related technologies, tunnel image stitching methods have the following drawbacks: (1) The problem of non-uniform circumferential overlap ratio under U-shaped cross-section has not been solved. For U-shaped tunnel cross-section, when the nine line scan cameras are arranged circumferentially, the distance and shooting angle between different cameras and the tunnel surface are different, resulting in different circumferential overlap ratios between adjacent cameras. Related technologies usually assume a uniform overlap ratio based on a unified model, or use a fixed clipping width for alignment, which cannot adapt to this non-uniform overlap situation, resulting in misalignment and ghosting in the overlapping area after splicing.

[0030] (2) Failure to eliminate splicing seams caused by differences in illumination between multiple cameras. The different installation positions of the cameras, the different angles of auxiliary lighting in the tunnel, the changes in the curvature of the lining surface, and the uneven reflectivity resulted in significant differences in brightness, contrast, and hue in the images acquired by the nine cameras. Related technologies only used simple global preprocessing such as histogram equalization or gamma correction, without jointly optimizing the local grayscale differences in the overlapping areas of adjacent cameras. Direct splicing would produce obvious illumination change boundaries, i.e., splicing seams.

[0031] (3) Differences between non-adaptive shield tunnel and non-shield tunnel structures. Shield tunnel segments are segmented with regular arc surfaces, resulting in smoother surfaces and regular textures; non-shield tunnels (such as shotcrete linings from mining methods) have rougher surfaces and uneven textures. The two structures differ in cross-sectional shape, overlap ratio distribution, and light reflection characteristics. Related technologies generally use fixed parameter models, which cannot adaptively adjust splicing parameters according to the tunnel structure type, leading to a decrease in splicing quality in different structural sections.

[0032] (4) None of the related technologies simultaneously take into account the combined working conditions of circumferential non-uniform overlap + variable illumination + dual structure type + horizontal non-overlap, and have not solved the problem of uneven illumination and non-uniform overlap ratio of multiple cameras; they have not addressed the adaptability of continuous acquisition by line scan cameras and the structural differences between shield and non-shield tunnels, and none of them can meet the seamless splicing requirements of 9 line scan cameras on the entire cross section of a U-shaped tunnel under non-uniform overlap, variable illumination, and dual structure type conditions.

[0033] (5) The problem of stable stitching under non-overlapping conditions in the horizontal direction (i.e., longitudinal direction) has not been solved. When the line scan camera collects data along the longitudinal direction of the tunnel, there is no overlapping area between adjacent data strips, and the position is determined solely by the encoder. The relevant technologies do not adequately compensate for encoder errors and vehicle motion vibrations, which can easily lead to misalignment or breakage in longitudinal stitching.

[0034] In other words, the relevant technologies currently face the following challenges: how to adaptively estimate the varying circumferential overlap ratios among the nine line-scan cameras in a U-shaped tunnel section to achieve accurate registration of circumferential images and overcome stitching misalignment caused by non-uniform overlap; how to eliminate uneven illumination caused by differences in lighting conditions among the nine cameras by using global joint illumination correction and multi-band fusion to achieve seamless boundary transitions in the stitched large image; how to make the stitching algorithm adaptively adaptable to two different structural types, shield tunnels and non-shield tunnels, by establishing parameter models for the overlap ratio distribution and illumination characteristics differences of the two structures to achieve adaptive stitching; and how to achieve stable longitudinal image stitching based on encoder information and motion compensation under conditions of no horizontal (vertical) overlap, eliminating line misalignment and breakage phenomena.

[0035] To address any of the problems mentioned above, embodiments of this application provide an image stitching method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating an image stitching method provided in an embodiment of this application.

[0036] In this embodiment, the method includes: Step S10: Acquire tunnel lining images and encoder output pulse data synchronously acquired by multiple linear array cameras arranged along the tunnel circumference. It should be noted that a line scan camera is a camera that acquires only one line of images at a time, relying on the vehicle mounted on it to move longitudinally along the tunnel to obtain a complete two-dimensional image. Compared with an area scan camera, it has higher resolution, higher line frequency, and is more suitable for continuous motion acquisition.

[0037] Multiple linear array cameras are arranged along the tunnel circumferential direction (i.e., the cross-sectional circumference) to simultaneously acquire image data of the tunnel lining surface.

[0038] Pulse data: Pulse signals generated by the encoder as the wheel rotates. The encoder is mounted on the wheel axle; as the vehicle moves, the wheel rotation drives the encoder to generate pulses, each pulse corresponding to a fixed physical displacement. Pulse data is used to record the distance traveled by the vehicle.

[0039] The simultaneous acquisition by multiple line scan cameras ensures that each line scan camera maintains consistency in time, guaranteeing that the images from each line scan camera at the same cross-sectional location are aligned vertically.

[0040] Step S20: Based on the pulse data, calculate the longitudinal physical location of the tunnel corresponding to each row of the tunnel lining image; It should be noted that the longitudinal physical location of the tunnel is the actual distance along the tunnel's extension direction, which is used to map the image rows to the actual location of the tunnel.

[0041] Step S20 converts the encoder pulse data into physical spatial location information. Since the linear scan camera records the corresponding pulse count value while acquiring each line of images, the actual physical location of that line of images in the tunnel extension direction can be determined by multiplying the pulse count by the encoder calibration coefficient (the longitudinal displacement corresponding to each pulse).

[0042] Step S30: Perform tunnel structure type identification on the tunnel lining image to determine the tunnel structure type to which the current image segment belongs; It should be noted that tunnel structure type: a category of tunnels classified according to construction methods, including shield tunnels and non-shield tunnels (mining method, cut-and-cover method, etc.).

[0043] Tunnels are mainly classified into two categories based on their construction methods: shield tunnels and non-shield tunnels. Shield tunnels are assembled from precast segments, exhibiting a regular, segmented, arc-shaped surface with relatively uniform texture. Non-shield tunnels (such as those constructed using the mining method or cut-and-cover method) are constructed using cast-in-place or shotcrete methods, resulting in a relatively rough surface with random texture. These two structures differ in cross-sectional shape, surface reflection characteristics, and light distribution, affecting the overlap ratio distribution and illumination correction parameters.

[0044] In the specific implementation, each of the tunnel lining images is first preprocessed, and the longitudinal physical position of the tunnel corresponding to each row of the tunnel lining image is calculated based on the pulse data; then, the tunnel structure type is identified in the preprocessed tunnel lining image to determine the tunnel structure type to which the current image segment belongs.

[0045] Step S40: Based on the tunnel structure type, determine the overlap ratio between circumferentially adjacent linear array cameras; It should be noted that the overlap ratio is the proportion of the width of the overlapping area between adjacent linear scan camera images to the total width of the image.

[0046] In a U-shaped tunnel cross-section, due to the irregular shape (flat bottom, curved upwards on both sides), the installation angles and shooting distances of each line array camera are different, resulting in varying overlap ratios between adjacent line array cameras. Step S40 selects the corresponding overlap ratio parameter based on the identified structure type, thus resolving the non-uniform overlap problem.

[0047] Step S50: Based on the overlap ratio, determine the overlapping area between the images acquired by the adjacent linear scan cameras in a circumferential manner; It should be noted that the overlapping area refers to the portion of the same physical area of ​​the tunnel lining captured in two adjacent images.

[0048] Once the overlap ratio is determined, the specific overlapping area can be defined in two adjacent images. A region of the corresponding width is cropped from the right side of the left image, and a region of the corresponding width is cropped from the left side of the right image. This overlapping area serves as the operational space for subsequent image registration, illumination correction, and multi-band fusion.

[0049] Step S60: Perform image registration on the overlapping area, align the images acquired by each line scan camera to a unified global coordinate system, and obtain an aligned image; It should be noted that image registration refers to transforming multiple images to the same coordinate frame to eliminate geometric deviations.

[0050] Optionally, the image registration specifically includes: extracting and matching feature points in the overlapping area, estimating the geometric transformation relationship between adjacent line scan cameras, and then transforming all images to a unified global coordinate system. Using the line scan camera at the arch position as a reference, its transformation matrix is ​​the identity matrix. Other line scan cameras sequentially obtain their respective global transformation matrices through adjacent transformation relationships, ultimately mapping all images to the same coordinate system to achieve spatial alignment.

[0051] Global coordinate system: A unified coordinate space established with the dome camera as the reference, in which all images are spatially aligned.

[0052] Aligned images: Camera images that have the correct spatial position in the global coordinate system after registration transformation.

[0053] Step S70: Based on the tunnel structure type, determine the lighting parameter set corresponding to the tunnel structure type; It should be noted that the illumination parameter set is a set of gain coefficients and bias coefficients corresponding to the structure type. The illumination parameter set is used to correct the illumination differences of each camera.

[0054] Different tunnel lining surface reflectivity varies depending on the structural type: shield tunnel segments have smooth surfaces with high reflectivity, while non-shield tunnel segments have rough surfaces with low reflectivity. Selecting a pre-established set of illumination parameters based on the structural type ensures that the illumination correction parameters match the surface characteristics of the current section.

[0055] Step S80: Perform illumination correction on the aligned image using the illumination parameter set to obtain the corrected image; It should be noted that due to differences in installation location and auxiliary lighting angle, the images captured by each line scan camera exhibit variations in brightness and contrast. Linear correction is applied to the images from each line scan camera using gain and bias coefficients to ensure consistent grayscale levels in overlapping areas between adjacent cameras, thus eliminating stitching seams.

[0056] Step S90: Perform multi-band fusion on the corrected image within the overlapping area to obtain a seamless circumferential stitched image; It should be noted that multi-band fusion refers to decomposing an image into different frequency bands, fusing them separately, and then reconstructing the image. Low-frequency components reflect overall illumination and color trends, and a smooth, weighted fusion method is used to eliminate illumination abrupt changes. High-frequency components reflect texture and edge details, and a selective preservation strategy is used to maintain sharpness. After fusion, the various frequency band components are recombined into a seamless, circumferentially stitched image, covering the entire tunnel cross-section.

[0057] Seamless circumferential stitched image: A circumferential stitched image covering a complete cross-section of the tunnel, with no visible misalignment or seams.

[0058] Step S100: Based on the pulse data and the longitudinal physical location of the tunnel, the multiple segments of the circumferential seamless stitched image collected along the longitudinal direction of the tunnel are longitudinally stitched together according to their physical locations to obtain a complete tunnel inspection image.

[0059] It should be noted that longitudinal stitching involves arranging multiple circumferentially stitched images along the length of the tunnel in sequence to form a complete large-scale tunnel inspection map.

[0060] As the vehicle travels longitudinally along the tunnel, it continuously outputs circumferentially stitched image segments along the longitudinal direction. These image segments are then arranged sequentially according to their calculated longitudinal physical positions to form a complete two-dimensional inspection image covering the entire length of the tunnel. Since there is no overlap between adjacent longitudinal image segments (due to the working principle of the line scan camera), stitching primarily relies on encoder positioning, and motion jitter compensation eliminates misalignment caused by vehicle speed fluctuations.

[0061] In this embodiment, by identifying the structural type, dynamically estimating the non-uniform overlap ratio, jointly correcting global illumination, fusion of different frequency bands, and encoder-assisted longitudinal positioning, it is possible to adapt to tunnel cross-sections of two different structural types, shield tunneling and non-shield tunneling, and handle complex working conditions where the overlap ratios of each linear array camera in the circumferential direction are different and the illumination conditions are significantly different. It outputs high-quality tunnel inspection images that cover the entire cross-section and length without obvious misalignment or boundary changes in illumination, providing reliable image data for tunnel defect detection.

[0062] Based on any of the above embodiments, the step of identifying the tunnel structure type of the tunnel lining image and determining the tunnel structure type to which the current image segment belongs includes: Multiple image features are extracted from the current image segment and concatenated into a feature vector. The current feature vector is then normalized using preset normalization parameters. These multiple image features include gray-level statistical features, local texture features, gray-level co-occurrence matrix texture features, and frequency domain features. The gray-level statistical features include gray-level mean, standard deviation, skewness, and kurtosis. The local texture features are extracted using a local binary mode, with a radius of a first preset value and a sampling number of a second preset value. The gray-level co-occurrence matrix texture features are used to calculate contrast, energy, homogeneity, and correlation in different directions. The frequency domain features are used to extract the energy distribution and dominant frequency information of each frequency band after performing a Fourier transform on the current image segment. The normalized feature vector is input into the classifier model to obtain the structure type label of the current image segment. The classifier model is trained through the following steps: acquiring tunnel lining image samples with known structure types and labeling them with structure type labels; extracting multi-class image features from each tunnel lining image sample and concatenating them into a feature vector; normalizing the feature vectors of all samples in the training set; using the normalized feature vector as input and the labeled structure type label as output to train the initial classifier model, thereby obtaining the trained classifier model and normalization parameters.

[0063] It should be noted that this embodiment shows two stages: the online inference stage (feature extraction and classification) and the offline training stage (model building and saving).

[0064] The reason for extracting multiple image features from the current image segment and concatenating them into a feature vector is that the image itself is a pixel matrix, which cannot be directly used for classification. It is necessary to extract statistical quantities that can characterize its essential attributes as a basis for judgment.

[0065] The current image segment refers to the image section that needs to be identified in terms of structure type. It also refers to the image section divided into fixed lengths along the longitudinal direction of the tunnel, serving as the smallest unit for a single structure type identification. In the specific implementation, a fixed longitudinal length (e.g., 10 meters) is used as a segment, and identification is performed segment by segment, ultimately outputting a sequence of structure type labels for the entire tunnel along its longitudinal direction.

[0066] For example, the multiple image features include four types: Gray-level statistical characteristics: mean, standard deviation, skewness, and kurtosis extracted from the image gray-level histogram. The mean reflects overall brightness, the standard deviation reflects contrast, the skewness reflects the symmetry of the histogram, and the kurtosis reflects the sharpness of the histogram. Shield tunnel segments have smooth surfaces, concentrated gray-level distribution, and high kurtosis; non-shield tunnels have rough surfaces, dispersed gray-level distribution, and larger standard deviation. Local texture features: Extracted using local binary mode. Local binary mode generates binary codes by comparing the grayscale of each pixel with its neighboring pixels, and counts the frequency of occurrence of various coding patterns in the entire image to form a histogram describing the texture structure. The joints of shield tunnel segments exhibit regular periodic textures, with the histogram concentrated on a few patterns; the surface texture of non-shield tunnels is random, and the histogram distribution is more uniform. Gray-level co-occurrence matrix texture features: Texture is characterized by the joint probability distribution of pixel pairs with specific gray values ​​appearing simultaneously in a specific direction and at a specific distance. Four statistical measures—contrast, energy, homogeneity, and correlation—are calculated in different directions to comprehensively characterize the roughness, uniformity, directionality, and local similarity of the texture. Frequency domain features: A Fourier transform is performed on the image to convert the grayscale information in the spatial domain into frequency information in the frequency domain. The energy distribution of each frequency band reflects the coarseness of the texture, and the dominant frequency information reflects the periodicity of the texture. The joints of shield tunnel segments exhibit periodic textures, with obvious peaks in the spectrum at specific frequencies; the surface texture of non-shield tunnels is random, and the spectral energy is more uniformly distributed across frequency bands.

[0067] Feature vector: A multidimensional numerical vector formed by concatenating the first and last features of multiple image types.

[0068] Because the numerical ranges of various features differ (e.g., the mean gray value may be around 100, while the LBP histogram value is a probability distribution between 0 and 1), directly using them would allow features with larger values ​​to dominate the classification results. In this embodiment, normalization converts each feature dimension into a standardized value with a mean of zero and a standard deviation of one, thereby eliminating the influence of dimensions and ensuring that each feature dimension contributes equally to the classifier.

[0069] The preset normalization parameters refer to the mean and standard deviation of each feature dimension saved during the training phase. They are used directly during online inference to ensure that the data transformation methods for training and inference are consistent.

[0070] The classifier model is a trained binary classifier that outputs the label as either shield tunneling or non-shield tunneling. It finds the optimal classifying hyperplane in the feature space, separating the feature vectors of different structure types. After inputting a normalized feature vector, the classifier model determines which side of the hyperplane the vector lies on and outputs the corresponding structure type label.

[0071] Obtaining image samples of tunnel linings with known structural types and labeling them with structural type tags refers to pre-acquiring a large number of image samples with known structural types. Each sample is manually labeled with its true structural type (i.e., shield tunnel or non-shield tunnel) as the standard answer for training the classifier model. For example, there are no fewer than 500 samples each for shield tunnels and non-shield tunnels.

[0072] Following the exact same parameters and procedures as online inference, four types of features are extracted from each sample image and concatenated into a feature vector. The feature extraction parameters must be consistent between the training and inference phases; otherwise, the trained classifier model will be unable to correctly classify the online data.

[0073] Calculate the mean and standard deviation of the feature vectors of all samples in the training set in each dimension, and normalize the feature vector of each sample using the Z-Score formula. Save the calculated mean and standard deviation as preset normalization parameters for the online inference stage.

[0074] The normalized sample feature vectors and corresponding ground truth labels are input into the classification algorithm. The classifier model parameters are adjusted through optimization to minimize the error between the classifier model's classification prediction results for training samples and the ground truth labels. After training, the classifier model file and normalized parameters are saved for use in the online inference phase. Optionally, a support vector machine with a radial basis function kernel is used, and the kernel parameters and penalty parameters are determined through cross-validation.

[0075] In this embodiment, feature vectors were constructed from four dimensions: grayscale distribution, local texture, grayscale co-occurrence relationship, and frequency domain characteristics. These features comprehensively characterize the differences between the lining surfaces of shield tunnels and non-shield tunnels from different perspectives, ensuring that the classifier model can effectively distinguish between the two structural types. The feature extraction parameters are consistent between the training and inference phases, and the normalized parameters are reused to guarantee the classification accuracy and stability of the classifier model during online inference.

[0076] Based on any of the above embodiments, determining the overlap ratio between circumferentially adjacent line scan cameras based on the tunnel structure type includes: Based on the installation calibration parameters of each linear array camera and the tunnel cross-sectional profile data, the theoretical overlap ratio of each adjacent linear array camera pair is calculated by geometric projection. It should be noted that this step is a rough estimation stage, which uses known geometric parameters to calculate theoretical values ​​without relying on the actual image content.

[0077] Installation and calibration parameters: After the line scan camera is installed, the geometric parameters obtained through calibration include position coordinates, optical axis direction angle, lens field of view angle, working distance, etc., which characterize the spatial attitude and field of view range of each camera in the tunnel cross section.

[0078] Tunnel cross-section profile data is a geometric description of the cross-section of a U-shaped tunnel. It is generally stored in the form of point clouds and is used to reflect the cross-sectional characteristics of a flat bottom, curved sides, and arched top.

[0079] Geometric projection refers to a geometric calculation method that projects the field of view of a line scan camera onto a three-dimensional cross-sectional profile to calculate the coverage area. By projecting the field of view of each line scan camera onto the cross-sectional profile, its coverage arc length in the circumferential direction is calculated. There is an overlap between the coverage arc lengths of adjacent line scan cameras, and the ratio of the overlapping arc length to the single-frame coverage arc length is the theoretical overlap ratio.

[0080] Because the curvature varies at different points along the U-shaped cross-section, and the installation angles and distances of the line scan cameras are different, the theoretical overlap ratios of the eight pairs of adjacent line scan cameras are not uniform values. This step only provides an initial estimate to define the starting range and reference position for subsequent searches.

[0081] Within the neighborhood of the theoretical overlap ratio, the edge stripes of adjacent linear scan camera images are coarsely located to obtain the initial overlap width; It should be noted that this step is the first level of fine estimation, coarse positioning. It obtains a pixel-level estimate of the actual overlap width near the theoretical value.

[0082] The neighborhood of the theoretical overlap ratio is a search interval centered on the theoretical overlap ratio. Because there may be errors in the installation calibration parameters, there is a deviation between the actual overlap ratio and the theoretical value. Therefore, it is necessary to search within a certain range near the theoretical value. For example, the search half-width can be taken as 8%, corresponding to a candidate overlap width range of ±8% of the theoretical overlap width.

[0083] Edge strips are local image regions cropped from the overlapping areas of two adjacent images and used for matching calculations. Coarse localization quickly obtains a preliminary estimate of the overlap width by analyzing the edge strips of two images.

[0084] The initial overlap width output in this step has a certain pixel-level precision, which narrows the search range for further fine matching.

[0085] Traverse candidate overlap widths around the initial overlap width and calculate the normalized cross-correlation value of the overlapping region of adjacent linear array cameras under each candidate overlap width; It should be noted that this step is the second level of fine estimation, normalized cross-correlation fine matching, which determines the optimal overlap width with higher accuracy within a smaller range.

[0086] Traversing candidate overlap widths refers to trying every possible overlap width value one by one within a very small range, centered on the initial overlap width and with a step size of a single pixel. For example, traversal can be performed within a range of plus or minus five pixels of the initial overlap width.

[0087] Normalized cross-correlation is a statistical measure of the similarity between regions in two images. For the two overlapping regions corresponding to candidate overlap widths, the normalized cross-correlation of their grayscale values ​​is calculated. This value ranges from -1 to 1; the closer it is to 1, the more similar the two regions are. The advantage of normalized cross-correlation is its robustness to linear changes in image grayscale values, making it suitable for handling differences in lighting conditions between different cameras.

[0088] For example, normalized cross-correlation calculations can be accelerated in parallel using a graphics processor to meet real-time processing requirements.

[0089] The overlap width corresponding to the maximum normalized cross-correlation value is taken as the target overlap width, and the overlap ratio is calculated based on the ratio of the target overlap width to the image width.

[0090] It should be noted that when the normalized cross-correlation value is at its maximum, it means that the overlapping areas of the two images are most consistent in gray-level distribution, and the corresponding overlap width is the best estimated value.

[0091] The target overlap width is the overlap width expressed in pixels. Dividing this width by the number of pixel columns in a single-frame line scan camera image yields a ratio value between 0 and 1, which is the final overlap ratio.

[0092] The overlap ratio changes dynamically with the longitudinal position of the tunnel: the estimate is updated at regular intervals along the longitudinal direction of the tunnel, forming a sequence of overlap ratio changes with longitudinal position. Since the cross-sectional shapes and overlap ratio distributions of shield tunnels and non-shield tunnels are different, independent mapping tables can be established for each, and linear interpolation can be used to avoid abrupt parameter changes in the transition region between structural types.

[0093] In this embodiment, accurate estimation of the overlap ratio of adjacent linear array cameras in the circumferential direction under a U-shaped tunnel cross-section is achieved, solving the shortcomings of related technologies that cannot adapt to non-uniform overlap distributions due to the assumption of a fixed overlap ratio or the inability of a unified model. Specifically, geometric coarse estimation limits the search range to the vicinity of the theoretical value, improving search efficiency; coarse positioning provides a pixel-level initial estimate within the neighborhood of the theoretical value, narrowing the search range for fine matching; and normalized cross-correlation fine matching performs a pixel-by-pixel fine search near the initial value. Its normalized cross-correlation value is robust to linear changes in grayscale, providing accurate matching results even under inconsistent lighting conditions. The combination of these three methods keeps the overlap ratio estimation error within a small range, providing accurate overlap region boundaries for subsequent image registration and eliminating splicing misalignment and ghosting problems caused by overlap ratio estimation deviations.

[0094] Based on any of the above embodiments, the step of image registration of the overlapping region, aligning the images acquired by each line scan camera to a unified global coordinate system to obtain an aligned image, includes: Using the line scan camera at the arch position as a reference, the transformation matrix of the line scan camera at the arch position is used as the identity matrix. For each line scan camera on one side of the line scan camera at the arch position, the global transformation matrix of each line scan camera is obtained by multiplying it by the inverse of the transformation matrix of the adjacent line scan camera in turn. For each line scan camera on the other side of the line scan camera at the arch position, the global transformation matrix of each line scan camera is obtained by multiplying the transformation matrix of the adjacent line scan cameras by the right in sequence. Based on the global transformation matrix of each line scan camera, the images acquired by each line scan camera are transformed into the global coordinate system to obtain the aligned image.

[0095] It should be noted that in this embodiment, the line scan camera at the arch position is used as the reference benchmark. A global coordinate system is established through the transfer operation of the transformation matrix, thereby realizing the coordinate unification of images from multiple line scan cameras.

[0096] A reference datum is a standard position used as a coordinate reference during multi-image alignment. Images from all other line scan cameras are mapped to the global coordinate system through transformations relative to this reference camera, and their own coordinate transformations are defined as no-transformation states.

[0097] The transformation matrix is ​​a mathematical expression describing the mapping relationship between the original coordinates and the target coordinates of the image. The transformation matrix of the line scan camera at the dome position is set to the identity matrix, which means that the image of the line scan camera remains unchanged in the global coordinate system without any geometric transformation.

[0098] The reason for choosing the line scan camera at the arch position as the reference is that the line scan camera at the arch position has the most upright viewing angle, the shooting direction is almost perpendicular to the lining surface, the image geometric distortion is minimal, and the illumination is relatively uniform. Using this line scan camera as the reference can make the registration error propagate symmetrically to the left and right sides, avoiding excessive cumulative error on one side.

[0099] "One side" refers to all the line array cameras located on one side of the arch-shaped line array camera in the circumferential direction. In the U-shaped tunnel cross-section, the line array cameras are arranged sequentially along the circumferential direction from the bottom left to the arch and then to the bottom right. The arch-shaped line array camera divides the entire line array camera sequence into left and right sides. The left multiplication sequentially defines the direction and order of transformation propagation. Starting from the arch-shaped line array camera, the transformation is propagated to each line array camera further away on that side: the global transformation matrix of the line array camera immediately adjacent to the arch is obtained by multiplying the arch-shaped line array camera's transformation matrix by the inverse of its adjacent transformation matrix; for the next line array camera further away, the transformation is multiplied by the inverse of the next adjacent transformation matrix, and so on.

[0100] The adjacent line scan camera transformation matrix refers to the geometric transformation relationship between two adjacent line scan cameras estimated during image registration. The inverse matrix is ​​used because the transfer direction is opposite to the registration estimation direction. During registration, the transformation from the current line scan camera to the adjacent line scan camera is estimated, while the global transformation needs to be transformed back from the far-end line scan camera to the dome reference.

[0101] The other side refers to all the line array cameras located on the opposite side of the dome-shaped line array camera in the circumferential direction. The transmission direction on this side is opposite to that of the side mentioned above.

[0102] Sequential right multiplication: When propagating from the dome-shaped line array camera to the far end on that side, each step multiplies the adjacent transformation matrix to the current global transformation matrix on the right. The difference between right and left multiplication stems from the non-commutative property of matrix multiplication and is related to the arrangement direction of the line array cameras and the composition order of coordinate transformations.

[0103] For the linear array cameras on both sides of the dome-shaped linear array camera, two different operation methods are used: left multiplication by the inverse matrix and right multiplication by the matrix. This is because the arrangement direction of the two linear array cameras is a mirror image of each other with respect to the reference datum. The non-commutative property of matrix multiplication requires that the two sides use the opposite multiplication order to ensure the correct synthesis of coordinate transformation.

[0104] In the specific implementation, the line scan camera to the left of the line scan camera at the dome position obtains the global transformation matrix by multiplying the adjacent transformation matrix on the left, and the line scan camera to the right obtains the global transformation matrix by multiplying the adjacent transformation matrix on the right.

[0105] The image pixel coordinates of each linear scan camera are mapped to new coordinate positions in the global coordinate system through the corresponding global transformation matrix. Since the mapped coordinates may not be integer positions, grayscale values ​​need to be sampled using interpolation methods (such as bicubic interpolation).

[0106] The global coordinate system is a unified coordinate space established with the dome-shaped line scan camera as the origin. After transformation, the images from all line scan cameras have the correct spatial positional relationship in this coordinate space. After transforming the images to the global coordinate system, the images from each line scan camera are spatially aligned in the global coordinate system. The same physical position of adjacent line scan cameras in the overlapping area corresponds to a nearby coordinate position, which provides a pixel-level correspondence for subsequent illumination correction and multi-band fusion.

[0107] In this embodiment, a global coordinate system is established using the linear array camera at the arch location as a reference. A strategy of transferring transformation matrices from both sides is employed to achieve high-precision spatial alignment of multi-camera images. The arch linear array camera has the most accurate viewpoint and minimal distortion. Using this camera as a reference, the registration error is propagated symmetrically to both sides, effectively avoiding the problem of excessive cumulative error on one side. Left and right multiplication operations are used on both sides respectively to ensure correct synthesis of coordinate transformations in different circumferential directions.

[0108] Based on any of the above embodiments, the step of performing illumination correction on the aligned image using the illumination parameter set to obtain the corrected image includes: The alignment images of each line scan camera are corrected using a correction model. The pixel values ​​of the corrected image of each line scan camera are obtained by linearly transforming the pixel values ​​of the alignment image based on the gain coefficient and bias coefficient of the line scan camera. Using the line scan camera at the arch position as a reference, the gain coefficient and offset coefficient of the line scan camera at the arch position are fixed. The optimization objective is to minimize the sum of squared grayscale differences of all adjacent linear scan camera pairs after correction in the overlapping region. Under the constraint of gain coefficient normalization, the gain coefficient and bias coefficient of all linear scan cameras are solved jointly. The aligned images of each line scan camera are corrected based on the obtained gain coefficients and bias coefficients of each line scan camera to obtain the corrected images.

[0109] It should be noted that the correction model is a mathematical expression for transforming the grayscale values ​​of an image, describing the numerical relationship between the image before and after correction. For example, the correction model adopts a gain-bias linear form, that is, a global gain-bias correction model is used to correct the aligned images of each linear scan camera.

[0110] The gain factor reflects the differences in photosensitivity and light intensity of line scan cameras; it is a multiplicative factor. Different line scan cameras, due to differences in model, aperture setting, or the light intensity of their location, will capture images with different overall brightness levels. The gain factor is used to scale the overall grayscale range of the image.

[0111] The bias coefficient is used to reflect the dark level offset and the difference between the ambient light base and the background light; it is an additive factor. Even in the absence of illumination, the linear scan camera sensor may still have a background signal; in addition, diffuse ambient light in the tunnel will also have an overall uplift effect on the image. The bias coefficient is used to shift the overall gray level of the image.

[0112] Linear transformation means that the value of each pixel in the corrected image is equal to the original pixel value multiplied by the gain coefficient plus the bias coefficient.

[0113] The goal of illumination correction is to eliminate relative illumination differences between line scan cameras, rather than adjusting the image to an absolute standard. Therefore, a reference line scan camera needs to be selected, and its parameters fixed as a benchmark, while the parameters of other line scan cameras are optimized relative to this benchmark.

[0114] The line scan camera at the arch is selected as the reference. The gain coefficient of the line scan camera at the arch is fixed at the reference value, and the offset coefficient is fixed at zero, meaning that the image from the line scan camera at the arch is not subjected to any grayscale transformation.

[0115] The reason for fixing the parameters of the arch-top line array camera is that the shooting angle of the line array camera at the arch is the most accurate and the lighting conditions are relatively uniform. Its image quality is the best among the nine line array cameras. Using it as a benchmark can make the correction direction clear and avoid the simultaneous fluctuation of the parameters of all line array cameras, which would result in no unique solution.

[0116] All adjacent line scan camera pairs refer to combinations of line scan cameras that are circumferentially adjacent. With nine line scan cameras arranged circumferentially, there are eight pairs of adjacent line scan cameras. The overlapping area of ​​these eight pairs covers the entire circumference. The optimization objective considers the differences between all adjacent line scan camera pairs to ensure global illumination consistency.

[0117] Sum of squared grayscale differences after correction in overlapping regions: This measures the degree of grayscale difference between two adjacent images after correction within the overlapping region. For each pixel location within the overlapping region, the corrected grayscale difference between the two images is calculated and squared, and the sum of the squared differences for all pixel locations is then calculated. The smaller this value, the better the illumination consistency between the two linear scan cameras in the overlapping region.

[0118] Gain coefficient normalization constraints are added to prevent parameter degradation in the optimization problem. Since scaling both the gain and bias coefficients simultaneously results in the same correction effect, without constraints, the optimization problem would have infinitely many equivalent solutions. The normalization constraint requires that the sum of the gain coefficients of all line scan cameras equal the total number of line scan cameras, meaning the average gain coefficient remains one, thus ensuring the uniqueness of the solution.

[0119] Joint solution refers to simultaneously solving for the gain and bias coefficients of all nine line scan cameras, rather than solving them independently for each camera. Individual point correction can only guarantee the grayscale range stretching of each camera's own image, but cannot guarantee consistency between adjacent line scan cameras in overlapping areas. Joint optimization solves for the parameters of all line scan cameras as a whole, achieving optimal global illumination consistency.

[0120] The optimization problem described above is a quadratic programming problem with equality constraints in mathematics. It can be solved by constructing the Lagrange function using the Lagrange multiplier method, taking the partial derivatives with respect to each parameter and setting them to zero, and then obtaining a system of linear equations.

[0121] The gain and bias coefficients of each linear scan camera are obtained by joint solution and substituted into the linear transformation formula to calculate the corrected grayscale value pixel by pixel. The corrected image pixel values ​​are cropped to the effective grayscale range (e.g., 0~255) to ensure normal image display.

[0122] After the above processing steps, the grayscale distribution of each camera image in the overlapping area tends to be consistent, and the boundary of illumination change in the circumferential direction is effectively eliminated, which provides input images with good grayscale consistency for subsequent multi-band fusion.

[0123] In this embodiment, the dome-shaped line array camera is used as a reference, ensuring a clear correction direction and using the image with the highest quality as a reference. The objective function is to minimize the sum of squared grayscale differences between all adjacent line array cameras in the overlapping region after correction, elevating illumination correction from independent single-point processing to global consistency optimization. The introduction of gain normalization constraints ensures the uniqueness and numerical stability of the solution to the optimization problem. After the above correction, the grayscale jump amplitude between the circumferential line array cameras is significantly reduced, and circumferential grayscale consistency is improved, providing a uniformly distributed input image for subsequent multi-band fusion and eliminating visual stitching seams caused by illumination differences.

[0124] Based on any of the above embodiments, the step of performing multi-band fusion of the corrected image within the overlapping area to obtain a circumferentially seamless stitched image includes: The corrected images from each linear array camera are used to construct multi-layer pyramids, and each corrected image is decomposed into components of different frequency bands. For low-frequency components, a fusion weight is determined within the overlapping region based on the distance from each pixel position to the center position of the overlapping region. The fusion weight is then used to perform weighted fusion of the pixel values ​​of the corrected images of two adjacent line scan cameras within the overlapping region. For high-frequency components, the gradient energy in the neighborhood of each pixel is calculated within the overlapping area, and the pixel value of the corrected image on the side with larger gradient energy is determined as the fusion result at the pixel location. The fused frequency band components are reconstructed layer by layer to obtain the seamless circumferential stitched image.

[0125] It should be noted that this embodiment employs multi-band fusion based on pyramid decomposition, decomposing the image into different frequency bands and processing them separately. The low-frequency band reflects the overall illumination and color trends, while the high-frequency band reflects details such as texture and edges. Different fusion methods are used for the characteristics of different frequency bands, and finally, a seamless stitched image is obtained by reconstructing layer by layer.

[0126] A multi-layered pyramid is a multi-scale hierarchical representation structure for images. It forms a series of image layers from high resolution to low resolution by progressively filtering and downsampling the image. A Gaussian pyramid is constructed by progressively downsampling after Gaussian low-pass filtering, with each layer being half the size of the layer above.

[0127] The components of different frequency bands are obtained through the Laplacian pyramid. Each layer of the Laplacian pyramid is obtained by subtracting the upsampling result of the current layer of the Gaussian pyramid from that of the layer above it. Specifically, the lowest layer of the Laplacian pyramid, i.e., the zeroth layer, contains the highest frequency information, which is used to reflect the texture and edge details of the image; as the number of layers increases, the frequency bands gradually decrease, and the information contained becomes more and more macroscopic; the highest layer contains the lowest frequency information, which is used to reflect the overall illumination distribution and global brightness trend of the image.

[0128] A Gaussian pyramid is a pyramid constructed by downsampling after successive Gaussian low-pass filtering, with the spatial resolution decreasing at each level. A Laplacian pyramid is obtained by subtracting the upsampling results of each level of the Gaussian pyramid from the upsampling results of the previous level; each level represents image information for one frequency band.

[0129] The significance of frequency band decomposition lies in separating the hierarchical structure of image content: high-frequency components correspond to the fine textures such as cracks and peeling on the surface of tunnel lining, and their clarity needs to be preserved; low-frequency components correspond to the overall illumination distribution and the brightness differences between cameras, and a smooth transition is needed.

[0130] Low-frequency components refer to the higher-level components in the pyramid, used to reflect the overall illumination trend and grayscale distribution of the image. In the specific implementation, the Laplacian pyramid has five layers, with the third and fourth layers serving as low-frequency layers for weighted fusion.

[0131] The distance from each pixel location to the center of the overlapping region is used to determine the fusion weights. For each pixel location within the overlapping region, its distance to the center of the overlapping region is calculated. The closer the pixel location is to the center of the overlapping region, the closer it is to the middle of the fusion boundary; the closer the pixel location is to the edge of the overlapping region, the closer it is to the non-overlapping region of the image.

[0132] The fusion weights are calculated based on the aforementioned distances, and within the overlapping region, they exhibit a gradual change from 1 to 0 from one side to the other. When a pixel is closer to the left edge of the overlapping region, the left image has a larger weight; when it is closer to the right edge of the overlapping region, the right image has a larger weight; at the center of the overlapping region, the two images have equal weights.

[0133] Weighted fusion refers to summing the pixel values ​​at corresponding positions within the overlapping region of two images according to the aforementioned weights. Because the weights change smoothly within the overlapping region, the pixel values ​​after fusion also transition smoothly without abrupt changes.

[0134] In this embodiment, the reason for using weighted fusion in the low-frequency band is that the low-frequency component reflects the overall brightness trend. The human eye is more sensitive to abrupt changes in low-frequency information. Using smooth weighted transition can effectively eliminate brightness jumps at the splicing seam.

[0135] High-frequency components refer to the lower-level components in the pyramid, used to reflect information such as texture, edges, and details of the image. In the specific implementation, layers 0 to 2 of the Laplacian pyramid are used as high-frequency layers for gradient energy selective fusion.

[0136] A pixel neighborhood refers to a local window centered on the current pixel. In the specific implementation, a 3x3 pixel neighborhood is used.

[0137] Gradient energy reflects the degree of gray-level change in an image's neighborhood, and is obtained by calculating the sum of the gradient magnitudes in the horizontal and vertical directions. A larger gradient energy indicates richer texture, sharper edges, and more significant details in that region.

[0138] Determining the pixel value of the corrected image with the larger gradient energy as the fusion result at the pixel location means comparing the gradient energy of the two images in that neighborhood at the same pixel location and selecting the pixel value of the image with the larger gradient energy as the fusion result, rather than weighting the pixel values ​​of the two images.

[0139] The reason for using gradient energy-based selective fusion in the high-frequency band is that high-frequency information determines the image's sharpness. Using a weighted average would lead to blurred texture details and reduced edge sharpness. Selecting the side with the larger gradient energy is equivalent to retaining the content of an image with richer texture and clearer details at each position, thereby maximizing the visual sharpness of the stitched image.

[0140] Layer-by-layer reconstruction starts from the highest layer of the pyramid, i.e. the lowest frequency layer, and upsamples and superimposes the corresponding high-frequency layers step by step: the highest layer image is upsampled by 2 times and added to the high-frequency fusion result of the next layer to obtain the reconstruction result of that layer; then upsampling continues and is added to the high-frequency fusion result of the next lower layer, and so on, until the original resolution is restored.

[0141] The reconstruction process essentially recombines the frequency components fused in different ways into a complete image. The low-frequency components have undergone weighted fusion for a smooth transition, while the high-frequency components have undergone selective fusion to maintain clarity. The reconstructed image has no visible seams and retains rich texture details.

[0142] In this embodiment, the contradiction between eliminating stitching seams and maintaining sharpness in the stitched image is resolved through multi-band decomposition and differential fusion. Low-frequency components employ distance-based weighted fusion, ensuring a smooth transition of overall illumination within the overlapping area and effectively eliminating residual illumination jumps. High-frequency components utilize selective fusion based on local gradient energy, preserving information from the side with clearer texture and sharper edges at each pixel location, avoiding high-frequency detail blurring caused by weighted averaging. The reconstructed circumferential stitched image achieves both a visually seamless transition and maximizes the preservation of subtle texture features on the tunnel lining surface.

[0143] Based on any of the above embodiments, the step of longitudinally stitching multiple segments of the circumferential seamless image acquired along the longitudinal direction of the tunnel according to their physical positions, based on the pulse data and the longitudinal physical position of the tunnel, to obtain a complete tunnel inspection image includes: Based on the pulse data and calibration coefficients, the starting position of each segment of the seamless circumferential stitched image on the vertical canvas is determined. At the vertical seam of two adjacent seamlessly stitched circumferential images, extract the first row of grayscale mean sequence at the end of the previous segment (preset number of rows) and the second row of grayscale mean sequence at the beginning of the next segment (preset number of rows). Calculate the normalized cross-correlation values ​​of the first row grayscale mean sequence and the second row grayscale mean sequence at different offsets; The offset corresponding to the maximum normalized cross-correlation value is used as the vertical offset correction amount; Based on the offset correction amount, the starting row position of the next segment of the circumferential seamless stitched image is finely adjusted; At the longitudinal joint, the preset number of rows at the end of the previous segment and the preset number of rows at the beginning of the next segment are linearly weighted and fused to obtain the complete tunnel inspection image.

[0144] It should be noted that when the line scan camera collects data longitudinally along the tunnel, there is no overlapping area between adjacent data collection strips, so image feature matching cannot be used for stitching. Therefore, it is necessary to rely on encoder positioning and supplement it with motion compensation of image content.

[0145] The calibration factor is the physical displacement corresponding to each pulse, measured in millimeters per pulse. The calibration factor is obtained through calibration, converting the discrete values ​​of pulse counts into continuous spatial distance values.

[0146] The vertical canvas is a two-dimensional canvas used to stitch together complete tunnel images, with its height representing the longitudinal extension direction of the tunnel. The vertical position of each seamless circumferential stitched image segment on the canvas is determined by its corresponding physical position. The vertical physical position of the first row of that image segment is obtained by multiplying the pulse count value by a calibration coefficient, and the image segment is then placed in its corresponding row position on the canvas. The image segments are arranged in ascending order of their physical positions to form a complete vertical coverage.

[0147] The longitudinal seam is the boundary between two seamlessly stitched circumferential images on the longitudinal canvas. Due to encoder positioning errors or vehicle speed fluctuations, the two images at the seam may not be perfectly aligned in terms of content, specifically manifested as misalignment or duplication at the same physical location.

[0148] The preset number of rows is a pre-defined number of image rows used for matching calculations. For example, a preset number of rows can be taken above and below the seam, such as 20 rows. The number of rows should not be too few or too many. If the number of rows is too few, the statistical information will be insufficient; if the number of rows is too many, the amount of calculation will increase, and rows far from the seam will be less affected by speed fluctuations.

[0149] The row grayscale mean sequence is the average grayscale value of all pixels in each row of the image. Compressing an image row into a single value simplifies matching a two-dimensional image into matching a one-dimensional sequence, reducing computational complexity.

[0150] The first and second grayscale mean sequences correspond to the grayscale mean sequences of the last preset number of rows in the previous segment and the first preset number of rows in the next segment, respectively.

[0151] Different offsets refer to different values ​​for the relative translation of two sequences. Due to fluctuations in vehicle speed, there may be a deviation of a few pixels between the position recorded by the encoder and the actual image content. Therefore, it is necessary to try multiple offsets within a certain range to find the best-matching alignment position. For example, the search range can be ±5 pixels.

[0152] Normalized cross-correlation is a statistic that measures the similarity between two one-dimensional sequences. For each candidate offset, the normalized cross-correlation value is calculated by shifting the second row's gray-level mean sequence relative to the first row's gray-level mean sequence by that offset. This value ranges between -1 and 1; the closer it is to 1, the more similar the two sequences are. The maximum normalized cross-correlation value indicates that the row gray-level distribution of the two images at the seam is most consistent, and the corresponding offset is the optimal alignment offset. This offset can compensate for encoder positioning errors caused by vehicle speed fluctuations, ensuring precise vertical alignment of the image content on both sides of the seam.

[0153] The step of fine-tuning the starting row position of the next seamlessly stitched image segment based on the offset correction amount means applying the calculated offset correction amount to the starting row position of the next image segment, adjusting it from the coarse position determined by the encoder to the precise position verified by the image content. After fine-tuning, the end of the previous segment and the beginning of the next segment at the seam are aligned in terms of content, eliminating row misalignment caused by positioning errors.

[0154] Understandably, even after fine-tuning the position, the images on both sides of the seam may still have differences in grayscale values, and direct splicing would create a horizontal line of abrupt grayscale change. A smooth grayscale transition is achieved by merging several rows on each side of the seam.

[0155] Linear gradient weighted fusion refers to a fusion weight that changes linearly as the row position moves from the center of the seam to both sides. At the center of the seam, the weights on both sides are equal; gradually upwards, the weights increase for the upper image segment and decrease for the lower image segment; the opposite is true downwards.

[0156] Linear gradient weighted blending makes the pixel values ​​at the seam change smoothly from one segment to the next, making the grayscale jump imperceptible to the human eye, thus achieving a visually seamless stitching.

[0157] After the above processing steps, the multiple seamlessly stitched circumferential images are combined into a complete inspection image covering the entire length of the tunnel.

[0158] Optionally, the method further includes: The quality evaluation index of the complete tunnel inspection image is calculated in multiple preset dimensions. The quality evaluation index includes the registration accuracy index of overlapping areas, the smoothness index of splicing seams, and the illumination consistency index. The registration accuracy index of overlapping areas includes the root mean square error and structural similarity of pixels in the overlapping areas. The smoothness index of splicing seams includes the gradient difference and grayscale jump value on both sides of the splicing seam. The illumination consistency index includes the grayscale mean fluctuation amplitude of the spliced ​​image along the circumferential direction. When any quality assessment indicator fails to meet the preset quality requirements, the corresponding parameter adjustment is triggered according to the type of the non-compliant indicator, and the relevant steps of image stitching are re-executed and evaluated again. Repeat the above process until all quality assessment indicators meet the preset quality requirements, or the preset maximum number of iterations is reached.

[0159] In this embodiment, encoder position mapping provides preliminary positioning based on physical distance, ensuring that each image segment is arranged in the correct order on the canvas. One-dimensional cross-correlation motion compensation, by comparing the row grayscale distribution on both sides of the seam, finely corrects the positioning error caused by vehicle speed fluctuations based on encoder positioning, eliminating row misalignment. Linear gradient weighted fusion eliminates grayscale jumps at the seam through smooth transition, making the stitching result visually natural and continuous. Stable and reliable longitudinal stitching is achieved without relying on image feature matching, ensuring the row alignment accuracy and visual continuity of the complete tunnel inspection image in the longitudinal direction.

[0160] Furthermore, based on the same concept as the image stitching method described above, this application also provides an adaptive seamless stitching method and system for U-shaped tunnel inspection images.

[0161] In this embodiment, for a U-shaped tunnel cross-section using a circumferential arrangement of nine linear array cameras, with no horizontal overlap, varying overlap ratios between adjacent cameras, different lighting conditions, and both shield and non-shield tunnel structures, the small images captured by each linear array camera are stitched together into a large image without significant misalignment or boundaries of lighting variations. The overall data flow of the adaptive seamless stitching method for U-shaped tunnel inspection images is as follows: Original image and encoder data → Step S1 preprocessing → Preprocessed image →Step S2 Structure Identification → Structure Type Labeling →Step S3 Overlap Ratio Estimation → Overlap Ratio Mapping Table →Step S4 Circumferential Registration → Aligned Image →Step S5 Illumination Correction → Corrected Image →Step S6 Multi-Band Fusion → Circumferential Stitching Large Image →Step S7 Longitudinal Stitching → Complete Tunnel Large Image →Step S8 Quality Assessment → Quality Indicators and Feedback Parameters.

[0162] The following details the input data, processing steps, and output results for each step: Step S1: Simultaneous acquisition of images by multiple cameras and image preprocessing: Input data: Raw grayscale images simultaneously acquired by 9 line scan cameras C1~C9 (i=1,2,...,9), each original image is a single-channel grayscale image with a pixel bit depth of 12 bits or 8 bits; the pulse count sequence P(t) output by the incremental encoder mounted on the wheel axle, and the encoder calibration parameter k_d (the longitudinal displacement corresponding to each pulse, in mm / pulse). C1 is located at the bottom left side of the tunnel, C5 is located directly above the arch, and C9 is located at the bottom right side of the tunnel, arranged sequentially along the circumference.

[0163] Processing procedure: raw images from each camera Perform the following four sub-steps in sequence: (a) Dark current correction: Dark field images of each camera were pre-acquired under no-light conditions. (Reference image taken with the light source off), subtract the dark field image pixel by pixel from the original image, i.e. Eliminate sensor dark current noise and fixed pattern noise to obtain I_darkcorrected_i.

[0164] (b) Flat field correction: Flat field images of each camera are pre-acquired using an integrating sphere uniform light source. The image after dark current correction is divided pixel by pixel by the flat field image and multiplied by the reference gray value (taken as 128), i.e., (I_darkcorrected_i / ×128, eliminating lens vignetting and uneven sensor pixel response.

[0165] (c) Denoising: The non-local means (NLM) denoising algorithm is used on the flat-field corrected image. The search window size is set to 7×7 pixels, the similarity window size is set to 3×3 pixels, and the filter intensity parameter h is set to 10. This preserves edge details while suppressing random Gaussian noise.

[0166] (d) Distortion correction: using camera intrinsic parameters (focal length) obtained in advance through checkerboard calibration. , Main point , ) and distortion coefficient (radial distortion) , , Tangential distortion , A radial-tangential distortion model is used to perform pixel-by-pixel remapping correction on each row of the linear array image. The distortion correction formula is: , The grayscale values ​​are sampled using bilinear interpolation on the corrected coordinates.

[0167] Simultaneously, based on the encoder pulse count P(t) and calibration parameters... Calculate the longitudinal physical position of the tunnel corresponding to each row of images: L(t) = P(t) × The location information is then appended to the image metadata for use in step S7 for vertical stitching.

[0168] Output: Preprocessed images from 9 cameras (i=1,2,...,9), each image is a single-channel grayscale image, with dark current noise, flatness, random noise, and lens distortion eliminated; simultaneously, the vertical physical position sequence L(t) corresponding to each image row is output. These data serve as inputs for steps S2 and S3.

[0169] Step S2: Automatic identification of tunnel structure type: Input data: The preprocessed image output from step S1 The area near the arch (images from cameras C4, C5, and C6) from nine camera images was selected as the input image for structure recognition, as this area best reflects the structural characteristics of the tunnel lining. A fixed longitudinal length (10 meters, corresponding to N_row = 10000 / k_d rows) was used as a segment, and structure type recognition was performed on the image of each segment.

[0170] Processing procedure: Includes two stages: offline training and online inference. Phase 1: Offline Training (Executed only once): (a1) Training Sample Collection: Image samples of linings of known structural types were collected in advance in different tunnels. No fewer than 500 images were collected for shield tunnels and no fewer than 500 images were collected for non-shield tunnels. Each image was uniformly cropped to 512×512 pixels. Each sample image was labeled with the true structural type label y∈{0=shield,1=non-shield}.

[0171] (a2) Training sample feature extraction: For each training sample image, extract 4 types of features according to the following fixed parameters, and concatenate them into an 89-dimensional feature vector: Type 1 grayscale statistical features (4 dimensions): Calculate the image grayscale histogram and extract the mean (reflecting overall brightness), standard deviation (reflecting contrast), skewness (reflecting histogram symmetry), and kurtosis (reflecting histogram sharpness). Shield tunnel segments have smoother surfaces, concentrated grayscale distribution, and higher kurtosis; non-shield tunnels have rougher surfaces, dispersed grayscale distribution, and larger standard deviation.

[0172] Type 2 LBP Texture Features (59-dimensional): Uniform LBP (LBP_u2) is used, with a neighborhood radius R=1 and sampling points P=8. The specific calculation method is as follows: For each pixel in the image, P=8 neighboring pixels are uniformly sampled around it within a circle of radius R=1. The gray value of each neighboring pixel is compared with the gray value of the center pixel: values ​​greater than or equal to the center pixel's gray value are recorded as 1, and values ​​less are recorded as 0, resulting in an 8-bit binary code. Uniform LBP classifies patterns with no more than two 0→1 or 1→0 transitions in the binary code as uniform patterns, totaling 58. The remaining patterns are grouped into one category, for a total of 59 patterns. The frequency of each pattern in the entire image is counted and normalized into a probability histogram, resulting in a 59-dimensional feature vector. LBP_u2 (R=1, P=8) generates 59 bins of histograms, which are normalized and used as the 59-dimensional feature vector. The joints of shield tunnel segments exhibit regular periodic textures, with LBP histograms concentrated on a few patterns; the surface textures of non-shield tunnels are random, with LBP histograms distributed more evenly.

[0173] Type 3 GLCM texture features (16-dimensional): First, the image grayscale levels are compressed from 256 to 64 (i.e., the grayscale values ​​from 0 to 255 are linearly mapped to 0 to 63). Then, the Gray-Level Co-occurrence Matrix (GLCM) is calculated, with a distance d = 1 pixel, and calculation directions of 0° (horizontal), 45° (diagonal), 90° (vertical), and 135° (anti-diagonal). Four statistics are extracted from the GLCM in each direction: contrast (measures grayscale difference, with lower values ​​for smooth shield surfaces), energy (measures texture uniformity), homogeneity (measures local similarity), and correlation (measures linear dependence of grayscale). 4 directions × 4 statistics = 16-dimensional features.

[0174] Type 4 FFT spectral features (10-dimensional): After performing a 2D FFT transform on the image, the amplitude spectrum is extracted. The spectrum is divided into 8 equally spaced frequency bands (from low to high frequency) according to radial frequency. The proportion of energy in each frequency band to the total energy of the spectrum is calculated to obtain 8-dimensional frequency band energy distribution features. Then, the frequency component with the highest energy in the spectrum and its amplitude are extracted to obtain 2-dimensional dominant frequency features. A total of 10 dimensions are achieved. Shield tunnel segment joints exhibit periodic textures, and the spectrum shows obvious peaks at specific frequencies (frequency band energy is concentrated in a certain frequency band); non-shield tunnel surface textures are random, and the spectral energy is more uniformly distributed between frequency bands.

[0175] (a3) Feature concatenation and normalization: The above four types of features are concatenated into an 89-dimensional feature vector V = [grayscale statistics 4-dimensional, LBP texture 59-dimensional, GLCM texture 16-dimensional, FFT spectrum 10-dimensional]. Z-score normalization is then performed on the feature vectors of all samples in the training set. ,in and This represents the mean and standard deviation of the feature vectors in the training set in each dimension. Save. and For online reasoning use.

[0176] (a4) SVM Model Training: A Support Vector Machine (SVM) classifier is trained using a normalized 89-dimensional feature vector as input and the true structure type label y∈{0,1} as output. The SVM employs a Radial Basis Function (RBF) kernel. The kernel parameter gamma and the penalty parameter C are determined through a 5-fold cross-validation grid search (the gamma search range is {... ,..., }, C search range { ,..., After training, save the SVM model file model.svm, along with the feature extraction parameters (R=1 / P=8 for LBP, 64 levels / d=1 / 4 direction for GLCM, and 8 frequency bands for FFT) and normalization parameters ( , These are packaged together as a pre-trained structure recognition model package. Note: The feature extraction parameters are bound to the SVM model. The exact same feature extraction parameters must be used during online inference; otherwise, the SVM model will not be able to classify correctly.

[0177] Phase Two: Online Reasoning (Executed in real-time during actual tunnel inspection): (b1) Online feature extraction: The input image of the current segment (the stitching result of three camera images C4, C5 and C6) is cropped into a 512×512 pixel sub-image, and four types of features are extracted according to the same parameters as in the training stage (LBP_u2, R=1, P=8; GLCM, 64 levels, d=1, 4 directions; FFT, 8 frequency bands), and stitched into an 89-dimensional feature vector V.

[0178] (b2) Online normalization: using data saved during the training phase and Perform Z-Score normalization on the current feature vector to obtain .

[0179] (b3) Online SVM classification: Input the pre-trained SVM model model.svm. The SVM outputs the predicted structure type label for the current segment, label∈{0=shield tunnel, 1=non-shield tunnel}, and also outputs the decision confidence score (the distance from the SVM's decision function value to the classification boundary). When |score| < threshold 0.3, it is marked as uncertain, triggering manual review or re-identification using a longer segment.

[0180] (b4) Segment Labeling and Merging: The above identification is performed segment by segment along the tunnel longitudinal direction in 10-meter units, and the structure type label sequence {(location L, label, score)} of each segment is output. When multiple consecutive segments have the same label, they are merged into one large segment; when the labels of adjacent segments change, a transition area (2-5 meters in length) is marked as a transition area, and the structure type in the transition area is determined by weighting the confidence of the two labels.

[0181] Output: Tunnel longitudinal structure type labeling sequence {( )},in This represents the vertical starting position of the k-th segment. ∈{shield tunnel, non-shield tunnel, transition zone}. This annotation sequence serves as input to steps S3 and S5, used to select the overlap ratio mapping table and illumination parameter set for the corresponding structure.

[0182] Step S3: Adaptive estimation of circumferential non-uniform overlap ratio: Input data: The preprocessed image output from step S1 (i=1,...,9), the structure type label sequence output by step S2. Camera installation calibration parameter table (pre-calibrated): position coordinates of each camera ( ), optical axis direction angle Lens field of view Working distance And point cloud data of tunnel cross-section contour.

[0183] Processing procedure: For each pair of circumferentially adjacent cameras ( , (A total of 8 pairs, i=1,...,8), an adaptive estimation of the actual overlap ratio is performed using a two-stage method of coarse estimation followed by fine estimation. Based on the structural type of the current longitudinal position output in step S2, the parameter set of the corresponding structure is selected for processing.

[0184] (a) Coarse estimation, calculated based on the theoretical overlap ratio using installation geometry parameters: Using the installation and calibration parameters of the line scan cameras and the point cloud of the tunnel cross-section contour, the theoretical coverage area of ​​each pair of adjacent line scan cameras is calculated through geometric projection. The line scan cameras are then... field of view Projecting this onto the tunnel cross-sectional profile yields its circumferential arc length. Similarly, we can obtain Coverage arc length Based on the geometric relationship between the installation angle difference and the coverage arc length of the two linear array cameras, the theoretical overlap arc length is calculated. Theoretical overlap ratio = / min( , Because the curvature varies at different points along the U-shaped cross-section, This is only an initial estimate; the theoretical overlap ratio of 8 pairs of linear scan cameras. ~ They are all different.

[0185] (b) Precise estimation, based on phase correlation and precise overlap ratio search of NCC: In theoretical overlap ratio Set search range nearby [ , The search half-width Δ is set to 0.08 (i.e., 8%). Based on the image width... Based on (number of pixel columns), the number of pixels corresponding to the candidate overlap width range is w∈[( )× , ( )× The search step size is 1 pixel.

[0186] Step 1, Phase Correlation Coarse Localization: Image Acquisition The width on the right is (Δ×) + × The edge strip of the ) is taken as the reference area, and then... The edge strips of equal width on the left are used as the registration region. A two-dimensional FFT is performed on each strip to calculate the phase of the cross-power spectrum. The phase correlation peak is obtained through inverse FFT, and the peak position corresponds to the translation between the two images. (Subpixel precision). This is the preliminary estimated pixel-level overlap width. Phase correlation method, based on frequency domain phase information, is robust to linear grayscale differences between two images (brightness / contrast differences caused by different lighting conditions).

[0187] The second step is precise matching by the NCC: Within a radius of ±5 pixels, traverse the candidate overlap widths w with a step size of 1 pixel. For each w, from... Extract the overlapping area with width w on the right. ,from Extract the overlapping area with width w on the left. Calculate the normalized cross-correlation value NCC(w) between the two regions = Σ[( - ()( - )] / [ × [×N], where , The average gray level of the region. , Let N be the standard deviation and N be the number of pixels in the region. The w corresponding to the maximum value of NCC(w) is the optimal overlap width. NCC computation is accelerated in parallel using GPUs (CUDA) to meet real-time requirements. Overlap ratio = / .

[0188] (c) Dynamic updates and structural adaptation: The overlap ratio estimate is updated every 1 meter along the tunnel longitudinal direction to obtain a sequence of how the overlap ratio of each pair of linear array cameras changes with longitudinal position. Based on the structural type label output in step S2, the shield tunnel section is... The sequence is stored in the shield overlap ratio mapping table. Storage of non-shield tunnel sections Within the transition zone, the overlap ratio of the two structures is linearly interpolated according to the proportion of the transition zone location: , where t is the normalized position (0~1) in the transition region to avoid splicing jumps caused by parameter abrupt changes.

[0189] Output: Overlap ratio mapping table for 8 pairs of adjacent linear scan cameras { } (i=1,...,8), where each overlap ratio is a proportion of the image pixel width (between 0 and 1), dynamically varying along the longitudinal position L. Independent mapping tables exist for shield tunneling and non-shield tunneling respectively. This result serves as input for step S4 to determine the boundaries of the overlapping region.

[0190] Step S4: Fine registration of circumferential images: Input data: The preprocessed image output from step S1 The overlap ratio mapping table {r_i(L)} is output in step S3.

[0191] Processing procedure: For each pair of adjacent linear scan cameras ( , In step S3, feature-level fine registration is performed within the overlapping area to eliminate geometric errors.

[0192] (a) Overlapping region cropping: Based on the overlap ratio output in step S3 and image width Calculate the width of overlapping pixels = × .from The right side cut-off width is area ,from The width of the left side is area .

[0193] (b) Feature extraction: In and SIFT feature points were extracted from each element. SIFT feature extraction parameters were: Octaves=4 scale-space layers, Scales=3 per scale group, contrast_threshold=0.04, edge_threshold=10, and initial sigma value of 1.6. Each SIFT feature point contains its position (x,y), scale sigma, principal orientation, and a 128-dimensional descriptor.

[0194] (c) Feature matching: for Each feature point in, In this method, the nearest and second nearest neighbor feature points are searched using a KD-tree, and the ratio of the nearest neighbor distance d1 to the second nearest neighbor distance d2, NNDR = d1 / d2, is calculated. A valid matching point pair is considered when NNDR < 0.75. The SIFT descriptor is scale-invariant and rotation-invariant, and has a certain robustness to grayscale differences between two images.

[0195] (d) Mismatch removal and transformation estimation: The RANSAC algorithm is used to iteratively estimate the similarity transformation matrix from the matching point pairs. (2×3 matrix, including translation) Rotation angle (Scaling factor s). RANSAC parameters: maximum number of iterations 1000, minimum of 3 pairs of points sampled per iteration, inlier threshold 3 pixels. In each iteration, the transformation residuals of all matching point pairs are calculated, the model with the most inliers is retained, and finally, least-squares fitting is performed again using all inliers. .

[0196] (e) Establishment of global coordinate system: Using the dome-shaped linear array camera C5 as the reference reference (reference camera), set the transformation matrix of C5. =I (identity matrix, i.e., no transformation). For the linear scan cameras C1~C4 to the left of C5, the global transformation matrix is ​​obtained by multiplying them sequentially from C5 to the left: , , ..., For the line scan cameras C6~C9 to the right of C5, the same principle applies to the rightward transition: , ..., The advantage of using C5 as the reference is that the line array camera at the dome has the most correct viewing angle and relatively uniform lighting, and the registration error propagates symmetrically to the left and right sides, avoiding excessive cumulative error on one side.

[0197] (f) Image resampling: based on the global transformation matrix Images from each linear scan camera Each pixel coordinate is obtained through Coordinates mapped to the global coordinate system , The system samples grayscale values ​​on the global canvas using bicubic interpolation to generate an aligned circular image. .

[0198] Output: Images of the nine linear scan cameras aligned in the global coordinate system. (i=1,...,9), and the position coordinates of the overlapping area of ​​each pair of adjacent linear scan cameras in the global coordinate system. This result serves as the input for illumination correction in step S5.

[0199] Step S5: Adaptive illumination correction and gain compensation: Input data: Aligned image output from step S4 The structure type label sequence output in step S2 is used to select the lighting parameter set of the corresponding structure.

[0200] Processing procedure: A global gain-bias illumination model (i.e., the correction model) is used for joint correction to eliminate illumination differences among the nine line scan cameras.

[0201] (a) Illumination model definition: for each linear scan camera Establish a gain-bias illumination correction model (i.e., the correction model): .in This is the gain coefficient (reflecting the differences in photosensitivity and light intensity of the line array camera). This is the bias coefficient (reflecting dark level offset and ambient light substrate difference). Corrected image. In the overlapping area, the grayscale should be consistent with the corrected image of the adjacent linear scan camera.

[0202] (b) Calculation of grayscale statistics in overlapping areas: For each pair of adjacent linear scan cameras ( , The overlapping regions of the global coordinate system are calculated separately. and gray average value in this area , and standard deviation , These statistics form the data foundation for subsequent optimization.

[0203] (c) Global Joint Optimization: With the optimization objective of minimizing the sum of squared grayscale differences after correction in the overlapping region for all 8 pairs of adjacent linear scan cameras, the illumination correction parameters of the 9 linear scan cameras are jointly estimated. , (i=1,...,9). The optimization objective function is: J=Sum_{i=1}^{8}Sum_{(x,y)inOmega_i}[ * (x,y)+ - * (x',y')- ]^2; Where Omega_i is the overlapping region of the i-th pair of adjacent linear scan cameras, and (x',y') is the adjacent image coordinate of (x,y) in the global coordinate system.

[0204] (d) Reference datum and constraints: The arch-top linear array camera C5 is used as the reference datum, and is fixed. =1.0、 =0 (i.e., C5 image not corrected). Add gain normalization constraint Sum( )=9 (i=1,...,9), to prevent all Simultaneous scaling leads to parameter degradation. This optimization problem is a quadratic programming problem with equality constraints, which is solved using the Lagrange multiplier method: Construct the Lagrange function L = J + ×(Sum( )-9), for each , By taking the partial derivative of lambda and setting it to zero, we obtain a system of linear equations, which can be solved by matrix inversion.

[0205] (e) Structural Adaptive Parameters: Based on the structural type label output in step S2, estimate the illumination correction parameters for shield tunneling and non-shield tunneling sections respectively. Shield tunnel segments have smooth surfaces and high reflectivity, resulting in a generally smaller gain coefficient alpha (<1.0); non-shield tunnels have rough surfaces and low reflectivity, resulting in a generally larger gain coefficient alpha (>1.0). The illumination parameter sets for the two structures are stored as { ^s, ^s} (shield tunnel) and { ^n, ^n} (non-shield tunnel), transition is performed by linear interpolation according to position ratio within the transition zone.

[0206] (f) Illumination correction application: Alignment images for each line scan camera Apply correction formula pixel by pixel (x,y)= × (x,y)+ Output the corrected image The pixel values ​​of the corrected image are cropped to the range [0, 255].

[0207] Output: Images corrected by 9 linear scan cameras (i=1,...,9), where the grayscale difference in the overlapping region has been minimized, and the corresponding lighting parameter set for the structural type { , This result serves as the input for step S6, multi-band fusion.

[0208] Step S6: Multi-band fusion and seamless splicing: Input data: The corrected image output from step S5 The global transformation matrix output in step S4 Coordinates of the overlapping area.

[0209] Processing procedure: Multi-band fusion is performed on the overlapping areas of adjacent cameras to eliminate residual stitching seams and preserve texture details.

[0210] (a) Construction of the Laplacian pyramid: for each corrected image Construct a 5-layered Laplace's Pyramid. Specific process: First, construct the Gaussian Pyramid: Layer 0. (Original image), kth layer pass Gaussian low-pass filter (5×5 Gaussian kernel, =1.0) then downsampled by 2 times, for a total of 5 layers. ~ Then construct the Laplace pyramid: the kth level. = (First upsample by 2 times, then subtract), 4th layer = (Lowest frequency). Layer 0 The highest frequency detail layer (containing textures and edges), layer 4. The lowest frequency illumination layer (including overall brightness and illumination trend).

[0211] (b) Low-frequency band linear weighted fusion: for the low-frequency layer of the pyramid and A distance-based linear weighted fusion method is employed. For each pixel (x, y), the distance to the left edge of its corresponding image is calculated. and distance to the center of the overlapping region fusion weight = / ( + The low-frequency fusion formula transitions linearly from 1 to 0 on both sides of the center of the overlapping region. Where k=3,4. This fusion ensures a smooth transition of low-frequency illumination components and eliminates residual illumination abrupt changes.

[0212] (c) High-frequency band gradient energy fusion: for the high-frequency layer of the pyramid ~ A fusion strategy based on local gradient energy is adopted. For each pixel (x, y) in the overlapping region, the Sobel gradient energy of the two images in the 3×3 neighborhood of that pixel is calculated. (x,y)=| (x,y)|+| (x,y)|and (x, y). Select the side with the larger gradient energy as the fusion result: when... (x,y)>= When (x,y) (x,y), otherwise (x,y)= (x,y), where k=0,1,2. This strategy preserves the side with larger gradient energy (i.e., clearer texture) and avoids high-frequency detail blurring caused by linear averaging.

[0213] (d) Pyramid Reconstruction: Starting from the lowest frequency layer F_4, upsampling is performed layer by layer (2x bicubic interpolation) and high-frequency layers are superimposed. The reconstruction formula is: (k=4,3,2,1), and finally reconstructed to obtain the fused ring stitched image I_stitch_ring.

[0214] (e) Circular stitching completed: The above fusion is performed sequentially on 8 pairs of adjacent line scan cameras to obtain a complete circular stitched image I_stitch_ring of 9 line scan camera images. This image covers the entire cross section of the U-shaped tunnel without obvious misalignment or illumination change boundaries.

[0215] Output: A large circumferentially seamless stitched image I_stitch_ring, and a record of the fusion of overlapping areas from each stitch (used for quality assessment in step S8). This result serves as input for the longitudinal stitching in step S7.

[0216] Step S7: Horizontal (i.e., longitudinal) encoder-assisted stitching: Input data: the large circular stitched image I_stitch_ring (multiple segments along the longitudinal direction) output in step S6, the encoder longitudinal position sequence L(t) output in step S1, and the longitudinal physical position corresponding to each row of images.

[0217] Processing procedure: Multiple circumferential stitched images acquired along the tunnel's longitudinal direction are sequentially stitched together into a complete large image according to their physical locations. Since the line scan cameras do not overlap longitudinally, the stitching relies on encoder positioning supplemented by motion compensation.

[0218] (a) Encoder position mapping: based on encoder pulse count P(t) and calibration parameters Calculate the vertical physical positions of the first and last rows of each circumferential stitched image. and .according to Sort the stitched images from smallest to largest to determine the stitching order. On the global vertical canvas, place the first row of each stitched image at y= / The pixel row position.

[0219] (b) Motion jitter compensation: Due to fluctuations in vehicle speed, the longitudinal position recorded by the encoder may deviate from the actual content of the image. At the longitudinal seam of the two stitched images, extract the average grayscale sequence of the last N=20 lines of the previous segment and the first N=20 lines of the next segment. and (n=1,...,20). Calculate the one-dimensional normalized cross-correlation of two sequences: CC( )= / ,search The largest CC in the range [-5, +5] . This is the vertical offset correction amount, used to fine-tune the starting row position of the next stitched image segment: = + This compensation eliminates line misalignment caused by speed fluctuations.

[0220] (c) Smoothing of longitudinal seams: At the longitudinal seams, a linear gradient weighted fusion is applied to the last 10 lines of the previous segment and the first 10 lines of the next segment. For the nth lines on both sides of the seam (n=1,...,10), the fusion weight w = n / 10, and the fusion formula is... This fusion eliminates abrupt changes in grayscale between lines, resulting in a natural transition between vertical stitching.

[0221] Output: A complete tunnel inspection map I_stitch_full (covering the entire cross-section and longitudinal length of the U-shaped tunnel), along with the location and offset correction records for each longitudinal joint. This result serves as input for the quality assessment in step S8.

[0222] Step S8: Assess and optimize stitching quality. Input data: the large circumferential stitched image I_stitch_ring and its overlapping area fusion record output in step S6, and the complete large image I_stitch_full and its longitudinal seam record output in step S7.

[0223] Processing procedure: Calculate multidimensional quality indicators for the splicing results. When the indicators are below the threshold, the corresponding parameter adjustment steps are automatically triggered.

[0224] (a) Evaluation of registration accuracy in overlapping areas: For the overlapping area of ​​each pair of adjacent linear scan cameras, from and Extract the overlapping region pixels and calculate the root mean square error (RMSE) = SSIM = structural similarity Where C1 = (0.01 × 255)^2, C2 = (0.03 × 255)^2. When RMSE > 5 gray levels or SSIM < 0.85, the registration of the line array cameras is deemed unqualified, triggering step S3 to expand the search half-width Delta to 0.12 and re-estimate the overlap ratio, and step S4 to relax the NNDR threshold to 0.80 and re-match.

[0225] (b) Evaluation of seam smoothness: Take 5 rows of pixels on each side of the circumferential seam and calculate the gradient difference. and grayscale transition .when Threshold T_G (taken as 10 gray levels / pixel) or When the gray level is >3, step S5 is triggered to re-optimize the lighting parameters. , }, and step S6 increases the number of low-frequency fusion layers (from 2 layers to 3 layers).

[0226] (c) Illumination consistency assessment: Calculate the grayscale mean curve mu_ring (I_stitch_ring) along the circumferential direction of the stitched large image. ), The circumferential angle is used. The fluctuation amplitude of the curve is calculated as Delta_mu = (max(mu_ring) - min(mu_ring)) / mean(mu_ring). When Delta_mu > 10%, step S5, global illumination parameters, is triggered to re-optimize, increasing the number of optimization iterations.

[0227] (d) Vertical joint quality assessment: For each vertical joint, calculate the difference in mean grayscale values ​​and the difference in texture features between the two sides of the joint. When the difference exceeds the threshold, trigger step S7 to increase the motion compensation search range and the number of smoothed joint rows.

[0228] (e) Feedback and Optimization Execution: Based on the above evaluation results, a gradual strategy is adopted to adjust the parameters. Each time, only the parameter corresponding to the indicator with the largest deviation is adjusted, and the corresponding steps are re-executed and evaluated again to avoid oscillations caused by adjusting multiple parameters simultaneously. A maximum of 3 iterations are performed. If the target is still not met, a quality warning flag is output to prompt manual review.

[0229] Output: A stitching quality assessment report (including RMSE and SSIM values ​​for each overlapping area, gradient differences and grayscale jump values ​​for each stitch, and overall illumination consistency index), and the final stitched image I_stitch_final after feedback adjustments. This report is used for quality documentation and subsequent maintenance decisions.

[0230] The U-shaped tunnel inspection image adaptive seamless stitching system includes the following modules: (1) Image acquisition module: includes 9 line scan cameras (numbered C1~C9, arranged circumferentially along the U-shaped tunnel), an incremental encoder, and a synchronization controller. The synchronization controller sends a synchronization trigger signal to ensure synchronous acquisition by the 9 line scan cameras, and the encoder outputs a pulse sequence for subsequent longitudinal positioning. Output: Original grayscale image and encoder pulse sequence P(t).

[0231] (2) Preprocessing module: Input And P(t), perform dark current correction, flat field correction, NLM denoising and distortion correction, and output the preprocessed image. And the corresponding vertical physical position L(t) for each row.

[0232] (3) Structure recognition module: Input preprocessed image Following fixed feature extraction parameters (LBP_u2 R=1 P=8 for 59-dimensional extraction, GLCM 64-level d=1 for 16-dimensional extraction in four directions, FFT 8-band extraction for 10-dimensional extraction, and grayscale statistics for 4-dimensional extraction, totaling 89 dimensions), after Z-score normalization, the data is input into a pre-trained SVM model, outputting a structure type label sequence {( The module stores the offline-trained SVM model file and feature normalization parameters.

[0233] (4) Overlap ratio estimation module: Input preprocessed image And structural labels, perform geometric coarse estimation → phase correlation coarse localization → NCC fine matching (CUDA acceleration), and output an overlap ratio mapping table of 8 pairs of adjacent linear array cameras { }, shield tunneling and non-shield tunneling are stored separately.

[0234] (5) Image registration module: Input preprocessed image Based on the overlap ratio mapping table, SIFT feature points are extracted in the overlapping region → NNDR matching → RANSAC estimation of the similarity transformation matrix, a global coordinate system is established with C5 as the reference, and the aligned image is output. .

[0235] (6) Illumination correction module: Input aligned image And structural labels, calculate the gray-scale statistics of overlapping regions, and jointly optimize the { gain-bias model} , Output corrected image Parameter sets are established separately for shield tunneling and non-shield tunneling.

[0236] (7) Fusion and stitching module: Input the corrected image and global transformation matrix A five-layer Laplacian pyramid is constructed, and low-frequency linear weighted fusion and high-frequency gradient energy fusion are used to reconstruct the output cyclic seamless stitched large image I_stitch_ring.

[0237] (8) Longitudinal stitching module: Input the circumferential stitching large image and encoder position sequence, perform encoder position mapping → one-dimensional cross-correlation motion jitter compensation → longitudinal joint linear gradient smoothing, and output the complete tunnel large image I_stitch_full.

[0238] (9) Quality assessment feedback module: Input the stitched large image and fusion record, calculate RMSE, SSIM, gradient difference, grayscale jump and illumination consistency index, when the values ​​are below the threshold, feed back parameter adjustment instructions to the corresponding module, and output quality assessment report and final stitched large image I_stitch_final.

[0239] The above modules are implemented through computer programs and run on industrial control computers or servers equipped with GPUs. Among them, the SVM inference of the structure recognition module, the NCC calculation of the overlap ratio estimation module, and the Laplace pyramid decomposition and reconstruction of the fusion and stitching module are all accelerated using CUDA.

[0240] Optionally, (1) in the precise estimation of overlap ratio, NCC can be replaced with Mutual Information (MI), phase-correlated subpixel interpolation, or deep learning-based feature matching methods, such as SuperGlue. Phase correlation can also be replaced with ORB feature matching for coarse localization; (2) in feature extraction and matching, SIFT can be replaced with traditional feature points such as SURF, ORB, and AKAZE, or deep learning feature points such as SuperPoint, D2-Net, and R2D2. The corresponding NNDR threshold and RANSAC parameters need to be adjusted synchronously; (3) in the feature extraction of structural recognition, LBP can be replaced with rotation-invariant uniform mode LBP (LBP_riu2) or circular LBP. R and P can take other values ​​(such as R=2, P=16), but it is necessary to ensure that offline training and online inference use the exact same parameters. The gray level of GLCM can be compressed to 32 or 128 levels, the distance can be calculated as d=2, and the direction can be increased to 8 directions. SVM can be replaced with a lightweight CNN (such as MobileNetV3 fine-tuning) or a Transformer classifier, but the input feature dimension must match the model; (4) In illumination correction, the gain-bias model can be replaced with histogram matching, multi-scale Retinex (MSR) correction based on Retinex theory, or image enhancement methods based on deep learning; (5) In fusion strategies, Laplacian pyramid fusion can be replaced with Poisson fusion, gradient domain fusion, or image fusion methods based on deep learning (such as seamless fusion based on GAN). The number of pyramid layers can be adjusted to 3~7 layers, and the low-frequency / high-frequency fusion boundary layer can be adjusted; (6) In the selection of global reference cameras, the dome C5 can be replaced with other linear array cameras as reference benchmarks, or a global Bundle Adjustment without reference can be used to jointly optimize all camera parameters; (7) In vertical stitching, an IMU (Inertial Measurement Unit) can be added to the encoder to assist in localization, or visual odometry can be used for motion estimation compensation. The number of cross-correlation rows and the search range for motion compensation can be adjusted according to vehicle speed.

[0241] In this embodiment, it can adapt to tunnel cross-sections of two different structural types, shield tunneling and non-shield tunneling, and handle complex working conditions where the overlap ratio of each circumferential linear array camera is different and the lighting conditions are significantly different. It outputs high-quality tunnel inspection images that cover the entire cross-section and length without obvious misalignment or the boundary of lighting changes, providing reliable image data for tunnel defect detection.

[0242] Based on the methods described in any of the above embodiments, this application also provides, as follows: Figure 2 The diagram shows the structure of an electronic device. Figure 2 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the methods described in any of the above embodiments.

[0243] Based on the methods described in any of the above embodiments, this application also provides a computer storage medium storing a computer program, which, when executed by a processor, can be used to perform the methods described in any of the above embodiments.

[0244] Based on the methods described in any of the above embodiments, this application also provides a computer program product, which includes one or more computer programs or instructions. The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. When executed by a processor, the computer program implements the methods described in any of the above embodiments.

[0245] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0246] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0247] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0248] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0249] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0250] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. An image stitching method, characterized in that, The method includes: Acquire tunnel lining images and encoder output pulse data simultaneously from multiple linear array cameras arranged circumferentially along the tunnel. Based on the pulse data, calculate the longitudinal physical location of the tunnel corresponding to each row of the tunnel lining image; The tunnel lining image is used to identify the tunnel structure type and determine the tunnel structure type to which the current image segment belongs; Based on the tunnel structure type, the overlap ratio between circumferentially adjacent line scan cameras is determined; Based on the overlap ratio, the overlapping area between the images acquired by adjacent circumferential line scan cameras is determined; The overlapping areas are image registered, and the images acquired by each line scan camera are aligned to a unified global coordinate system to obtain an aligned image; Based on the tunnel structure type, determine the lighting parameter set corresponding to the tunnel structure type; The aligned image is then illuminated using the illumination parameter set to obtain the corrected image. The corrected image is fused in multiple frequency bands within the overlapping area to obtain a seamless circumferential stitched image. Based on the pulse data and the longitudinal physical location of the tunnel, the multiple segments of the seamless circumferential image collected along the longitudinal direction of the tunnel are longitudinally stitched together according to their physical locations to obtain a complete tunnel inspection image.

2. The method according to claim 1, characterized in that, The step of identifying the tunnel structure type of the tunnel lining image and determining the tunnel structure type of the current image segment includes: Multiple image features are extracted from the current image segment and concatenated into a feature vector. The current feature vector is then normalized using preset normalization parameters. These multiple image features include gray-level statistical features, local texture features, gray-level co-occurrence matrix texture features, and frequency domain features. The gray-level statistical features include gray-level mean, standard deviation, skewness, and kurtosis. The local texture features are extracted using a local binary mode, with a radius of a first preset value and a sampling number of a second preset value. The gray-level co-occurrence matrix texture features are used to calculate contrast, energy, homogeneity, and correlation in different directions. The frequency domain features are used to extract the energy distribution and dominant frequency information of each frequency band after performing a Fourier transform on the current image segment. The normalized feature vector is input into the classifier model to obtain the structure type label of the current image segment. The classifier model is trained through the following steps: acquiring tunnel lining image samples with known structure types and labeling them with structure type labels; extracting multi-class image features from each tunnel lining image sample and concatenating them into a feature vector; normalizing the feature vectors of all samples in the training set; using the normalized feature vector as input and the labeled structure type label as output to train the initial classifier model, thereby obtaining the trained classifier model and normalization parameters.

3. The method according to claim 1, characterized in that, The determination of the overlap ratio between circumferentially adjacent linear scan cameras based on the tunnel structure type includes: Based on the installation calibration parameters of each linear array camera and the tunnel cross-sectional profile data, the theoretical overlap ratio of each adjacent linear array camera pair is calculated by geometric projection. Within the neighborhood of the theoretical overlap ratio, the edge stripes of adjacent linear scan camera images are coarsely located to obtain the initial overlap width; Traverse candidate overlap widths around the initial overlap width and calculate the normalized cross-correlation value of the overlapping region of adjacent linear array cameras under each candidate overlap width; The overlap width corresponding to the maximum normalized cross-correlation value is taken as the target overlap width, and the overlap ratio is calculated based on the ratio of the target overlap width to the image width.

4. The method according to claim 1, characterized in that, The step of image registration of the overlapping region, aligning the images acquired by each line scan camera to a unified global coordinate system to obtain an aligned image, includes: Using the line scan camera at the arch position as a reference, the transformation matrix of the line scan camera at the arch position is used as the identity matrix. For each line scan camera on one side of the line scan camera at the arch position, the global transformation matrix of each line scan camera is obtained by multiplying it by the inverse of the transformation matrix of the adjacent line scan camera in turn. For each line scan camera on the other side of the line scan camera at the arch position, the global transformation matrix of each line scan camera is obtained by multiplying the transformation matrix of the adjacent line scan cameras by the right in sequence. Based on the global transformation matrix of each line scan camera, the images acquired by each line scan camera are transformed into the global coordinate system to obtain the aligned image.

5. The method according to claim 1, characterized in that, The step of performing illumination correction on the aligned image using the illumination parameter set to obtain the corrected image includes: The alignment images of each line scan camera are corrected using a correction model. The pixel values ​​of the corrected image of each line scan camera are obtained by linearly transforming the pixel values ​​of the alignment image based on the gain coefficient and bias coefficient of the line scan camera. Using the line scan camera at the arch position as a reference, the gain coefficient and offset coefficient of the line scan camera at the arch position are fixed. The optimization objective is to minimize the sum of squared grayscale differences of all adjacent linear scan camera pairs after correction in the overlapping region. Under the constraint of gain coefficient normalization, the gain coefficient and bias coefficient of all linear scan cameras are solved jointly. The aligned images of each line scan camera are corrected based on the obtained gain coefficients and bias coefficients of each line scan camera to obtain the corrected images.

6. The method according to claim 1, characterized in that, The step of fusing the corrected image into multiple frequency bands within the overlapping area to obtain a seamless circumferential stitched image includes: The corrected images from each linear array camera are used to construct multi-layer pyramids, and each corrected image is decomposed into components of different frequency bands. For low-frequency components, a fusion weight is determined within the overlapping region based on the distance from each pixel position to the center position of the overlapping region. The fusion weight is then used to perform weighted fusion of the pixel values ​​of the corrected images of two adjacent line scan cameras within the overlapping region. For high-frequency components, the gradient energy in the neighborhood of each pixel is calculated within the overlapping area, and the pixel value of the corrected image on the side with larger gradient energy is determined as the fusion result at the pixel location. The fused frequency band components are reconstructed layer by layer to obtain the seamless circumferential stitched image.

7. The method according to claim 1, characterized in that, Based on the pulse data and the longitudinal physical location of the tunnel, the multiple segments of the seamless circumferential image acquired along the longitudinal direction of the tunnel are longitudinally stitched together according to their physical locations to obtain a complete tunnel inspection image, including: Based on the pulse data and calibration coefficients, the starting position of each segment of the seamless circumferential stitched image on the vertical canvas is determined. At the vertical seam of two adjacent seamlessly stitched circumferential images, extract the first row of grayscale mean sequence at the end of the previous segment (preset number of rows) and the second row of grayscale mean sequence at the beginning of the next segment (preset number of rows). Calculate the normalized cross-correlation values ​​of the first row grayscale mean sequence and the second row grayscale mean sequence at different offsets; The offset corresponding to the maximum normalized cross-correlation value is used as the vertical offset correction amount; Based on the offset correction amount, the starting row position of the next segment of the circumferential seamless stitched image is finely adjusted; At the longitudinal joint, the preset number of rows at the end of the previous segment and the preset number of rows at the beginning of the next segment are linearly weighted and fused to obtain the complete tunnel inspection image.

8. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store processor-executable instructions; Wherein, when the processor invokes the executable instructions, it implements the method described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the method described in any one of claims 1-7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1-7.

Images