Bridge beam bottom full-coverage live-action image generation method
Through image stitching and feature point matching technology, a full-coverage real-life image of the bridge beam bottom was generated, solving the problem of experience dependence and incomplete coverage in traditional detection methods, providing high-quality data support, and providing important significance for bridge safety inspection.
Patent Information
- Application Number
- CN202310610242.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-29
- Publication Date
- 2024-12-03
AI Technical Summary
The existing technology is difficult to generate real-life images of bridge beam bottoms. Traditional manual detection relies on experience and cannot form panoramic images. UAV detection has problems such as flight control and incomplete image coverage, making it difficult to achieve long-term research and disease detection.
The image stitching method is used to process and match feature points through recursive method, combined with denoising processing, gamma correction and linear fusion to generate a full-coverage real-life image of the bridge beam bottom, which is suitable for bridges in different structures.
The real-life image of the bridge beam bottom is generated, providing visual data support for the bottom of the bridge, improving the accuracy and reliability of detection, and is suitable for bridges of different structures and sizes.
Smart Images

Figure CN119067840A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of bridge detection, and in particular relates to a method for generating a full-coverage real-scene image of a bridge beam bottom. Background Art
[0002] As a country with a large infrastructure, my country has 912,800 highway bridges by 2020, with an annual increase of 20,000 to 30,000 bridges from 2016 to 2020. As the number of years of traffic increases, the complex natural environment and the continuous vehicle loads will cause the bridge structure to suffer varying degrees of damage. Therefore, it is necessary to conduct regular inspections of bridges to understand the development of bridge damage.
[0003] Traditionally, bridge beam bottom inspections are done manually, by observing beam bottom defects and taking pictures. However, the basis for judgment is mainly the personal experience of engineers, which is difficult to quantify, and it is impossible to form a panoramic view of the beam bottom for follow-up research. In addition, drone detection also has problems such as difficult flight control, difficult upward shooting, and incomplete images. How to conveniently generate a full-coverage real-life image of the bridge beam bottom for long-term research and use is a problem that needs to be solved at present. Summary of the invention
[0004] The present invention provides a method for generating a real-life image of the full coverage of the bottom of a bridge beam. The method can more conveniently obtain a real-life image of the full coverage of the bottom of the beam, providing data support for subsequent inspection of beam bottom defects and visualization through images.
[0005] In order to achieve the above-mentioned object of the invention, the technical solution provided by the present invention is as follows:
[0006] A method for generating a full-coverage real-scene image of a bridge beam bottom, characterized in that it comprises the following steps:
[0007] S1: sequentially collect n images covering the bottom of all bridge beams;
[0008] S2: Image stitching sequence, stitching planning for the acquired image set, using recursive processing:
[0009] Assume the total number n, the splicing method is:
[0010] (1) If n = 1, the spliced image is this image;
[0011] (2) If n = 2k, k∈Z +, then the first and second images are spliced to obtain image (1, 2), and image (1, 2) and the third image are spliced to obtain image (1, 3), until the spliced image of the first to k-th images is obtained, called image (1, k), and the spliced image of the k+1-th to 2k-th images obtained by the same splicing method is spliced to obtain the spliced image of n images, called image (1, 2k);
[0012] (3) If n = 2k + 1, k∈Z + , then the first and second images are spliced to obtain image (1, 2), and image (1, 2) is spliced with the third image to obtain image (1, 3), until the spliced image of the first to k-th images is obtained, to obtain image (1, k), and the k+1 to 2k-th images are spliced in the same way to obtain image (k+1, 2k), and image (k+1, 2k) and image (2k+1) are spliced to obtain image (k+1, 2k+1), and image (1, k) and image (k+1, 2k+1) are spliced to obtain image (1, 2k+1);
[0013] S3: The processing steps for stitching two images are as follows:
[0014] (1) De-noising: When two images are stitched together, median filtering is used to denoise the image.
[0015] (2) Gamma correction: Gamma correction is used to adjust the grayscale distribution of the image to obtain the processed image;
[0016] (3) Feature point extraction: Extract feature points from the first two images before stitching and use an adaptive non-maximum suppression algorithm to screen all identified feature points, describe all feature points, perform moderate Gaussian blur on the image, take a certain pixel area with the feature point as the center, downsample it to generate a high-dimensional normalized vector, use this vector as the feature descriptor of the feature point, and complete the description of all extracted feature points of the two images;
[0017] (4) Feature point matching: Using the RANSAC (Random Sample Consensus) algorithm, a certain number of feature points are extracted from the first image each time, and the corresponding paired feature points are found in the second image through the feature descriptor. The homography matrix of the second image relative to the first image is calculated, and the number of paired points is counted. This step is repeated N times to obtain the homography matrix with the most paired points, which is used as the optimal perspective transformation matrix.
[0018] (5) Image fusion: Based on the optimal perspective transformation matrix, the two images are aligned in the overlapping area and linear fusion is used to achieve a natural transition in the overlapping area.
[0019] S4: Follow the above image stitching sequence and processing method until the last image is obtained, which is a real scene image that fully covers the bottom of the bridge beam.
[0020] Preferably, the denoising process in step S3 is as follows:
[0021] (1) For a certain pixel position in the image, use a square moving window with a side length of 5 to make the center coincide with it;
[0022] (2) Read the grayscale values of the image pixels under the moving window and sort them in positive order;
[0023] (3) Assign the middle value in the grayscale sequence to the current center pixel.
[0024] Preferably, the gamma correction method in step S3 is:
[0025] A nonlinear operation is performed on the grayscale value of the input image so that the grayscale value of the output image is exponentially related to the input grayscale. The two follow the following relationship:
[0026]
[0027] In the above formula, V out is the output grayscale value, A is the adjustment coefficient, which needs to be determined by experiment, V in is the input / actual grayscale value, and γ represents the gamma parameter, which needs to be adjusted through actual measurement.
[0028] Preferably, the step of using the adaptive non-maximum suppression algorithm to select feature points in step S3 is:
[0029] First calculate the autocorrelation matrix:
[0030]
[0031] In the above formula, I x (x) and I y (x) represents the partial derivative of the gray value in the x and y directions, reflecting the gray change; ω(x, y) is the Gaussian window function, which is used to smooth the noise, and the formula is:
[0032]
[0033] Then the corner point response function is calculated as:
[0034] R(x,y)=det(M)-ktr 2 (M) = λ 1 λ 2 -k(λ 1 +λ 2 ) 2
[0035] Where k is a constant, usually 0.04. 1 With λ 2 is the characteristic value.
[0036] You can select and sort the feature points that satisfy the following formula:
[0037] R(x,y) i >R(x,y) j , (x, y) j ∈(x, y) i±r .
[0038] Where r represents the suppression radius, which needs to be determined according to the needs. The feature points selected by the above formula are within the suppression radius, and the maximum point is the maximum value. Adjust the suppression radius until the number of filtered feature points drops to the expected value.
[0039] Preferably, the feature point extraction and matching method in step S3 is:
[0040] For the two images to be stitched, after finding their feature points, take a (2N+1)*(2N+1) size correlation window with each feature point as the center in the two images, and then use each feature point in the reference image as the reference point to find the corresponding feature point in the image to be stitched. Image feature registration is achieved by calculating the correlation coefficient between the feature point correlation windows:
[0041]
[0042] In the formula, I and I′ represent the grayscale values of the two images. and Represents the grayscale average of all pixels in the two images. Calculate all the correlation values R above and perform threshold processing. The threshold is set to 0.8. R exceeding 0.8 is selected as one of the candidate matching points. The feature point pair with the largest correlation value R is selected as the matching point.
[0043] Preferably, the method for calculating the homography matrix of the second image relative to the first image in S3 is:
[0044] (1) From the matched feature point pair set P = {(p 1 , p 1 ′), (p 2 , p 2 ′),…,(p n , p n ′)} randomly select four groups of feature points to ensure that these four groups of feature points are not collinear. Calculate the initial homography matrix, denoted as For the feature point p(x i ,y i ) and p′(x i ′,y i′), so that it satisfies:
[0045]
[0046] (2) Using the initial homography matrix Calculate the transformation error for all matching points:
[0047]
[0048] If the transformation error E is less than the set threshold, the feature point is marked as an internal point, otherwise it is marked as an external point.
[0049] (3) Recalculate the homography matrix and repeat the above steps until the number of iterations reaches the maximum. The homography matrix with the largest number of inliers is the optimal model.
[0050] Preferably, the image fusion method in step S3 is:
[0051] Align the overlapping areas and use linear fusion to achieve a natural transition. Calculate the gray value of the image after fusion according to the following formula:
[0052] g(x, y) = αI 1 (x,y)+(1-α)I 2 (x, y)
[0053] Where g(x, y) represents the gray value of the fused image, I 1 (x, y) and I 2 (x, y) represents the grayscale value of the pixel of the two images before fusion, and α represents the linear fusion coefficient, which can be determined by yourself and can be set to 0.5 here.
[0054] Compared with the prior art, the present invention has the following beneficial effects:
[0055] The present invention can more conveniently obtain a real-life image of the full coverage of the beam bottom, without being restricted by the specific bridge structure form and size, and provides data support for subsequent inspection of beam bottom defects and visualization through images, which is of great significance to bridge safety. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 It is a schematic diagram of T-beam bridge structure and image acquisition.
[0057] Figure 2 It is a fully covered real scene image of the bottom of a T-beam bridge generated by the present invention.
[0058] Figure 3 It is a detailed picture of a fully covered real-life image of the bottom of a T-beam bridge generated by the present invention.
[0059] Figure 4 It is a schematic diagram of the hollow slab girder bridge structure and image acquisition.
[0060] Figure 5 It is a fully covered real scene image of the bottom of a hollow slab girder bridge generated by the present invention.
[0061] In the figure: 1. T-beam bridge; 11. Bottom of T-beam bridge; 2. Bridge inspection vehicle; 3. Hollow slab bridge; 31 Bottom of hollow slab bridge. DETAILED DESCRIPTION
[0062] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0063] The technical solution of the present invention is applicable to bridges of different structures. The following describes the implementation method in detail by taking a T-beam bridge and a hollow slab beam bridge as examples.
[0064] Example 1
[0065] A method for generating a full-coverage real-life image of the bottom of a bridge beam, combined with Figure 1 The schematic diagram of the T-beam bridge structure and image acquisition is shown in FIG. 1 , a T-beam bridge 1, a bridge inspection vehicle 2, and a T-beam bridge bottom 11. First, n images covering the entire bridge bottom are sequentially acquired. In this embodiment, an image acquisition device is provided on the bridge inspection vehicle 2, and the mobile bridge inspection vehicle 2 sequentially acquires n images covering the entire T-beam bridge bottom 11. Image acquisition can also be performed by other methods such as drones.
[0066] Then clarify the image stitching method, that is, the order, the image stitching order, and plan the stitching of the collected image set, using the recursive method:
[0067] Assume the total number n, the splicing method is:
[0068] (1) If n = 1, the spliced image is this image;
[0069] (2) If n = 2k, k∈Z + , then the first and second images are spliced to obtain image (1, 2), and image (1, 2) and the third image are spliced to obtain image (1, 3), until the spliced image of the first to k-th images is obtained, called image (1, k), and the spliced image of the k+1-th to 2k-th images obtained by the same splicing method is spliced to obtain the spliced image of n images, called image (1, 2k);
[0070] (3) If n = 2k + 1, k∈Z +, then the first and second images are spliced to obtain image (1, 2), and image (1, 2) is spliced with the third image to obtain image (1, 3), until the spliced image of the first to k-th images is obtained, and image (1, k) is obtained. The k+1 to 2k-th images are spliced in the same way to obtain image (k+1, 2k), and image (k+1, 2k) and image (2k+1) are spliced to obtain image (k+1, 2k+1), and image (1, k) and image (k+1, 2k+1) are spliced to obtain image (1, 2k+1).
[0071] Among them, the processing steps for stitching every two images are as follows:
[0072] (1) De-noising: When two images are stitched together, median filtering is used to denoise the images. The specific steps are as follows:
[0073] ① For a certain pixel position in the image, use a square moving window with a side length of 5 to make the center coincide with it;
[0074] ② Read the grayscale values of the image pixels under the moving window and sort them in positive order;
[0075] ③Assign the middle value in the grayscale sequence to the current center pixel.
[0076] (2) Gamma correction: Gamma correction is used to adjust the grayscale distribution of the image to obtain the processed image. The specific method is as follows:
[0077] A nonlinear operation is performed on the grayscale value of the input image so that the grayscale value of the output image is exponentially related to the input grayscale. The two follow the following relationship:
[0078]
[0079] In the above formula, V out is the output grayscale value, A is the adjustment coefficient, which needs to be determined by experiment, V in is the input / actual grayscale value, and γ represents the gamma parameter, which needs to be adjusted through actual measurement.
[0080] (3) Feature point extraction: Extract feature points from the first two images before stitching and use an adaptive non-maximum suppression algorithm to screen all identified feature points, describe all feature points, perform moderate Gaussian blur on the image, take a certain pixel area with the feature point as the center, downsample it to generate a high-dimensional normalized vector, use this vector as the feature descriptor of the feature point, and complete the description of all extracted feature points of the two images;
[0081] The steps of selecting feature points using the adaptive non-maximum suppression algorithm are as follows:
[0082] First calculate the autocorrelation matrix:
[0083]
[0084] In the above formula, I x (x) and I y (x) represents the partial derivative of the gray value in the x and y directions, reflecting the gray change; ω(x, y) is the Gaussian window function, which is used to smooth the noise, and the formula is:
[0085]
[0086] Then the corner point response function is calculated as:
[0087] R(x,y)=det(M)-ktr 2 (M) = λ 1 λ 2 -k(λ 1 +λ 2 ) 2
[0088] Where k is a constant, usually 0.04. 1 With λ 2 is the characteristic value.
[0089] You can select and sort the feature points that satisfy the following formula:
[0090] R(x,y) i >R(x,y) j , (x, y) j ∈(x, y) i±r .
[0091] Where r represents the suppression radius, which needs to be determined according to the needs. The feature points selected by the above formula are within the suppression radius, and the maximum point is the maximum value. Adjust the suppression radius until the number of selected feature points drops to the expected value.
[0092] (4) Feature point matching: Using the RANSAC (Random Sample Consensus) algorithm, a certain number of feature points are extracted from the first image each time, and the corresponding paired feature points are found in the second image through the feature descriptor. The homography matrix of the second image relative to the first image is calculated, and the number of paired points is counted. This step is repeated N times to obtain the homography matrix with the most paired points, which is used as the optimal perspective transformation matrix.
[0093] Feature point extraction and matching method:
[0094] For the two images to be stitched, after finding their feature points, take a (2N+1)*(2N+1) size correlation window with each feature point as the center in the two images, and then use each feature point in the reference image as the reference point to find the corresponding feature point in the image to be stitched. Image feature registration is achieved by calculating the correlation coefficient between the feature point correlation windows:
[0095]
[0096] In the formula, I and I′ represent the grayscale values of the two images. and Represents the grayscale average of all pixels in the two images. Calculate all the correlation values R above and perform threshold processing. The threshold is set to 0.8. R exceeding 0.8 is selected as one of the candidate matching points. The feature point pair with the largest correlation value R is selected as the matching point.
[0097] The homography matrix of the second image relative to the first image is calculated as:
[0098] ① From the matched feature point pair set P = {(p 1 , p 1 ′), (p 2 , p 2 ′),…,(p n , p n ′)} randomly select four groups of feature points to ensure that these four groups of feature points are not collinear. Calculate the initial homography matrix, denoted as For the feature point p(x i ,y i ) and p′(x i ′,y i ′), so that it satisfies:
[0099]
[0100] ②Use the initial homography matrix Calculate the transformation error for all matching points:
[0101]
[0102] If the transformation error E is less than the set threshold, the feature point is marked as an internal point, otherwise it is marked as an external point.
[0103] ③ Recalculate the homography matrix and repeat the above steps until the number of iterations reaches the maximum. The homography matrix with the largest number of internal points is the optimal model.
[0104] (5) Image fusion: Based on the optimal perspective transformation matrix, the two images are aligned in the overlapping area and linear fusion is used to achieve a natural transition in the overlapping area.
[0105] Image fusion method:
[0106] Align the overlapping areas and use linear fusion to achieve a natural transition. Calculate the grayscale value of the image after fusion according to the following formula:
[0107] g(x, y) = αI 1 (x,y)+(1-α)I 2 (x, y)
[0108] Where g(x, y) represents the gray value of the fused image, I 1 (x, y) and I 2 (x, y) represents the grayscale value of the pixel of the two images before fusion, and α represents the linear fusion coefficient, which can be determined by yourself and can be set to 0.5 here.
[0109] Follow the above image stitching sequence and processing method until the final image is obtained, which is a real-life image that fully covers the bottom of the bridge beam. Figure 2 It is a fully covered real scene image of the bottom of a T-beam bridge generated by the present invention. Figure 3 It is a detailed picture of a fully covered real-life image of the bottom of a T-beam bridge generated by the present invention.
[0110] Example 2
[0111] A method for generating full coverage real scene images of the bottom of a hollow slab beam bridge, combined with Figure 4 Schematic diagram of hollow slab girder bridge structure and image acquisition, bridge inspection vehicle 2, hollow slab girder bridge 3, hollow slab girder bridge bottom 31, firstly, n images covering all bridge girder bottoms are collected in sequence, in this embodiment, an image acquisition device is set on the bridge inspection vehicle 2, and the mobile bridge inspection vehicle 2 collects n images covering all hollow slab girder bridge bottoms 31 in sequence. Image acquisition can also be performed by other methods such as drones.
[0112] Then, the same image stitching sequence and processing method as in Example 1 are followed until the last image is obtained, which is a real-life image that fully covers the bottom of the bridge beam. Figure 5 It is a fully covered real scene image of the bottom of a hollow slab girder bridge generated by the present invention.
[0113] It can be seen from the above embodiments that the present invention can more conveniently obtain a real-life image of the full coverage of the beam bottom, is not limited by the specific bridge structure form and size, and the generated image quality and restoration degree are very high.
Claims
1. A method for generating a real-life image with full coverage of the bottom of a bridge beam. It is characterized in that The following steps are involved: S1: sequentially collect n images covering the bottom of all bridge beams; S2: Image stitching sequence, stitching planning for the acquired image set, using recursive processing: Assume the total number n, the splicing method is: (1) If n = 1, the spliced image is this image; (2) If n = 2k, k∈Z + , then the first and second images are spliced to obtain image (1, 2), and image (1, 2) is spliced with the third image to obtain image (1, 3), until the spliced image of the first to k-th images is obtained, called image (1, k), and the spliced image of the k+1-th to 2k-th images obtained by the same splicing method is spliced together to obtain the spliced image of n images, called image (1, 2k); (3) If n = 2k + 1, k∈Z + , then the first and second images are spliced to obtain image (1, 2), and image (1, 2) is spliced with the third image to obtain image (1, 3), until the spliced image of the first to k-th images is obtained, to obtain image (1, k), and the k+1 to 2k-th images are spliced in the same way to obtain image (k+1, 2k), and image (k+1, 2k) and image (2k+1) are spliced to obtain image (k+1, 2k+1), and image (1, k) and image (k+1, 2k+1) are spliced to obtain image (1, 2k+1); S3: The processing steps for stitching two images are as follows: (1) De-noising: When two images are stitched together, median filtering is used to denoise the image. (2) Gamma correction: Gamma correction is used to adjust the grayscale distribution of the image to obtain the processed image; (3) Feature point extraction: Extract feature points from the first two images before stitching and use an adaptive non-maximum suppression algorithm to screen all identified feature points, describe all feature points, perform moderate Gaussian blur on the image, take a certain pixel area with the feature point as the center, downsample it to generate a high-dimensional normalized vector, use this vector as the feature descriptor of the feature point, and complete the description of all extracted feature points of the two images; (4) Feature point matching: Using the RANSAC (Random Sample Consensus) algorithm, a certain number of feature points are extracted from the first image each time, and the corresponding paired feature points are found in the second image through the feature descriptor. The homography matrix of the second image relative to the first image is calculated, and the number of paired points is counted. This step is repeated N times to obtain the homography matrix with the most paired points, which is used as the optimal perspective transformation matrix. (5) Image fusion: Based on the optimal perspective transformation matrix, the two images are aligned in the overlapping area and linear fusion is used to achieve a natural transition in the overlapping area. S4: Follow the above image stitching sequence and processing method until the last image is obtained, which is a real scene image that fully covers the bottom of the bridge beam.
2. The method for generating a full-coverage real-scene image of the bridge beam bottom according to claim 1, It is characterized in that The detailed steps of the denoising process in step S3 are as follows: (1) For a certain pixel position in the image, use a square moving window with a side length of 5 to make the center coincide with it; (2) Read the grayscale values of the image pixels under the moving window and sort them in positive order; (3) Assign the middle value in the grayscale sequence to the current center pixel.
3. The method for generating a full-coverage real-scene image of the bridge beam bottom according to claim 1, It is characterized in that The gamma correction method in step S3 is: A nonlinear operation is performed on the grayscale value of the input image so that the grayscale value of the output image is exponentially related to the input grayscale. The two follow the following relationship: In the above formula, γ represents the gamma parameter, which needs to be adjusted through actual measurement.
4. The method for generating a full-coverage real-scene image of the bridge beam bottom according to claim 1, It is characterized in that The steps of using the adaptive non-maximum suppression algorithm to select feature points in step S3 are: First calculate the autocorrelation matrix: In the above formula, I x (x) and I y (x) represents the partial derivative of the gray value in the x and y directions, reflecting the gray change; ω(x, y) is the Gaussian window function, which is used to smooth the noise, and the formula is: Then the corner point response function is calculated as: R(x,y)=det(M)-ktr 2 (M)=λ 1 l 2 -k(λ 1 +λ 2 ) 2 You can select and sort the feature points that satisfy the following formula: R(x,y) i >R(x,y) j ,(x,y) j ∈(x,y) i±r Where r represents the suppression radius, which needs to be determined according to specific needs. The feature points selected by the above formula are within the suppression radius range, and the maximum point is the maximum value. The suppression radius is adjusted until the number of screened feature points drops to the expected value.
5. The method for generating a full-coverage real-scene image of the bridge beam bottom according to claim 1, It is characterized in that The feature point extraction and matching method in step S3: For the two images to be stitched, after finding their feature points, take a (2N+1)*(2N+1) size correlation window with each feature point as the center in the two images, and then use each feature point in the reference image as the reference point to find the corresponding feature point in the image to be stitched, and realize image feature registration by calculating the correlation coefficient between the feature point correlation windows: In the formula, I and I′ represent the grayscale values of the two images. and Represents the grayscale average of all pixels in the two images, calculates all the above correlation values R and performs threshold processing, and selects the feature point pair with the largest correlation value R as the matching point.
6. The method for generating a full-coverage real-scene image of the bridge beam bottom according to claim 1, It is characterized in that The method for calculating the homography matrix of the second image relative to the first image in step S3 is: (1) From the matched feature point pair set P = {(p 1 ,p 1 ′),(p 2 ,p 2 ′),…,(p n ,p n ′)} randomly select four groups of feature points to ensure that these four groups of feature points are not collinear; calculate the initial homography matrix, recorded as For the feature point p(x i ,y i ) and p′(x i ′,y i ′), so that it satisfies: (2) Using the initial homography matrix Calculate the transformation error for all matching points: If the transformation error E is less than the set threshold, the feature point is marked as an internal point, otherwise it is marked as an external point; (3) Recalculate the homography matrix and repeat the above steps until the number of iterations reaches the maximum. The homography matrix with the largest number of inliers is the optimal model.
7. The method for generating a full-coverage real-scene image of the bridge beam bottom according to claim 1, It is characterized in that The image fusion method in step S3: Align the overlapping areas, use linear fusion method to complete the natural transition of the overlapping areas, and calculate the gray value of the image after fusion according to the following formula: g(x,y)=αI 1 (x,y)+(1-α)I 2 (x,y) Where g(x,y) represents the gray value of the fused image, I 1 (x,y) and I 2 (x, y) represents the grayscale value of the pixel of the two images before fusion, and α represents the linear fusion coefficient.
