A shield segment joint opening measurement method based on image recognition technology
By using the Transformer model based on image recognition technology, the joints of tunnel segments can be automatically detected, solving the problems of cumbersome detection and large errors in existing technologies, and achieving efficient and accurate measurement of joint opening.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CCCC SECOND HARBOR ENGINEERING CO LTD
- Filing Date
- 2023-03-27
- Publication Date
- 2026-07-24
AI Technical Summary
In the current technology, the inspection of tunnel segment joints mainly relies on manual methods, which has the problems of cumbersome inspection process, low efficiency, limited scope, low degree of digitalization, and easy to cause problems such as untimely detection and inadequate repair of gaps.
An image recognition-based method is adopted to acquire images of the pipe segment joints by taking photos with a mobile phone. An encoder and decoder network model built using the Transformer model is used to automatically judge the image quality and perform joint edge and defect segmentation. The joint opening amount is calculated by combining distance measurement.
It achieves efficient and accurate measurement of shield tunnel segment joints, with a high degree of automation, and can be widely used in the shield tunnel field, improving detection efficiency and accuracy and reducing errors caused by manual intervention.
Smart Images
Figure CN116563215B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of tunnel boring machine (TBM) construction technology. More specifically, this invention relates to a method for measuring the opening of TBM segment joints based on image recognition technology. Background Technology
[0002] The lining of shield tunnels is typically constructed using precast segments assembled together. These segments have longitudinal and circumferential joints, which are the weakest points in the lining structure. During construction and operation, due to factors such as geological conditions, construction defects, load effects, and the surrounding environment, the joints between the segments inevitably open. This negatively impacts the deformation and internal forces of the tunnel structure. Once the joint opening reaches a certain value, water seepage may occur, and even localized stress concentration may lead to segment damage, affecting the tunnel's safety and usability. Especially during the construction phase, uneven settlement caused by ground disturbance can exacerbate these issues, making the joint opening deformation and water seepage problems more pronounced. To ensure the safe operation of shield tunnel structures and provide data support for tunnel maintenance, timely and effective inspection of the joint gaps between shield tunnel segments is necessary.
[0003] Currently, the gaps in tunnel segment joints are primarily obtained manually. Surveyors need to approach the joints and use tools such as feeler gauges and calipers for measurement. For difficult-to-reach locations, scaffolding is required, and these inaccessible areas become blind spots. Furthermore, the recording and processing of data are also mainly done manually, making them prone to omissions and errors. In summary, manual contact measurement methods suffer from drawbacks such as cumbersome processes, low efficiency, limited detection range, and low digitization, easily leading to delayed detection and inadequate repair of gaps. Therefore, research is needed to develop faster and more efficient methods for detecting joint gaps. Summary of the Invention
[0004] One objective of this invention is to provide a method for measuring the opening amount of tunnel segment joints based on image recognition technology, which can achieve efficient and high-precision measurement of the opening amount of tunnel segment joints.
[0005] To achieve these objectives and other advantages according to the present invention, a method for measuring the opening of tunnel segment joints based on image recognition technology is provided, comprising the following steps:
[0006] Step S1: Obtain the segment joint image and its parameters, as well as the corresponding shooting distance for the segment joint image;
[0007] Step S2: Construct and train a network model for the segment joint images;
[0008] Step S3: Input the segment joint image obtained in step S1 into the network model of the segment joint image obtained in step S2, and output a binary image of the segment joint edge position.
[0009] Step S4: Image distance measurement and conversion of the segment joint. Based on the binary image of the edge position of the segment joint, projection accumulation is performed in the X direction of the image coordinate axis. The dense peak of the accumulated pixel value corresponds to the edge X coordinate of the segment joint. The pixel value of the segment joint width can be obtained by calculating the difference between the left and right edge pixel X coordinates. Combined with the segment joint image parameters in step S1 and the corresponding shooting distance of the segment joint image, the actual physical width of the joint is obtained, which is the segment joint opening amount.
[0010] Preferably, in step S1, the segment joint image is obtained by taking a picture with a mobile phone, and the corresponding shooting distance of the segment joint image is obtained by a laser rangefinder.
[0011] Preferably, the network model of the segment joint image in step S2 is built based on the Transformer model, which is a modular structure with one encoder and three decoders. The three decoders are: image quality classification decoder, segment joint defect segmentation decoder, and segment joint edge segmentation decoder.
[0012] Preferably, step S2 specifically includes the following steps:
[0013] Step S2.1: Obtain multiple images of the pipe segment joints to be detected and corresponding labeled samples. The labeled samples include: binary images of the pipe segment joint edges to be segmented, binary images of pipe segment joint defects, and manual evaluation labels for the quality of the pipe segment joint images. Use the labeled image data as the training sample set.
[0014] Step S2.2: Take the training sample set consisting of the segment joint image to be detected and the segment joint image quality manual evaluation label as input, take the output variable of the corresponding image quality classification decoder as output, and then train the image feature encoder and segment joint image quality classification decoder based on the loss function.
[0015] Step S2.3: Take the training sample set consisting of the image of the segment joint to be detected and the binary image of the segment joint defect as input, and take the binary segmentation image generated by the corresponding segment joint defect segmentation decoder as output. Then, based on the loss function, train the image feature encoder and the segment joint defect segmentation decoder.
[0016] Step S2.4: Take the training sample set consisting of the image of the segment joint to be detected and the binary image of the segment joint edge as input, and take the binary segmentation image generated by the corresponding segment joint edge segmentation decoder as output. Then, based on the loss function, train the image feature encoder and the segment joint edge segmentation decoder.
[0017] Preferably, step S3 specifically includes the following steps:
[0018] Step S3.1: Input the segment joint image obtained in step S1 into the image feature encoder and segment joint image quality classification decoder trained in step S2.2 to obtain the image quality evaluation label corresponding to this segment joint image, and compare it with the preset value. If it meets the requirements, proceed to the next step S3.2; otherwise, discard the measurement data corresponding to this segment joint image.
[0019] Step S3.2: Input the segment joint image obtained in step S1 into the image feature encoder and segment joint edge segmentation decoder trained in step S2.4 to obtain the edge position binary map corresponding to this segment joint image, which is used for the calculation in step S4.
[0020] Preferably, step S3 further includes the following steps:
[0021] Step S3.3: Input the segment joint image obtained in step S1 into the image feature encoder and segment joint defect segmentation decoder trained in step S2.3 to obtain the defect segmentation binary image corresponding to this segment joint image.
[0022] Step S3.4: Compare the coordinates of the defect segmentation binary map obtained in step S3.3 with the coordinates of the edge position binary map obtained in step S3.2 to obtain the defect location of the segment joint and its corresponding coordinates. The measured segment joint opening data calculated from the corresponding coordinates is used as abnormal data for alerting. Abnormal data is then manually processed.
[0023] Preferably, in step S3.1, if the segment joint image does not meet the requirements, a prompt will pop up in the software, and a second manual judgment will be made on whether the image is usable. If it is usable, proceed to the next step S3.2; if it is not usable, the measurement data corresponding to this segment joint image will be discarded.
[0024] Preferably, the encoder's transformer network model includes three modules: a self-attention mechanism module, a confused forward propagation network module, and an overlapping image patch merging module. The self-attention mechanism module aims to assign higher weights to image feature vectors with high correlation. The confused forward propagation network module is used to further extract image features. The overlapping image patch merging module adjusts the feature vectors extracted by transformers at different levels to a unified dimension and then concatenates image features at different scales.
[0025] Preferably, the segment joint defect segmentation decoder performs semantic segmentation on the defects at the segment joints, and uses the UNet model to extract the defects between the segments.
[0026] Preferably, the edge feature decoder network model of the tunnel segment joint consists of multiple transformer network modules; the edge segmentation decoder of the tunnel segment joint consists of three modules: a feature fusion module, a feature space transformation module, and a feature concatenation module. The function of the feature fusion module is to fuse the image features of different levels concatenated in the encoder to obtain the global and local features of the tunnel segment joint. The feature space transformation module uses the high-order features in the encoder to cross-multiply the low-order features in the encoder, taking advantage of the global spatial continuity of the tunnel segment joint distribution. The function of the feature concatenation module is to convolve and extract the features extracted by the transformer network modules of each level in the encoder module.
[0027] The present invention has at least the following beneficial effects:
[0028] 1. This invention can automatically determine whether the shooting angle, shooting distance, and image shooting quality meet the measurement requirements, effectively solving the shortcomings of manual contact measurement methods, such as cumbersome detection process, low detection efficiency, limited detection range, and low degree of digitization, which can easily lead to problems such as untimely detection and inadequate repair of gaps at the joints.
[0029] 2. This invention, through the structure of a Transformer-based encoder and multiple decoders based on a spatial transformation network, can quickly complete defect detection of segment joints and high-precision detection of segment joint edges, and can be widely applied in the field of shield tunnels.
[0030] 3. This invention has high measurement accuracy and good anti-interference performance; by using a cascaded Transformer encoder, it can achieve feature extraction at multiple scales.
[0031] 4. This invention can simultaneously output defect segmentation diagrams of inter-segment seams, which facilitates tunnel management personnel in analyzing and judging the construction and maintenance status of the segments; in particular, this invention is based on a transformer segmentation network and consists of two parts: a classic encoder and a decoder.
[0032] Other advantages, objectives and features of the present invention will become apparent in part from the following description, and in part from those skilled in the art through study and practice of the invention. Attached Figure Description
[0033] Figure 1 This is a network framework diagram for the segment joint measurement and analysis of the present invention;
[0034] Figure 2 This is a schematic diagram of the encoder module of the segment joint measurement and analysis network of the present invention.
[0035] Figure 3 This is a schematic diagram of the image quality classification decoder of the segment joint measurement and analysis network of the present invention;
[0036] Figure 4 This is a schematic diagram of the converter network module of the inter-segment gap measurement and analysis network encoder of the present invention;
[0037] Figure 5 This is a schematic diagram of the joint segmentation decoder in the segment joint measurement and analysis network of the present invention. Detailed Implementation
[0038] The present invention will now be described in further detail with reference to the accompanying drawings, so that those skilled in the art can implement it based on the description.
[0039] It should be noted that, unless otherwise specified, the experimental methods described in the following embodiments are all conventional methods, and the reagents and materials described are all commercially available unless otherwise specified. In the description of this invention, the terms "lateral", "longitudinal", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0040] This invention provides a method for measuring the joint opening of tunnel lining segments based on image recognition technology, comprising the following steps:
[0041] Step S1: Obtain the segment joint image and its parameters, as well as the corresponding shooting distance for the segment joint image;
[0042] In step S1, the segment joint image is acquired by taking a picture with a mobile phone. The image's focal length, distance, ISO, F, S, AF-A, EV, and other information are obtained by taking a picture with a mobile phone. The shooting distance corresponding to the segment joint image is obtained by a laser rangefinder.
[0043] Step S2: Construct and train a network model for the segment joint images;
[0044] like Figure 1 As shown, the network model for the segment joint image in step S2 is built based on the Transformer model, which has a modular structure of one encoder and three decoders. The encoder module is composed of multiple cascaded Transformer modules. This cascading form ensures that the encoder can extract image feature information both globally and locally. The three decoders are: an image quality classification decoder, a segment joint defect segmentation decoder, and a segment joint edge segmentation decoder.
[0045] like Figure 2 As shown, the transformer network model in the segmentation network encoder mainly consists of three modules: an efficient self-attention mechanism module, a confused forward propagation network module, and an overlapping image patch merging module. To extract features from images at different resolutions, the encoder for the inter-segment seams is constructed from a hierarchical cascade of transformers, using multiple cascaded Transformer feature extraction modules as the encoder module. To ensure that the features extracted by the transformer are not destroyed due to the fixed coordinates of the image patches, the transformer utilizes patches with overlapping coordinate regions.
[0046] To address the issue that the transformer cannot encode the spatial information of the image, the encoder module of our model employs a position embedding scheme based on convolutional layers. This embeds the spatial coordinates corresponding to the image pixels into the feature vector, thus avoiding performance loss caused by changes in image resolution.
[0047] The purpose of the self-attention mechanism is to assign higher weights to image feature vectors with high relevance; the confuse forward propagation network is used to further extract image features; and the function of the final overlapping image patch merging module is to adjust the feature vectors extracted by transformers at different levels to a unified dimension, and then cascade image features at different scales.
[0048] Step S2 specifically includes the following steps:
[0049] Step S2.1: Obtain multiple images of the pipe segment joints to be detected and corresponding labeled samples. The labeled samples include: binary images of the pipe segment joint edges to be segmented, binary images of pipe segment joint defects, and manual evaluation labels for the quality of the pipe segment joint images. Use the labeled image data as the training sample set.
[0050] Step S2.2: Take the training sample set consisting of the segment joint image to be detected and the segment joint image quality manual evaluation label as input, take the output variable of the corresponding image quality classification decoder as output, and then train the image feature encoder and segment joint image quality classification decoder based on the loss function.
[0051] The image quality classification decoder is designed to judge the quality of photos of tunnel segment joints taken by on-site personnel. If the image contrast is lower than a preset value, or if the shooting angle or distance deviates from the preset value, the decoder will notify the photographer via mobile phone that the image quality is unacceptable and needs to be retaken to meet the requirements of high-precision measurement. The network structure of the image quality classification decoder is as follows: Figure 3 As shown, it consists of multiple parallel spatial feature extraction convolutional networks, a feature fusion layer, two fully convolutional layers, and a final SoftMax nonlinear prediction output layer.
[0052] Step S2.3: Take the training sample set consisting of the image of the segment joint to be detected and the binary image of the segment joint defect as input, and take the binary segmentation image generated by the corresponding segment joint defect segmentation decoder as output. Then, based on the loss function, train the image feature encoder and the segment joint defect segmentation decoder.
[0053] The function of the segment joint defect segmentation decoder is to perform semantic segmentation of defects such as adhesive cement spots and gap damage at the segment joints, which facilitates the manual maintenance of the segments. Since the defects in the segment gaps are relatively large and their features are relatively obvious, the common UNet model can be used to extract the defects in the segment gaps.
[0054] Step S2.4: Take the training sample set consisting of the image of the segment joint to be detected and the binary image of the segment joint edge as input, and take the binary segmentation image generated by the corresponding segment joint edge segmentation decoder as output. Then, based on the loss function, train the image feature encoder and the segment joint edge segmentation decoder.
[0055] like Figure 4 As shown, unlike traditional convolutional neural networks, the edge feature encoder network model of the pipe segment joint is mainly composed of multiple transformer network modules. The main function of these transformer network modules is to statistically adjust the channels of the cascaded features in the encoder to a uniform level, then upsample these features to 1 / 4 resolution of the original image and cascade them, and finally fuse these image features and input them into the last linear layer to predict the result.
[0056] The network model of the edge segmentation decoder for the segment joint is as follows: Figure 5As shown, unlike the common UNET, the segment seam edge segmentation decoder consists of three modules: a feature fusion module, a feature space transformation module, and a feature concatenation module. The feature fusion module fuses image features from different levels of the encoder to obtain global and local features of the inter-segment seams. The feature space transformation module cross-products higher-order features from the encoder with lower-order features, leveraging the global spatial continuity of the inter-segment seam distribution. The feature concatenation module performs convolution on the features extracted by the transformer network modules at each level of the encoder module before further extraction.
[0057] Step S3: Input the segment joint image obtained in step S1 into the network model of the segment joint image obtained in step S2, and output a binary image of the segment joint edge position.
[0058] Step S3 specifically includes the following steps:
[0059] Step S3.1: Input the segment joint image obtained in step S1 into the image feature encoder and segment joint image quality classification decoder trained in step S2.2 to obtain the image quality evaluation label corresponding to this segment joint image, and compare it with the preset value to evaluate the quality of the obtained image. If it meets the requirements, proceed to the next step S3.2; otherwise, discard the measurement data corresponding to this segment joint image.
[0060] In step S3.1, if the segment joint image does not meet the requirements, a prompt can be set in the software to allow for a second manual judgment on whether the image is usable. If it is usable, proceed to the next step S3.2; if it is not usable, the measurement data corresponding to this segment joint image is discarded.
[0061] Step S3.2: Input the segment joint image obtained in step S1 into the image feature encoder and segment joint edge segmentation decoder trained in step S2.4 to obtain the edge position binary map corresponding to this segment joint image, which is used for the calculation in step S4.
[0062] Step S3.3: Input the segment joint image obtained in step S1 into the image feature encoder and segment joint defect segmentation decoder trained in step S2.3 to obtain the defect segmentation binary image corresponding to this segment joint image.
[0063] Step S3.4: Compare the coordinates of the defect segmentation binary map obtained in step S3.3 with the coordinates of the edge position binary map obtained in step S3.2 to obtain the defect location of the segment joint and its corresponding coordinates. The measured segment joint opening data calculated from the corresponding coordinates is used as abnormal data for notification. Abnormal data can be manually processed, and the corresponding data can be applied or removed according to the actual situation.
[0064] The purpose of this application is to assist on-site manual assessment of segment quality. The assessment is conducted from three dimensions. First, the quality of the acquired image is assessed, and if it is unusable, no further operations are performed. Second, through steps S3.3 and S3.4, the reliability of the measured segment joint opening results is assessed by defect segmentation, etc., and accurate results are required. Finally, the joint opening amount is measured to determine the segment quality.
[0065] Step S4: Image distance measurement conversion of segment joints;
[0066] Step S4.1: Based on the binary image of the edge position of the pipe segment joint, project and accumulate the pixel values in the X direction of the image coordinate axis. The peak of the accumulated pixel values corresponds to the X coordinate of the edge of the pipe segment joint. The pixel value of the width of the pipe segment joint can be obtained by calculating the difference between the X coordinates of the left and right edge pixels.
[0067] The pixel values of the segment joint width obtained in steps S4.2 and S4.1, combined with the segment joint image parameters in step S1 and the corresponding shooting distance of the segment joint image, and using the pinhole imaging principle of the camera, yield the actual physical width of the joint, which is the segment joint opening amount. The specific calculation formula is as follows:
[0068]
[0069] The variables Dist, n, D, f, and y in the formula are the shooting distance of the segment joint, the number of imaging pixels of the segment joint, the pixel size of the mobile phone camera, the focal length of the mobile phone camera, and the actual physical distance between the segments obtained through shooting, respectively.
[0070] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.
Claims
1. A method for measuring the opening of tunnel segment joints based on image recognition technology, characterized in that, Includes the following steps: Step S1: Obtain the segment joint image and its parameters, as well as the corresponding shooting distance for the segment joint image; Step S2: Construct and train a network model for the segment joint images; Step S3: Input the segment joint image obtained in step S1 into the network model of the segment joint image obtained in step S2, and output a binary image of the segment joint edge position. Step S4: Image distance measurement and conversion of pipe segment joints. Based on the binary image of the edge position of the pipe segment joint, projection accumulation is performed in the X direction of the image coordinate axis. The dense peak of the accumulated pixel value corresponds to the edge X coordinate of the pipe segment joint. The pixel value of the width of the pipe segment joint can be obtained by calculating the difference between the X coordinates of the left and right edge pixels. Combined with the image parameters of the pipe segment joint in step S1 and the shooting distance corresponding to the pipe segment joint image, the actual physical width of the joint is obtained, which is the opening amount of the pipe segment joint. The network model for the segment joint image in step S2 is built based on the Transformer model. It has a modular structure of one encoder and three decoders. The three decoders are: image quality classification decoder, segment joint defect segmentation decoder, and segment joint edge segmentation decoder. Step S2 specifically includes the following steps: Step S2.1: Obtain multiple images of the pipe segment joints to be detected and corresponding labeled samples. The labeled samples include: binary images of the pipe segment joint edges to be segmented, binary images of pipe segment joint defects, and manual evaluation labels for the quality of the pipe segment joint images. Use the labeled image data as the training sample set. Step S2.2: Take the training sample set consisting of the segment joint image to be detected and the segment joint image quality manual evaluation label as input, take the output variable of the corresponding image quality classification decoder as output, and then train the image feature encoder and segment joint image quality classification decoder based on the loss function. Step S2.3: Take the training sample set consisting of the image of the segment joint to be detected and the binary image of the segment joint defect as input, and take the binary segmentation image generated by the corresponding segment joint defect segmentation decoder as output. Then, based on the loss function, train the image feature encoder and the segment joint defect segmentation decoder. Step S2.4: Take the training sample set consisting of the image of the segment joint to be detected and the binary image of the segment joint edge as input, and take the binary segmentation image generated by the corresponding segment joint edge segmentation decoder as output. Then, based on the loss function, train the image feature encoder and the segment joint edge segmentation decoder. Step S3 specifically includes the following steps: Step S3.1: Input the segment joint image obtained in step S1 into the image feature encoder and segment joint image quality classification decoder trained in step S2.2 to obtain the image quality evaluation label corresponding to this segment joint image, and compare it with the preset value. If it meets the requirements, proceed to the next step S3.2; otherwise, discard the measurement data corresponding to this segment joint image. Step S3.2: Input the segment joint image obtained in step S1 into the image feature encoder and segment joint edge segmentation decoder trained in step S2.4 to obtain the edge position binary map corresponding to this segment joint image, which is used for the calculation in step S4. Step S3.3: Input the segment joint image obtained in step S1 into the image feature encoder and segment joint defect segmentation decoder trained in step S2.3 to obtain the defect segmentation binary image corresponding to this segment joint image. Step S3.4: Compare the coordinates of the defect segmentation binary map obtained in step S3.3 with the coordinates of the edge position binary map obtained in step S3.2 to obtain the defect location of the segment joint and its corresponding coordinates. The measured segment joint opening data calculated from the corresponding coordinates is used as abnormal data for alerting. Abnormal data is then manually processed.
2. The method for measuring the opening of shield tunnel segment joints based on image recognition technology as described in claim 1, characterized in that, In step S1, the segment joint image is obtained by taking a picture with a mobile phone, and the corresponding shooting distance of the segment joint image is obtained by a laser rangefinder.
3. The method for measuring the opening of shield tunnel segment joints based on image recognition technology as described in claim 1, characterized in that, In step S3.1, if the segment joint image does not meet the requirements, a prompt will pop up in the software, and a second manual judgment will be made on whether the image is usable. If it is usable, proceed to the next step S3.2; if it is not usable, the measurement data corresponding to this segment joint image will be discarded.
4. The method for measuring the opening of shield tunnel segment joints based on image recognition technology as described in claim 1, characterized in that, The encoder's transformer network model includes three modules: a self-attention mechanism module, a confused forward propagation network module, and an overlapping image patch merging module. The self-attention mechanism module aims to assign higher weights to image feature vectors with high correlation. The confused forward propagation network module is used to further extract image features. The overlapping image patch merging module adjusts the feature vectors extracted by transformers at different levels to a unified dimension and then concatenates image features at different scales.
5. The method for measuring the opening of shield tunnel segment joints based on image recognition technology as described in claim 1, characterized in that, The segment joint defect segmentation decoder performs semantic segmentation of defects at the segment joints and uses the UNet model to extract defects between segments.
6. The method for measuring the opening of shield tunnel segment joints based on image recognition technology as described in claim 1, characterized in that, The edge feature decoder network model for tunnel segment joints consists of multiple transformer network modules; the edge segmentation decoder for tunnel segment joints consists of three modules: a feature fusion module, a feature space transformation module, and a feature concatenation module. The function of the feature fusion module is to fuse the image features of different levels concatenated in the encoder to obtain the global and local features of the tunnel segment joints. The feature space transformation module uses the cross product of the higher-order features in the encoder with the lower-order features in the encoder to take advantage of the global spatial continuity of the tunnel segment joint distribution. The function of the feature concatenation module is to convolve the features extracted by the transformer network modules at each level in the encoder module and then extract them.