Highway tunnel lining crack detection system based on improved YOLO network

Abstract

The invention discloses a highway tunnel lining crack detection system based on an improved YOLO network, and relates to the technical field of deep learning. Based on a target detection network YOLOv10 at the present stage, an MLCA module is added during construction of a feature extraction network to obtain global information and context information of a feature map, the feature expression ability of a long and thin crack target is effectively enhanced, and the detection accuracy of the tunnel lining crack is improved. A PMLCA module is put forward to focus on the area where the crack is located, unnecessary features are restrained, and calculation redundancy is reduced; meanwhile, a BiFPN module is further integrated in the feature fusion part of the network, a multi-level feature pyramid and a bidirectional cross-scale feature fusion mechanism of the BiFPN module can allow crack feature information to flow bidirectionally among different scales, crack information of different scales is captured, and tunnel lining crack real-time detection is achieved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a highway tunnel lining crack detection system based on an improved YOLO network. Background Art

[0002] As an important transportation infrastructure, the structural safety of highway tunnels is directly related to driving safety and smooth transportation. Tunnel lining is one of the key indicators for evaluating the safety of tunnel structures. Timely and accurate detection of cracks is of great significance for ensuring driving safety. However, traditional crack detection methods have many shortcomings, such as complex detection process, time-consuming and labor-intensive, and weak generalization ability, which makes it difficult to meet the needs of modern highway tunnel maintenance and management.

[0003] With the rapid development of deep learning technology, target detection algorithms based on deep learning have achieved remarkable results in various fields. In the field of highway tunnel lining crack detection, deep learning algorithms have gradually become a research hotspot due to their powerful feature extraction capabilities and generalization performance. Most of the existing deep learning crack detection networks are built based on convolutional neural networks (CNNs), which can automatically identify and locate crack targets by extracting feature information from images. However, these systems still face some challenges when dealing with highway tunnel lining crack detection tasks.

[0004] On the one hand, the cracks in highway tunnel linings have various shapes, including long and thin cracks, wide cracks, cross cracks, etc., and the size of the cracks varies greatly. Some cracks may be very small at the sub-millimeter level, while others are more obvious at the millimeter level. This requires the detection system to have the ability to fuse multi-scale features to capture crack information of different scales. Although the traditional feature pyramid network has achieved multi-scale feature fusion to a certain extent, its feature transmission method is single, which easily leads to the loss of detailed information of the underlying feature map during the transmission process, affecting the detection accuracy.

[0005] On the other hand, the lighting conditions in highway tunnels are complex. Due to problems such as fill light intensity or exposure, there may be a lot of noise in the tunnel lining image, and the difference between cracks and background is small, which increases the difficulty of crack identification. When processing such images, traditional crack detection systems often find it difficult to accurately focus on the crack area, resulting in unsatisfactory detection results. Therefore, it is necessary to introduce a more effective attention mechanism to increase the network's attention to the crack area and reduce computational redundancy. Summary of the invention

[0006] The object of the present invention is to provide a highway tunnel lining crack detection system based on an improved YOLO network to solve the problems raised in the prior art.

[0007] To achieve the above object, the present invention provides the following technical solutions: a highway tunnel lining crack detection system based on an improved YOLO network, the detection system comprising an image acquisition module, an image preprocessing module, a crack detection module, and a result display module;

[0008] An image acquisition module, mounted on a mobile platform, is used to acquire images of the highway tunnel lining;

[0009] An image preprocessing module is used to preprocess the collected highway tunnel lining images, wherein the preprocessing includes data expansion and annotation information conversion;

[0010] The crack detection module uses the MB-YOLO network to detect cracks in the preprocessed highway tunnel lining images;

[0011] The result display module is used to display the crack detection results.

[0012] Furthermore, the image preprocessing module performs data expansion on the highway tunnel lining image acquired by the image acquisition module by means of geometric transformation;

[0013] The geometric transformation methods in the above steps include flipping, rotating, cropping and other operations. The purpose of using geometric transformation to expand data is to avoid overfitting during the training process.

[0014] Furthermore, the image preprocessing module adds annotation information to the highway tunnel lining images after data expansion through an annotation tool, randomly divides the highway tunnel lining images after the annotation information is added into two groups, one group is a training set and the other group is a test set, and converts the format of the annotation information in the training set and the test set through a third-party tool;

[0015] The above-mentioned format conversion of the annotation information in the training set and the test set is because the annotation information output by the annotation tool will affect the network training.

[0016] Furthermore, the crack detection module inputs the training set output by the image preprocessing module into the MB-YOLO network to train the MB-YOLO network, and inputs the test set output by the image preprocessing module into the trained MB-YOLO network to detect cracks;

[0017] The above-mentioned MB-YOLO network is improved on the basis of the latest YOLOv10 network. Considering that tunnel lining cracks usually extend over a large span, the MLCA module is introduced into the feature extraction network of the YOLOv10 network, the C2fCIB module in the original YOLOv10 network is replaced with the C2fMLCA module, the PMLCA module is used at the backbone end of the YOLOv10 network, and the BiFPN module is introduced into the feature fusion network of the YOLOv10 network.

[0018] Furthermore, the MB-YOLO network used in the crack detection module includes a feature extraction network, a feature fusion network, and an output end;

[0019] The feature extraction network includes convolution module, C2fMLCA module, SPPF module and PMLCA module;

[0020] The feature extraction network first obtains a feature image with translation invariance through a convolution module, then obtains the local and global connections between pixels of the feature image through a C2fMLCA module, and then merges the context information of cracks of different scales through an SPPF module. At the end of the feature extraction network, a PMLCA module is used, which captures the scale characteristics of cracks by integrating local and global channel information and spatial information.

[0021] Compared with the original multi-head attention, the PMLCA module in the above feature network can capture the different scale characteristics of cracks, which is very important for balancing the global and local aspects of crack detection, so that the network can pay attention to the overall direction and local details of the cracks.

[0022] The feature fusion network includes a BiFPN module, which enables feature information to flow bidirectionally between different scales through a bidirectional cross-scale feature fusion mechanism, and adaptively allocates weights of features at different levels through a weighted fusion mechanism;

[0023] The output uses three prediction heads of different scales to perform bounding box regression on cracks and classify the cracks.

[0024] Furthermore, the MLCA mechanism in the C2fMLCA module and the PMLCA module extracts the local spatial information of the crack area by dividing the input features into small blocks and performing local pooling. Two parallel branches are used, one branch extracts global information and the other branch focuses on local spatial information. One-dimensional convolution is used in the feature fusion stage.

[0025] In the above steps, the MLCA hybrid local channel attention mechanism divides the input features into small blocks and performs local pooling to eliminate the interference of some background information, thereby extracting the local spatial information of the crack area, enabling the network to focus on the crack area more accurately. MLCA adopts two parallel branches, one branch extracts global information and the other branch focuses on local spatial information. This method realizes the recognition of cracks of different scales and adapts to various crack shapes and sizes. In order to reduce the amount of calculation, MLCA uses single-dimensional convolution instead of conventional two-dimensional convolution in the fusion stage, thereby accelerating the calculation.

[0026] Furthermore, the BiFPN module uses a bidirectional fusion mechanism to make the feature information in the feature network layer flow and fuse the feature information in both the top-down and bottom-up directions, and introduces learnable weights through a weighted fusion mechanism to adaptively allocate weights of features at different levels according to task requirements, while adopting node sharing and redundant feature reduction strategies to optimize computational efficiency;

[0027] The above-mentioned BiFPN module uses a bidirectional feature fusion mechanism to enable the network to capture detailed features while maintaining sensitivity to global information, thereby more accurately locating the boundary morphology of cracks. In particular, in the context of complex cracks, bidirectional fusion can enhance robustness to noise. The BiFPN module introduces learnable weights to enable the network to adaptively allocate weights of features at different levels according to task requirements. For example, features at certain levels contribute more to the detection of small cracks, while features at other levels are more conducive to the detection of wide cracks. BiFPN can automatically adjust these weights to improve detection accuracy, especially sensitivity to small cracks.

[0028] Furthermore, the result display module is used to display the crack detection result, and the detection result includes the crack position and the prediction box.

[0029] Compared with the prior art, the beneficial effects of the present invention are as follows: based on the single-stage target detection network YOLOv10, the present invention adds the MLCA module to obtain the global information and context information of the feature map when constructing the feature extraction network, effectively enhancing the feature expression ability of long and thin crack targets, and proposes a PMLCA module to focus on the area where the cracks are located, suppress unnecessary features and reduce computational redundancy; at the same time, the network also integrates the BIFPN module in the feature fusion part, and its multi-level feature pyramid and bidirectional cross-scale feature fusion mechanism can allow crack feature information to flow bidirectionally between different scales, capturing crack information of different scales. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the system structure of the highway tunnel lining crack detection system based on the improved YOLO network of the present invention;

[0031] Figure 2 A comparison diagram of the BiFPN network, FPN network, PANet network and NAS-FPN network of the highway tunnel lining crack detection system based on the improved YOLO network of the present invention;

[0032] Figure 3 This is a detailed expansion diagram of the BiFPN module of the highway tunnel lining crack detection system based on the improved YOLO network in the MB-YOLO network of the present invention. DETAILED DESCRIPTION

[0033] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0034] Example: Figure 1-3 As shown, the present invention provides a technical solution, a highway tunnel lining crack detection system based on an improved YOLO network, the detection system comprising: an image acquisition module, an image preprocessing module, a crack detection module, and a result display module;

[0035] An image acquisition module, mounted on a mobile platform, is used to acquire images of the highway tunnel lining;

[0036] An image preprocessing module is used to preprocess the collected highway tunnel lining images, wherein the preprocessing includes data expansion and annotation information conversion;

[0037] The crack detection module uses the MB-YOLO network to detect cracks in the preprocessed highway tunnel lining images;

[0038] A result display module, used to display crack detection results;

[0039] The image preprocessing module performs data expansion on the highway tunnel lining image acquired by the image acquisition module by means of geometric transformation;

[0040] The image preprocessing module adds annotation information to the highway tunnel lining images after data expansion through an annotation tool, randomly divides the highway tunnel lining images after the annotation information is added into two groups, one group is a training set and the other group is a test set, and converts the format of the annotation information in the training set and the test set through a third-party tool;

[0041] In an embodiment of the present invention, a FLIR ORX-10G-310S9 camera is mounted on a mobile platform to collect images of cracks in the lining of a highway tunnel. In order to avoid overfitting during the training process, operations such as flipping, rotation, and cropping are used to expand the data set. There are 1995 images in total, which are annotated using the popular annotation tool Labelme. The data set is randomly divided into two groups, one with 1494 images for training and the other with 251 images for verification and 250 for testing. The output of Labelme is in JSON format. Before network training, the annotation information is converted into YOLO format using a third-party library;

[0042] The crack detection module inputs the training set output by the image preprocessing module into the MB-YOLO network to train the MB-YOLO network, and inputs the test set output by the image preprocessing module into the trained MB-YOLO network to detect cracks;

[0043] The MB-YOLO network used in the crack detection module includes a feature extraction network, a feature fusion network, and an output end;

[0044] The feature extraction network includes convolution module, C2fMLCA module, SPPF module and PMLCA module;

[0045] The feature extraction network first obtains a feature image with translation invariance through a convolution module, then obtains the local and global connections between pixels of the feature image through a C2fMLCA module, and then merges the context information of cracks of different scales through an SPPF module. At the end of the feature extraction network, a PMLCA module is used, which captures the scale characteristics of cracks by integrating local and global channel information and spatial information.

[0046] The feature fusion network includes a BiFPN module, which enables feature information to flow bidirectionally between different scales through a bidirectional cross-scale feature fusion mechanism, and adaptively allocates weights of features at different levels through a weighted fusion mechanism;

[0047] The output end uses three prediction heads of different scales to perform bounding box regression on cracks and classify the cracks;

[0048] Among them, the MLCA hybrid local channel attention mechanism in the C2fMLCA module and the PMLCA module extracts the local spatial information of the crack area by dividing the input features into small blocks and performing local pooling. It adopts two parallel branches, one branch extracts global information and the other branch focuses on local spatial information, and uses single-dimensional convolution in the feature fusion stage;

[0049] Among them, the BiFPN module uses a bidirectional feature fusion mechanism to make the feature information in the feature network layer flow and fuse the feature information in both the top-down and bottom-up directions, and introduces learnable weights through a weighted fusion mechanism to adaptively allocate weights of features at different levels according to task requirements. At the same time, it adopts node sharing and redundant feature reduction strategies to optimize computing efficiency.

[0050] like Figure 2 As shown in the figure, the FPN feature pyramid network introduces a top-down path to fuse multi-scale features, and the PANet path aggregation network adds an additional bottom-up path based on FPN; the NAS-FPN feature pyramid network based on neural architecture search uses neural architecture search to find irregular feature network topology, and the BiFPN bidirectional feature pyramid network improves the recognition accuracy and efficiency through efficient bidirectional cross-scale connections and repeated block structures;

[0051] from Figure 3 The specific details of BiFPN in the MB-YOLO network can be seen in Figure 3 It includes the use of feature extraction network and BiFPN as feature fusion network. In the feature fusion network, BiFPN receives multi-scale input features from the MB-YOLO backbone network through the ability of bidirectional feature fusion, and then generates features for crack classification and border prediction.

[0052] The result display module is used to display the crack detection results, which include the crack location and the prediction box.

[0053] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. Highway tunnel lining crack detection system based on improved YOLO network, characterized by: The detection system includes: an image acquisition module, an image preprocessing module, a crack detection module, and a result display module; The image acquisition module is installed on the mobile platform and is used to acquire images of the highway tunnel lining; The image preprocessing module is used to preprocess the collected highway tunnel lining image, and the preprocessing includes data expansion and annotation information conversion; The crack detection module uses the MB-YOLO network to perform crack detection on the preprocessed highway tunnel lining image; The result display module is used to display the crack detection results.

2. The highway tunnel lining crack detection system based on the improved YOLO network according to claim 1 is characterized in that: The image preprocessing module performs data expansion on the highway tunnel lining image acquired by the image acquisition module by means of geometric transformation.

3. The highway tunnel lining crack detection system based on the improved YOLO network according to claim 1 is characterized in that: The image preprocessing module adds annotation information to the highway tunnel lining images after data expansion through an annotation tool, randomly divides the highway tunnel lining images after the annotation information is added into two groups, one group is a training set and the other group is a test set, and converts the format of the annotation information in the training set and the test set through a third-party tool.

4. The highway tunnel lining crack detection system based on the improved YOLO network according to claim 1 is characterized in that: The crack detection module inputs the training set output by the image preprocessing module into the MB-YOLO network to train the MB-YOLO network, and inputs the test set output by the image preprocessing module into the trained MB-YOLO network to detect cracks.

5. The highway tunnel lining crack inspection system based on the improved YOLO network according to claim 1 is characterized in that: The MB-YOLO network used in the crack detection module includes a feature extraction network, a feature fusion network, and an output end; The feature extraction network includes a convolution module, a C2fMLCA module, a SPPF module, and a PMLCA module; The feature extraction network first obtains a feature image with translation invariance through a convolution module, then obtains the local and global connections between pixels of the feature image through a C2fMLCA module, and then merges the context information of cracks of different scales through an SPPF module. A PMLCA module is used at the end of the feature extraction network. The PMLCA module captures the scale characteristics of the cracks by integrating local and global channel information and spatial information. The feature fusion network includes a BiFPN module, which enables feature information to flow bidirectionally between different scales through a bidirectional cross-scale feature fusion mechanism, and adaptively allocates weights of features at different levels through a weighted fusion mechanism; The output end uses three prediction heads of different scales to perform bounding box regression on cracks and classify the cracks.

6. The highway tunnel lining crack detection system based on the improved YOLO network according to claim 5 is characterized in that: The MLCA mechanism in the C2fMLCA module and the PMLCA module extracts local spatial information of the crack area by dividing the input features into small blocks and performing local pooling. Two parallel branches are adopted, one branch extracts global information and the other branch focuses on local spatial information, and one-dimensional convolution is used in the feature fusion stage.

7. The highway tunnel lining crack detection system based on the improved YOLO network according to claim 5 is characterized in that: The BiFPN module uses a bidirectional feature fusion mechanism to make the feature information in the feature network layer flow and fuse the feature information in both top-down and bottom-up directions, and introduces learnable weights through a weighted fusion mechanism to adaptively allocate weights of features at different levels according to task requirements. At the same time, it adopts node sharing and redundant feature reduction strategies to optimize computational efficiency.

8. The highway tunnel lining crack detection system based on the improved YOLO network according to claim 1 is characterized in that: The result display module is used to display the crack detection results, which include the crack position and the prediction box.

Images