Tunnel lining damage detection method, system and equipment based on data fusion and medium

Abstract

The invention relates to a tunnel lining damage detection method, system and equipment based on data fusion and a medium. The method comprises the following steps: acquiring shock wave audio frequency data and visible light image data of a tunnel to be detected; and inputting the obtained shock wave audio frequency data and visible light image data into a pre-established tunnel lining damage detection model based on data fusion to obtain a damage detection result of the to-be-detected tunnel lining. According to the method, the impact echo audio frequency data and the visible light image data are fused, so that the influence of steel bars and water on lining damage detection can be reduced, and meanwhile, the detection rate of surface defects is greatly increased. Therefore, the method can be widely applied to the technical field of tunnel detection and analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel detection and analysis, and in particular to a method, system, equipment and medium for detecting tunnel lining damage based on data fusion. Background Art

[0002] Tunnel lining structure damage is a very common disaster in tunnels, which mainly includes: cracking, corrosion, back cavities, deformation and displacement, lining crushing, peeling and insufficient thickness. The formation mechanism of tunnel lining structure damage is very complex and is an important problem in the world. Minor damage can affect the service life of the tunnel, and more serious damage can even threaten the driving safety of trains, causing unimaginable consequences.

[0003] At present, the commonly used tunnel lining damage detection method is based on geological radar. The main working principle of geological radar for lining crack detection is to use the broadband antenna carried by the geological radar itself to transmit high-frequency broadband short-pulse electromagnetic waves to the target medium to be tested. The electromagnetic waves pass through different dielectric layers, reflect and then return to be received by the receiving antenna. During the transmission process, the electromagnetic wave is affected by the dielectric properties and geometric shape of the medium it passes through. The waveform, path and magnetic field strength of the electromagnetic wave will change to a certain extent with different media. The lining thickness, lining concrete strength and lining structure can be detected by the parameters such as the time delay, shape and spectrum characteristics of the echo signal, and then it can be judged whether the lining has defects. In the detection of tunnel lining, if there is a double-layer steel mesh or the steel protection layer and thickness are thin, the radar wave often cannot penetrate the steel mesh, so it is impossible to effectively detect the concrete structure under the steel mesh. At the same time, geological radar has certain difficulties in detecting surface defects. At the same time, for some small and non-dense defects with small cavities, the radar wave reflection signal is not obvious.

[0004] In addition, some patents propose to use multiple sensors to detect the lining. For example, patent CN 118640959 A discloses a comprehensive tunnel lining detection system integrated on a lining trolley. This method uses geological radar, ultrasonic shear waves, and three-dimensional laser to jointly detect the lining. However, this method only makes a joint judgment on the detection results of multiple sensors and does not fully integrate the data of each sensor.

[0005] Through the analysis of existing technologies, it can be seen that the current tunnel lining damage detection methods still have some shortcomings: 1. The existing lining damage detection method is greatly affected by steel bars and water; 2. Existing lining damage detection methods are not sensitive to surface defects; 3. There is no end-to-end fusion model for lining detection that fuses multi-sensor data to obtain comprehensive detection results. Summary of the invention

[0006] In view of the problem that the existing lining damage detection methods are greatly affected by steel bars and water and are insensitive to surface defects, the purpose of the present invention is to provide a tunnel lining damage detection method, system, equipment and medium based on data fusion. By fusing impact echo audio data and visible light image data, the influence of steel bars and water on lining damage detection can be reduced, and the detection rate of surface defects can be greatly improved.

[0007] To achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for detecting tunnel lining damage based on data fusion, comprising the following steps: Acquiring shock wave acoustic frequency data and visible light image data of the tunnel to be inspected; The acquired shock wave acoustic frequency data and visible light image data are input into a pre-established tunnel lining damage detection model based on data fusion to obtain the damage detection result of the tunnel lining to be detected.

[0008] Furthermore, the tunnel lining damage detection model includes: Feature extraction module, feature fusion module, channel attention module and prediction module; The feature extraction module is used to extract visual features and sound features from the input shock wave audio data and visible light image data; The feature fusion module is used to fuse the visual features and the sound features to obtain fusion features in different dimensions; The channel attention module is used to process the fusion features in different dimensions to obtain a set of composite feature vectors; The prediction module is used to predict the damage detection result of the tunnel lining according to the composite feature vector.

[0009] Furthermore, the feature extraction module adopts ResNet50, ResNet101, ResNet34, ResNeXt, WideResNet, DenseNet, EfficientNet, ShuffleNet or MobileNets network model.

[0010] Furthermore, the feature fusion module includes an upsampling module, a connection module, a convolution module, a downsampling module and a fusion module; The upsampling module is used to unify the scales of the sound features and video features of different scales obtained in the feature extraction stage by upsampling using a bilinear interpolation method; The connection module is used to fuse features of different dimensions; The convolution module is used to perform 1×1 convolution on the fused features; The downsampling module is used to downsample the features through 3*3 convolution to unify the fusion features of different stages, so as to obtain the interaction information between different levels of modalities; The fusion module is used to realize multi-scale feature fusion and fuse information features of multiple stages.

[0011] Furthermore, the channel attention module adopts SENet, ECA-Net or BAM network model.

[0012] Furthermore, the SENet model includes: The global average pooling module is used to perform average pooling on the input composite feature vector to obtain a global feature vector; The excitation module is used to build the correlation between feature channels, use the activation function to perform nonlinear processing on the weights, and assign a weight to each feature channel; The output layer is used to multiply the weight of each feature channel by the original composite feature vector, and average pool and fully connect the feature map with weight information to obtain a fused feature map.

[0013] Furthermore, the prediction module uses a softmax function or a binary sigmoid function to predict tunnel lining damage.

[0014] In a second aspect, the present invention provides a tunnel lining damage detection system based on data fusion, comprising: A data acquisition module, used to acquire shock wave acoustic frequency data and visible light image data of the tunnel to be detected; The damage detection module is used to input the acquired shock wave acoustic data and visible light image data into a pre-established tunnel lining damage detection model based on data fusion to obtain the damage detection result of the tunnel lining to be detected.

[0015] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any method.

[0016] In a fourth aspect, the present invention provides a computing device, comprising: one or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, and the one or more programs include instructions for executing any method.

[0017] The present invention adopts the above technical solution, which has the following advantages: (1) The present invention uses impact echo acoustic frequency to detect lining damage. Compared with the electromagnetic waves in the existing method, the impact echo acoustic frequency is less affected by steel bars and water.

[0018] (2) The present invention uses visible light images to detect lining damage. Visible light images can show the texture details of the lining surface and are very sensitive to surface damage.

[0019] (3) The present invention adopts an end-to-end fusion model to fuse the impact echo acoustic data and visible light image data to obtain more accurate and comprehensive lining information and improve the accuracy of lining damage detection.

[0020] The present invention can be widely applied to the technical field of tunnel detection and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Throughout the accompanying drawings, the same reference numerals are used to represent the same components. In the accompanying drawings: Figure 1 is a flow chart of a tunnel lining damage detection method based on data fusion provided by an embodiment of the present invention; Figure 2 is a feature extraction module provided by an embodiment of the present invention; Figure 3 yes Figure 2 Block1 module structure diagram; Figure 4 yes Figure 2 Block2 module structure diagram; Figure 5 is a Skip-Concat module provided by an embodiment of the present invention; Figure 6 It is a SENet network structure diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0023] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0024] In some embodiments of the present invention, a method for detecting tunnel lining damage based on data fusion is provided. In order to fully explore modal associations and maximize the potential interactive information of each modality, the method adopts a multi-level fusion strategy of audio and visual. In order to obtain fusion information of different dimensions, the features extracted at each stage are fused through a fusion module to obtain fusion features under different dimensions, and then the fusion information of each stage is integrated to synthesize a set of composite feature vectors for result prediction. The present invention can reduce the influence of steel bars and water on lining damage detection by fusing impact echo audio data and visible light image data.

[0025] Correspondingly, in other embodiments of the present invention, a tunnel lining damage detection system, device and medium based on data fusion are provided.

[0026] Example 1 like Figure 1 As shown, the present invention provides a tunnel lining damage detection method based on data fusion, which includes the following steps: 1) Obtain shock wave acoustic frequency data and visible light image data of the tunnel to be inspected; 2) The acquired shock wave acoustic frequency data and visible light image data are input into a pre-established tunnel lining damage detection model based on data fusion to obtain the damage detection result of the tunnel lining to be detected.

[0027] Furthermore, in the above step 2), the tunnel lining damage detection model based on data fusion established in this embodiment includes a feature extraction module, a feature fusion module, a channel attention module and a prediction module. Among them, the feature extraction module is used to extract visual features and sound features from the input shock wave audio data and visible light image data; the feature fusion module is used to fuse the visual features and sound features extracted at different stages of the feature extraction module to obtain fusion features in different dimensions; the channel attention module is used to process the fusion features in different dimensions to obtain a set of composite feature vectors; the prediction module is used to predict the tunnel lining damage detection results according to the composite feature vectors.

[0028] Furthermore, in order to better extract visual features and sound features, the feature extraction module in this embodiment is implemented using the deep learning model ResNet50 (Residual Neural Network). At the same time, a certain proportion of the output of the previous network layer is retained during each learning process, and the original input information is allowed to be directly transmitted to the subsequent layer, thereby realizing the fusion of low-dimensional detail features and high-dimensional semantic features, so as to better ensure feature integrity and improve recognition effect.

[0029] Specifically, Figure 2 to Figure 4 As shown, the feature extraction module includes an input layer, the first to fifth feature extraction stages, and an output layer. Among them, the first to fifth feature extraction stages correspond to the five modules of ResNet50, and the module structure can adopt a conventional model, which is not described in detail in the present invention.

[0030] Furthermore, in order to obtain potential modal information under different receptive fields, the feature fusion module in this embodiment adopts a U-shaped fusion architecture, which improves the U-shaped symmetric architecture of the encoding and decoding mode to fuse the modal features of different stages in the feature extraction module through a skip connection fusion (Skip-Concat) module.

[0031] Specifically, Figure 5 As shown in the figure, the feature fusion module includes an up-sampling module (Up Sample), a connection module (Concat), a convolution module (1*1 CONV), a down-sampling module (Down Sample) and a fusion module (Element-wiseAdd).

[0032] Among them, the upsampling module is used to unify the scales of sound features and video features of different scales obtained in the feature extraction stage through the upsampling method of bilinear interpolation; the connection module is used to fuse features of different dimensions to achieve the fusion effect by increasing the channel, thereby ensuring the integrity of the fused features; when the connection module fuses features of different dimensions, the process is as follows: there are N-dimensional visual features and M-dimensional sound features, and the N-dimensional visual features and the M-dimensional sound features are spliced ​​and fused into N+M-dimensional features; the convolution module is used to perform 1×1 convolution on the fused N+M-dimensional features to achieve cross-channel interaction and information integration; the downsampling module is used to unify the fused features of different stages by downsampling the features through 3*3 convolution to obtain the interaction information between modalities at different levels; the fusion module is used to realize multi-scale feature fusion and fuse information features of multiple stages.

[0033] ResNet50 has 5 feature extraction stages with different receptive fields. As the number of network layers increases, the number of combinations of its potential basic network increases. The larger the stage, the more times the features are combined. However, in multiple feature combinations, information loss will occur and some potential information of the modalities will be missed. In order to obtain the intrinsic connection between the modalities at different stages, the Skip-Conat module is connected after each stage is completed to form a U-shaped fusion architecture. At different stages of the feature extraction process, the two modal features are fused by channel, and the modalities are fused while ensuring the integrity of the features. In the fusion process, 1×1 convolution further realizes cross-channel interaction and information integration to improve the efficiency of fusion. Finally, in order to integrate the fusion information of each stage and make the modal interaction closer in different dimensions, the features obtained at different stages are superimposed at the pixel level, and the fused features of each stage are superimposed and fused into a set of composite feature vectors, which not only realizes the fusion mining of more potential complementary features in different dimensions, but also reduces the amount of calculation to a certain extent.

[0034] Furthermore, in order to condense the fusion information while retaining the local information to the greatest extent, this embodiment introduces the channel attention mechanism SENet to model the importance of each feature channel and add different weights to the signals on each feature channel. The larger the weight, the higher the correlation. Through the channel attention mechanism, the information interaction between channels is increased, and audio or video features with obvious characteristics are selected, further improving the fusion accuracy.

[0035] like Figure 6 As shown in the figure, SENet was proposed by Jie et al. to obtain the importance of each feature map by learning feature weights, and then use this importance to assign a weight value to each feature channel, enhance effective information, suppress irrelevant information, and make the model achieve better results.

[0036] Specifically, the SENet module includes: The global average pooling module is used to perform average pooling on the input composite feature vector to obtain a global feature vector. That is, a composite feature vector of H×W×N with a total of N channels is input, and a spatial global average pooling is performed. Each channel obtains a scalar, and the output is 1×1×C. At this time, the feature map obtained has a global receptive field, which is equivalent to the entire spatial information on a channel being compressed into a global feature. The excitation module is used to build the correlation between feature channels, use the activation function to perform nonlinear processing on the weights, and assign a weight to each feature channel; SENet establishes the relationship between different channels and obtains the weight coefficient of each channel. This can enhance the expression of important features and reduce the impact of unimportant features, thereby improving the directionality of features; The output layer is used to multiply the weight of each feature channel by the original composite feature vector, and average pool and fully connect the feature map with weight information to obtain a fused feature map.

[0037] Furthermore, the prediction module uses the softmax function to complete the lining damage prediction. According to the type of lining damage, the following categories can be predicted: cracking, corrosion, back voids, deformation and displacement, lining crushing, spalling and insufficient thickness.

[0038] Softmax is also called the normalized exponential function. It is a generalization of the binary classification function sigmoid in multi-classification, and its purpose is to present the results of multi-classification in the form of probability. It maps the outputs of multiple neurons to the interval (0,1), which can be understood as probability, so as to perform multi-classification.

[0039] Furthermore, the feature extraction module ResNet50 can be replaced by other feature extraction networks such as ResNet101, ResNet34, ResNeXt, WideResNet, DenseNet, EfficientNet, ShuffleNet, MobileNets, etc.

[0040] Furthermore, the channel attention module SENet can be replaced by other modules such as ECA-Net and BAM.

[0041] Furthermore, the prediction module can be replaced with a binary classification sigmoid function.

[0042] Example 2 The above-mentioned embodiment 1 provides a tunnel lining damage detection method based on data fusion. Correspondingly, this embodiment provides a tunnel lining damage detection system based on data fusion. The system provided in this embodiment can implement the tunnel lining damage detection method based on data fusion of embodiment 1, and the system can be implemented by software, hardware, or a combination of software and hardware. For example, the system may include integrated or separate functional modules or functional units to execute the corresponding steps in each method of embodiment 1. Since the system of this embodiment is basically similar to the method embodiment, the process described in this embodiment is relatively simple. For relevant parts, please refer to the partial description of embodiment 1. The embodiment of the system provided in this embodiment is only illustrative.

[0043] The tunnel lining damage detection system based on data fusion provided in this embodiment includes: A data acquisition module, used to acquire shock wave acoustic frequency data and visible light image data of the tunnel to be detected; The damage detection module is used to input the acquired shock wave acoustic data and visible light image data into a pre-established tunnel lining damage detection model based on data fusion to obtain the damage detection result of the tunnel lining to be detected.

[0044] Example 3 This embodiment provides a processing device corresponding to the tunnel lining damage detection method based on data fusion provided in this embodiment 1. The processing device can be a processing device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of embodiment 1.

[0045] The processing device includes a processor, a memory, a communication interface and a bus, and the processor, the memory and the communication interface are connected through the bus to complete mutual communication. The memory stores a computer program that can be run on the processor, and the processor executes the tunnel lining damage detection method based on data fusion provided in this embodiment 1 when running the computer program.

[0046] Preferably, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0047] Preferably, the processor may be a central processing unit (CPU), a digital signal processor (DSP) or other general-purpose processors of various types, which are not limited here.

[0048] Example 4 The tunnel lining damage detection method based on data fusion of this embodiment 1 can be specifically implemented as a computer program product, which may include a computer-readable storage medium carrying computer-readable program instructions for executing the tunnel lining damage detection method based on data fusion described in this embodiment 1.

[0049] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0050] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 One or more processes and / or boxes Figure 1 A function specified in one or more boxes. These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A tunnel lining damage detection method based on data fusion, characterized in that: The following steps are involved: Acquiring shock wave acoustic frequency data and visible light image data of the tunnel to be inspected; The acquired shock wave acoustic frequency data and visible light image data are input into a pre-established tunnel lining damage detection model based on data fusion to obtain the damage detection result of the tunnel lining to be detected.

2. A method for detecting tunnel lining damage based on data fusion according to claim 1, characterized in that: The tunnel lining damage detection model includes: Feature extraction module, feature fusion module, channel attention module and prediction module; The feature extraction module is used to extract visual features and sound features from the input shock wave audio data and visible light image data; The feature fusion module is used to fuse the visual features and the sound features to obtain fusion features in different dimensions; The channel attention module is used to process the fusion features in different dimensions to obtain a set of composite feature vectors; The prediction module is used to predict the damage detection result of the tunnel lining according to the composite feature vector.

3. A method for detecting tunnel lining damage based on data fusion as claimed in claim 2, characterized in that: The feature extraction module adopts ResNet50, ResNet101, ResNet34, ResNeXt, WideResNet, DenseNet, EfficientNet, ShuffleNet or MobileNets network model.

4. A method for detecting tunnel lining damage based on data fusion as claimed in claim 2, characterized in that: The feature fusion module includes an upsampling module, a connection module, a convolution module, a downsampling module and a fusion module; The upsampling module is used to unify the scales of the sound features and video features of different scales obtained in the feature extraction stage by upsampling using a bilinear interpolation method; The connection module is used to fuse features of different dimensions; The convolution module is used to perform 1×1 convolution on the fused features; The downsampling module is used to downsample the features through 3*3 convolution to unify the fusion features of different stages, so as to obtain the interaction information between different levels of modalities; The fusion module is used to realize multi-scale feature fusion and fuse information features of multiple stages.

5. The method for detecting tunnel lining damage based on data fusion according to claim 2, characterized in that: The channel attention module adopts SENet, ECA-Net or BAM network model.

6. A method for detecting tunnel lining damage based on data fusion as claimed in claim 2, characterized in that: The SENet model includes: The global average pooling module is used to perform average pooling on the input composite feature vector to obtain a global feature vector; The excitation module is used to build the correlation between feature channels, use the activation function to perform nonlinear processing on the weights, and assign a weight to each feature channel; The output layer is used to multiply the weight of each feature channel by the original composite feature vector, and average pool and fully connect the feature map with weight information to obtain a fused feature map.

7. The method for detecting tunnel lining damage based on data fusion according to claim 2, characterized in that: The prediction module uses a softmax function or a binary sigmoid function to predict tunnel lining damage.

8. A tunnel lining damage detection system based on data fusion, characterized in that: include: A data acquisition module, used to acquire shock wave acoustic frequency data and visible light image data of the tunnel to be detected; The damage detection module is used to input the acquired shock wave acoustic data and visible light image data into a pre-established tunnel lining damage detection model based on data fusion to obtain the damage detection result of the tunnel lining to be detected.

9. A computer-readable storage medium storing one or more programs, characterized in that: The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any one of the methods of claims 1 to 7.

10. A computing device, characterized in that: include: One or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, wherein the one or more programs include instructions for executing any one of the methods described in claims 1 to 7.

Images