Unmanned aerial vehicle carried visible light and infrared imaging tunnel apparent disaster identification method
Through the visible light and infrared imaging technology carried by the drone, combined with the three-dimensional convolutional neural network and support vector machine, the problems of low efficiency and safety risks of tunnel disaster detection are solved, and accurate identification and severity division of tunnel disasters are achieved. , improves patrol efficiency and safety.
Patent Information
- Application Number
- CN202411570265.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-06
- Publication Date
- 2024-12-03
AI Technical Summary
Existing tunnels have a reliance on manual inspections, which are inefficient and have safety risks. Especially in the absence of light or complex environment, it is difficult to detect disasters with strong concealment or slightest concealment, which affects the long-term stability of the tunnel structure.
Using visible light and infrared imaging technology carried by the drone, the three-dimensional convolutional neural network and support vector machine can quickly identify the apparent disasters in the tunnel, extract spatiotemporal and spatial characteristics, and automatically classify the severity.
It realizes accurate identification and severity division of apparent disasters in tunnels, reduces the risks of construction workers, improves patrol efficiency and safety, and can conduct accurate monitoring in areas with complex construction environments or difficult to enter manually.
Smart Images

Figure CN119068371A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of geological body identification, and in particular relates to a tunnel surface disaster identification method using visible light and infrared imaging carried by an unmanned aerial vehicle. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] With the construction of underground projects, human beings are using underground space more and more widely. However, leakage, cracks and other problems may occur in the early stage of tunnel construction or operation, affecting the normal construction or operation of the tunnel and causing certain economic losses. However, the detection of apparent disasters in existing tunnels mainly relies on manual inspections, such as visual inspection, taking photos or video recordings, and manual judgment, which is easily affected by factors such as ambient light, complex structures in the tunnel, etc., especially in the case of insufficient light and narrow space. Some hidden or minor disasters are difficult to be discovered through manual inspections in the early stage, thus missing the best prevention and control period, leading to the deterioration of the disaster source. However, in some tunnels with serious structural damage or harsh environment, manual entry may increase the risk of safety accidents. From this, it can be seen that manual detection is not only inefficient, but also has a high safety risk. In addition, manual detection is highly subjective and easily misses hidden dangers such as small cracks and water seepage, affecting the long-term stability of the tunnel structure.
[0004] Therefore, it is crucial to adopt advanced and scientific inspection technology during the construction and operation periods of tunnels, so as to prevent disasters before they happen, reduce disaster risks and improve safety. During the construction period, due to the complex construction environment or areas that are difficult for people to enter, drone inspection technology is used to reduce the risk of collapse and leakage caused by geological instability during the inspection process. During the operation period, drone inspection technology can be used to carry out inspections without stopping operation, capturing small cracks, peeling or other potential problems on the surface of the tunnel. Summary of the invention
[0005] In order to solve the above problems, the present invention proposes a tunnel surface disaster identification method based on visible light and infrared imaging carried by an unmanned aerial vehicle. The present invention can quickly identify tunnel surface disasters based on visible light and infrared imaging, and can accurately and quickly identify tunnel surface disasters.
[0006] According to some embodiments, the present invention adopts the following technical solutions: A method for identifying tunnel surface hazards by visible light and infrared imaging carried by an unmanned aerial vehicle comprises the following steps: Obtain the image and video information of the target tunnel side wall, classify the acquired image and video information, establish a tunnel disaster interpretation database, and train a deep neural network learning model for tunnel disaster identification; The trained deep neural network learning model is used to extract features from the tunnel side wall image information; Based on the extracted features, the abnormal disaster types are identified and the disaster severity is classified using support vector machines.
[0007] As an optional implementation, the image video information includes visible light images taken by a drone when the light intensity / visibility is greater than a set value, and infrared images when the light intensity / visibility is less than or equal to the set value.
[0008] As an optional implementation, the features include information on potential cracks, deformation, collapse, landslide, seepage, water gushing and stability of rock mass after blasting on tunnel side walls extracted during the construction period, and information on cracks, deformation, seepage and exposed steel bars in the tunnel extracted during the operation period.
[0009] As an optional implementation, the neural network learning model is a three-dimensional convolutional neural network model, which is used to extract spatial features from images or video data, and includes an input layer, a 3D convolution layer, a 3D pooling layer, a fully connected layer and an output layer, wherein: The input layer is used to input the video data information of visible light and infrared imaging into the system, wherein the video data information includes video frame sequence, video width, video height, video time, and video length; The 3D convolution layer is used to process the spatial image information and the changes in the time dimension in the video information to capture the spatiotemporal characteristics of the disaster; The 3D pooling layer is used to reduce the dimension of the feature map, retain the most important feature information, and improve the computational efficiency of the model; The fully connected layer is used to integrate and classify the spatial and temporal features extracted by the 3D convolution layer and the 3D pooling layer; The output layer is used to output the categories of tunnel disasters, which include cracks, deformation, water seepage and leakage, collapse, landslide and stability of rock mass after blasting, and exposed steel bars.
[0010] As a further implementation, the shape of the convolution kernel of the three-dimensional convolutional neural network model is K d ×K h ×K w , at time K d and height K h and width K w Convolution operations are performed on the images to capture the characteristics of disaster development over time.
[0011] As a further implementation, the output disaster category features identified by the three-dimensional convolutional neural network model are as follows: The crack characteristics include the length, width, depth, direction and development speed of the crack; The deformation characteristics include the apparent morphology of the tunnel wall, the local or overall deformation in the tunnel, and the deformation rate of the tunnel wall or other structure over time; The water seepage and leakage characteristics include the distribution range of water stains or water marks in the tunnel; The stability characteristics of the rock mass after collapse, landslide and blasting include the scope, shape, specific location of the internal surface of the tunnel, the number and density of the fallen blocks; The exposed steel bar characteristics include whether the steel bar surface is exposed or corroded.
[0012] As a further implementation, the convolution calculation method of the three-dimensional convolutional neural network model is as follows: ; in: y(d, h, w, c) is the value of the output feature map, including depth, height, width and channel; is the value of the input feature map; is the weight of the convolution kernel, indicating the position of the convolution kernel in the time direction i, height direction j, and width direction k, and the input channel is c in , the output channel is the weight value of c.
[0013] As an optional implementation, the support vector machine determines the severity of the tunnel disaster based on supervised learning, and determines the severity according to the disaster features extracted by the three-dimensional convolutional neural network.
[0014] As a further implementation, the support vector machine further classifies the features extracted by the three-dimensional convolutional neural network, and finds classification boundaries of different severity levels through support vectors.
[0015] As a further implementation method, the output results of the support vector machine include: Level I representing a major dangerous situation, Level II representing a relatively large dangerous situation, and Level III representing a general dangerous situation.
[0016] The drone is equipped with visible light and infrared camera equipment to collect videos and images of the inner wall of the tunnel.
[0017] A tunnel surface hazard identification system using visible light and infrared imaging carried by an unmanned aerial vehicle, comprising: The recognition model training module is configured to obtain image and video information of the target tunnel side wall, classify the obtained image and video information, establish a tunnel disaster interpretation database, and train a tunnel disaster recognition deep neural network learning model; A feature extraction module is configured to extract features from tunnel side wall image information using a trained deep neural network learning model; The disaster identification and classification module is configured to identify abnormal disaster types based on the extracted features and classify the severity of disasters using a support vector machine.
[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method for identifying tunnel surface hazards by using visible light and infrared imaging carried by an unmanned aerial vehicle. The method realizes the identification of tunnel surface hazards by using visible light and infrared light carried by an unmanned aerial vehicle, solves the problem of complex construction environments or areas that are difficult for humans to enter, and reduces the risk of collapse and leakage caused by geological instability during the inspection process for construction personnel.
[0019] The present invention provides a tunnel surface disaster identification method using visible light and infrared imaging carried by an unmanned aerial vehicle. By adopting a three-dimensional convolutional neural network, various features related to tunnel disasters can be identified from the time and space dimensions, thus helping to carry out accurate disaster monitoring and assessment. These features provide a reliable basis for subsequent disaster classification, severity division and early warning.
[0020] The present invention provides a tunnel surface disaster identification method using visible light and infrared imaging carried by an unmanned aerial vehicle. Combined with a three-dimensional convolutional neural network and a support vector machine, the method can realize automatic classification of the severity of tunnel disasters. The three-dimensional convolutional neural network is used for feature extraction, and the support vector machine is used for the final classification decision. By inputting the extracted features into the support vector machine, the effective division of the severity of the disaster is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0022] Figure 1 A flowchart of tunnel surface hazard identification based on drone according to an embodiment; Figure 2 A schematic diagram of a drone tunnel identification system according to an embodiment; Figure 3 A schematic diagram of a three-dimensional convolutional neural network structure of an embodiment; Figure 4 A support vector machine flow chart of an embodiment.
[0023] Among them: 1. Tunnels; 2. Drones; 3. Infrared / visible light photography. DETAILED DESCRIPTION
[0024] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.
[0025] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0026] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.
[0027] In the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other.
[0028] Embodiment 1 A method for identifying tunnel surface hazards by using visible light and infrared imaging carried by an unmanned aerial vehicle is provided. The method identifies tunnel surface hazards by using visible light and infrared imaging carried by an unmanned aerial vehicle. Figure 1 As shown, the specific steps include: (1) Import the location information of the monitoring tunnel into the drone.
[0029] (2) Plan the flight path of UAV 2 according to the location information of Tunnel 1. First, model the environment of Tunnel 1 and use a random sampling path planning algorithm suitable for obstacle avoidance in a dynamic environment. Then, make UAV 2 take photos of the area of Tunnel 1 that needs to be inspected and perform infrared / visible light photography 3, such as Figure 2 shown.
[0030] (3) Choose the appropriate photography method according to the tunnel environment.
[0031] The image information of the target tunnel side wall taken by the UAV is obtained, and the image video information includes a visible light image when the light intensity / visibility is greater than a set value, and an infrared image when the light intensity / visibility is less than or equal to the set value.
[0032] (4) Input the collected video or image into a three-dimensional convolutional neural network (3D convolutional neural network) to extract information features from the video or image, such as Figure 3 shown.
[0033] In this process, the disaster areas in visible light and infrared imaging are used as positive samples, and the non-disaster areas in visible light and infrared imaging are used as negative samples; the constructed samples are trained using a neural network to obtain a trained tunnel disaster recognition neural network.
[0034] The trained deep neural network learning model is used to extract features from the tunnel side wall image information. During the construction period, the potential cracks, deformation, collapse, landslide, seepage, water gushing, and stability of the rock mass after blasting are extracted. During the operation period, the tunnel cracks, deformation, seepage, and exposed steel bars are extracted; abnormal disaster types are identified.
[0035] In this embodiment, during the training process of the deep neural network learning model, historical dam hidden danger images are used as training samples for training and testing.
[0036] In this embodiment, the neural network model is a three-dimensional convolutional neural network (3D-CNN), which can extract spatial features from images or video data.
[0037] like Figure 3 As shown, the 3D-CNN convolutional neural network includes: input layer, 3D convolution layer, 3D pooling layer, and fully connected layer, where: The input layer inputs visible light and infrared imaging videos, such as video frame sequences, including video width, height, and time.
[0038] The 3D convolution layer is capable of simultaneously processing spatial image information and changes in the temporal dimension in video data, thereby capturing the spatiotemporal characteristics of disasters.
[0039] The 3D pooling layer is used to reduce the dimension of the feature map, retain the most important feature information, and improve the computational efficiency of the model.
[0040] The fully connected layer is used to integrate and classify features after the convolution layer and the pooling layer extract the spatial and temporal features, so as to judge the type and severity of the disaster or predict the future development of the disaster.
[0041] The output layer is used in the classification task to finally output the category of tunnel disasters (such as cracks, leakage, deformation, etc.).
[0042] In this embodiment, the shape of the convolution kernel of the 3D-CNN neural network model is K d ×K h ×K w , that is, convolution operations are performed in time, height, and width to capture the characteristics of disaster development over time.
[0043] In this embodiment, the 3D-CNN neural network model identifies and outputs disaster features as follows: The crack characteristics mainly extract the geometric characteristics of the cracks, such as length, width, depth, crack direction and crack development speed.
[0044] The deformation features described above mainly extract the apparent morphology of the tunnel wall, and can capture the local or overall deformation conditions in the tunnel and the deformation rate of the tunnel wall or other structures over time.
[0045] The water seepage and leakage features described above mainly extract the distribution range of water stains or water traces in the tunnel, thereby determining the potential leakage location and analyzing the magnitude, flow rate and other information of the water seepage according to the dynamic changes of the water flow in the image.
[0046] The stability characteristics of the rock mass after collapse, landslide and blasting mainly extract the scope, shape, specific location of the internal surface peeling and falling blocks of the tunnel, and identify the number and density of falling blocks.
[0047] The exposed steel bar feature mainly extracts whether the steel bar surface is exposed or corroded.
[0048] The convolution calculation method of the 3D-CNN neural network model of this embodiment is as follows: ; in: y(d, h, w, c) is the value of the output feature map, including depth (i.e. time or frame sequence), height, width and channel.
[0049] is the value of the input feature map.
[0050] is the weight of the convolution kernel, indicating that the convolution kernel is at the position i (time direction), j (height direction), and k (width direction), and the input channel is c in , the output channel is the weight value of c.
[0051] (5) The features extracted by the three-dimensional convolutional neural network (3D CNN) are input into the support vector machine to classify the severity of tunnel apparent disasters.
[0052] like Figure 4 As shown, in this embodiment, the support vector machine (SVM) is based on supervised learning to judge the severity of the tunnel disaster, and its severity is judged according to the disaster features extracted by the 3D-CNN neural network.
[0053] Support vector machine (SVM) is a further classification of the features extracted by 3D-CNN. It performs efficient classification based on the features of 3D-CNN and can find the optimal classification boundaries of different severity levels through support vectors.
[0054] The output results of the support vector machine (SVM) can adopt multiple categories of labels according to actual needs, including: Level I (major danger), Level II (relatively dangerous situation), and Level III (general danger).
[0055] Embodiment 2 A tunnel surface hazard identification system using visible light and infrared imaging carried by an unmanned aerial vehicle, comprising: The recognition model training module is configured to obtain image and video information of the target tunnel side wall, classify the obtained image and video information, establish a tunnel disaster interpretation database, and train a tunnel disaster recognition deep neural network learning model; A feature extraction module is configured to extract features from tunnel side wall image information using a trained deep neural network learning model; The disaster identification and classification module is configured to identify abnormal disaster types based on the extracted features and classify the severity of disasters using a support vector machine.
[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principle of the present invention without creative labor shall be included in the protection scope of the present invention.
Claims
1. A tunnel surface hazard identification method using visible light and infrared imaging carried by an unmanned aerial vehicle, characterized in that: The following steps are involved: Obtain the image and video information of the target tunnel side wall, classify the acquired image and video information, establish a tunnel disaster interpretation database, and train a deep neural network learning model for tunnel disaster identification; The trained deep neural network learning model is used to extract features from the tunnel side wall image information; Based on the extracted features, the abnormal disaster types are identified and the disaster severity is classified using support vector machines.
2. The tunnel surface hazard identification method using visible light and infrared imaging carried by an unmanned aerial vehicle as claimed in claim 1, characterized in that: The image and video information includes visible light images taken by the drone when the light intensity / visibility is greater than a set value, and infrared images taken when the light intensity / visibility is less than or equal to the set value.
3. The tunnel surface hazard identification method using visible light and infrared imaging carried by an unmanned aerial vehicle as claimed in claim 1, characterized in that: The features include information on potential cracks, deformation, collapse, landslide, seepage, water gushing and stability of rock mass after blasting on tunnel side walls extracted during the construction period, and information on cracks, deformation, seepage and exposed steel bars in the tunnel extracted during the operation period.
4. The tunnel surface hazard identification method using visible light and infrared imaging carried by an unmanned aerial vehicle as claimed in claim 1, characterized in that: The neural network learning model is a three-dimensional convolutional neural network model, which is used to extract spatial features from images or video data, including an input layer, a 3D convolution layer, a 3D pooling layer, a fully connected layer and an output layer, wherein: The input layer is used to input the video data information of visible light and infrared imaging into the system, wherein the video data information includes video frame sequence, video width, video height, video time, and video length; The 3D convolution layer is used to process the spatial image information and the changes in the time dimension in the video information to capture the spatiotemporal characteristics of the disaster; The 3D pooling layer is used to reduce the dimension of the feature map, retain the most important feature information, and improve the computational efficiency of the model; The fully connected layer is used to integrate and classify the spatial and temporal features extracted by the 3D convolution layer and the 3D pooling layer; The output layer is used to output the categories of tunnel disasters, which include cracks, deformation, water seepage and leakage, collapse, landslide and stability of rock mass after blasting, and exposed steel bars.
5. The tunnel surface hazard identification method using visible light and infrared imaging carried by an unmanned aerial vehicle as claimed in claim 4, characterized in that: The shape of the convolution kernel of the three-dimensional convolutional neural network model is K d ×K h ×K w , at time K d and height K h and width K w Convolution operations are performed on the images to capture the characteristics of disaster development over time.
6. The tunnel surface hazard identification method using visible light and infrared imaging carried by an unmanned aerial vehicle as claimed in claim 4, characterized in that: The output disaster category characteristics identified by the three-dimensional convolutional neural network model are as follows: The crack characteristics include the length, width, depth, direction and development speed of the crack; The deformation characteristics include the apparent morphology of the tunnel wall, the local or overall deformation in the tunnel, and the deformation rate of the tunnel wall or other structure over time; The water seepage and leakage characteristics include the distribution range of water stains or water marks in the tunnel; The stability characteristics of the rock mass after collapse, landslide and blasting include the scope, shape, specific location of the internal surface of the tunnel, the number and density of the fallen blocks; The exposed steel bar characteristics include whether the steel bar surface is exposed or corroded.
7. The tunnel surface hazard identification method using visible light and infrared imaging carried by an unmanned aerial vehicle as claimed in claim 4, 5 or 6, characterized in that: The convolution calculation method of the three-dimensional convolutional neural network model is as follows: ; in: y(d, h, w, c) is the value of the output feature map, including depth, height, width and channel; is the value of the input feature map; is the weight of the convolution kernel, indicating the position of the convolution kernel in the time direction i, height direction j, and width direction k, and the input channel is c in , the output channel is the weight value of c.
8. The tunnel surface hazard identification method using visible light and infrared imaging carried by an unmanned aerial vehicle as claimed in claim 4, characterized in that: The support vector machine determines the severity of the tunnel disaster based on supervised learning and determines its severity based on the disaster features extracted by the three-dimensional convolutional neural network; The support vector machine further classifies the features extracted by the three-dimensional convolutional neural network, and finds classification boundaries of different severity levels through support vectors.
9. The tunnel surface hazard identification method using visible light and infrared imaging carried by an unmanned aerial vehicle as claimed in claim 8, characterized in that: The output results of the support vector machine include: Level I representing a major dangerous situation, Level II representing a relatively large dangerous situation, and Level III representing a general dangerous situation.
10. A tunnel surface hazard identification system using visible light and infrared imaging carried by an unmanned aerial vehicle, characterized in that: include: The recognition model training module is configured to obtain image and video information of the target tunnel side wall, classify the obtained image and video information, establish a tunnel disaster interpretation database, and train a tunnel disaster recognition deep neural network learning model; A feature extraction module is configured to extract features from tunnel side wall image information using a trained deep neural network learning model; The disaster identification and classification module is configured to identify abnormal disaster types based on the extracted features and classify the severity of disasters using a support vector machine.
Citation Information
Patent Citations
Tunnel detection method and device based on unmanned aerial vehicle
CN113284144A
Mountain torrent flood video identification monitoring method based on deep learning
CN116778395A
Deep learning based method for automatic geological disaster extraction from unmanned aerial vehicle image
WO2024000927A1
Cited By
Self-navigation tunnel crack detection method, system, equipment and medium
CN120259891A
A self-guided tunnel crack detection method, system, device and medium
CN120259891B
Detection method and system based on diversion tunnel structural joint
CN121207990A
