Dam piping identification method and system based on unmanned aerial vehicle carrying visible light and infrared imaging

Through the drone equipped with visible light and infrared imaging technology, combined with deep neural network model, we can identify the characteristics of dam pipe surges and predict dangerous situations, solving the problem of dam pipe surge hidden danger identification in the existing technology, and achieving rapid and accurate detection of dam risk and analysis of hazards. Improved levee safety.

CN119068372APending Publication Date: 2024-12-03SHANDONG UNIV +1

Patent Information

Application Number
CN202411570270.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-06
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing technology cannot quickly and accurately identify hidden dangers and dangerous situations of dam pipe surges, and cannot intelligently judge the dangerous situation, which makes it difficult to prevent the risk of dam collapse.

Method used

UAVs are equipped with visible light and infrared imaging technology, combined with deep neural network models (such as Seg-Net) to extract and divide the dam pipe surge characteristics and risk levels. Through multi-scale and multi-level spatial analysis and hierarchical analysis methods, the pipes of pipe surges are identified. The size of the mouth, head height, distance from the embankment, flow rate, turbidity and obvious degree of sand ring are predicted to the degree of disaster hazards.

Benefits of technology

It realizes fast and accurate detection of hidden dangers and investigation of dangerous situations, and can intelligently judge the dangerous situation status, guide early warnings, and improves the safety of dikes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068372A_ABST
    Figure CN119068372A_ABST
Patent Text Reader

Abstract

The invention belongs to the field of dam hidden danger identification, and provides an unmanned aerial vehicle-carried visible light and infrared imaging dam piping identification method and system, and the method comprises the steps: obtaining the back image information of a target dam shot by an unmanned aerial vehicle, and enabling the image information to comprise a visible light image when the light intensity / visibility is greater than a set value; the light intensity / visibility is smaller than or equal to a set value; performing feature extraction on the back image information of the target dam by using a pre-trained deep neural network learning model to obtain the pipe orifice size, the water head height, the distance from the dam, the flow, the turbidity and the sand ring obvious degree of piping; and based on the extracted features, establishing a judgment matrix to calculate factor weight vectors of the features, and comprehensively dividing dam piping risk levels according to factor weight vector information and the features. According to the invention, the method can rapidly recognize the lifting piping, can achieve the rapid and accurate detection of dike hidden troubles and the troubleshooting and analysis of dangerous cases, and can intelligently judge the state of the dangerous cases, so as to guide the early warning.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of dam hidden danger identification, and specifically relates to a dam piping identification method and system using an unmanned aerial vehicle equipped with visible light and infrared imaging. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Hidden dangers in levees are the main factors that endanger the safe operation of levees. During the long-term service, the levees are gradually raised and thickened. Due to uneven local compaction, uneven settlement will occur, resulting in cracks and gaps. Some are formed by the erosion of animals such as badgers, rats, snakes, and termites, or the holes formed by the decay of tree roots and piles buried inside the levees, as well as the residues of hidden pipes and abandoned wells that were not cleaned during the levee repair. Due to the existence of the above hidden dangers, when encountering floods, the levees often suffer from leakage, pipe bursts, landslides, collapses and other dangerous situations, which will lead to the collapse of the levees in serious cases.

[0004] Therefore, using advanced, fast and accurate detection technology to conduct quality inspections on levees during the non-flood season can prevent disasters before they occur; using advanced, fast and accurate levee hidden danger detection and dangerous situation investigation technology during the flood season can effectively prevent levee breaches and dike bursts; at the same time, the basis for levee risk management is to use large-scale management methods for the basic levee data detected and investigated during the non-flood and flood seasons, conduct machine learning mining and analysis of hidden dangers and dangerous situations, and intelligently judge the dangerous situation status.

[0005] In recent years, many scholars at home and abroad have continuously studied and explored a large number of methods, technologies, instruments and equipment for levee hidden danger detection and dangerous situation inspection. For example, China's invention patent CN 112184628 B, a flood prevention and danger inspection system and method for levee patrol with infrared dual-wave images and cloud warning, provides a technical solution for using drones to collect images and performing image processing and early warning on the ground cloud. However, it can only perform quality inspections on levees, and cannot achieve rapid and accurate levee hidden danger detection and dangerous situation inspection, analysis, and intelligent judgment of dangerous situation status. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a method and system for identifying dam piping using drones equipped with visible light and infrared imaging. The present invention can quickly identify and prevent dam piping hazards, and predict the degree of disaster damage. It can achieve rapid and accurate detection of dam hazard hazards and dangerous situation investigation, and analyze and intelligently judge the dangerous situation status to guide early warning.

[0007] According to some embodiments, the present invention adopts the following technical solutions: A method for identifying dam piping by using an unmanned aerial vehicle (UAV) equipped with visible light and infrared imaging, comprising the following steps: Acquire image information of the back side of the target dam taken by the drone, the image information including visible light images when the light intensity / visibility is greater than a set value, and infrared images when the light intensity / visibility is less than or equal to the set value; The pre-trained deep neural network learning model is used to extract the features of the image information on the back of the target dam, and the size of the pipe mouth, head height, distance from the dam, flow rate, turbidity and sand ring visibility of the pipe burst are extracted; Based on the extracted features, a judgment matrix is ​​established to calculate the factor weight vector of each feature. According to the factor weight vector information and characteristics, the risk level of dam piping is divided in combination with multi-scale and multi-level spatial analysis.

[0008] As an optional implementation, during the training process of the deep neural network learning model, historical dam hidden danger images are used as training samples for training and testing.

[0009] As an optional implementation, the deep neural network learning model is a Seg-Net image semantic segmentation network model, including an image input layer, an encoder network, a decoder network, a softmax classifier and a pixel segmentation result output layer connected in sequence, wherein: The encoder network consists of a convolutional layer and a pooling layer, which is used to extract image features and generate feature maps; The decoder network is used to restore the feature map to the same spatial resolution as the original input image and generate a segmentation result; The softmax classifier is used to predict the category of each pixel, which is connected to the last layer of the decoder.

[0010] As a further implementation, the loss function of the Seg-Net image semantic segmentation network model is calculated using a cross entropy method, which is applicable to multi-classification problems, and its calculation formula is as follows: ; Where M is the number of categories; is a symbolic function, if the sample i If the true category of is equal to category c, it takes 1, otherwise it takes 0; For sample i Belongs to category c The predicted probability of .

[0011] As an optional implementation method, based on the extracted features, the process of establishing a judgment matrix to calculate the factor weight vector of each feature includes: using the hierarchical analysis method to assign different values ​​according to the weight of the piping influencing factors, and the evaluation index weight function is as follows: ; ; Among them: a ij is the weight of each influencing factor, n is the number of influencing factors, is the factor vector weight.

[0012] The pre-training process of the deep neural network learning model includes: Collect historical piping image data, the data including information about pipe orifice, water head, distance from bank, flow, turbidity and sand ring visibility, to form a data set; Annotate the data set, including marking each pixel in the image as a corresponding piping feature category; The Seg-Net image semantic segmentation network model is trained using a dataset with labeled information. During the training process, the model will learn to extract features from the input image and correctly classify each pixel.

[0013] As an optional implementation, the process of extracting features from the image information on the back of the target dam using a pre-trained deep neural network learning model includes: the orifice size feature extraction is determined by calculating the area or perimeter of the orifice region in the segmentation result; The water head height is obtained by converting the pixel height into the actual height using the image coordinate system; The distance from the dike is calculated by dividing the pipe opening and the dike area in the result of segmentation. The flow rate is directly predicted by the training model. The turbidity is estimated by analyzing the color and texture characteristics of the water area in the segmentation result; The degree of sand ring visibility is determined by calculating the area and shape characteristics of the sand ring region in the segmentation result.

[0014] As an optional implementation, the multi-scale, multi-level spatial analysis includes a macro-scale level, a meso-scale level and a local-scale level, wherein the macro-scale level is to analyze the hydrogeological conditions in the entire basin or region and the dangerous conditions in the entire basin, including flood risks and upstream water volume; The mesoscale level described is to analyze the structure and status of the entire dam; The local scale level is to extract characteristic information of dam piping based on drone photography to comprehensively analyze the development of piping hazards.

[0015] A dam piping identification system using visible light and infrared imaging carried by an unmanned aerial vehicle, comprising: The drone is equipped with an image acquisition device for acquiring image information of the back side of the target dam taken by the drone, wherein the image information includes a visible light image when the light intensity / visibility is greater than a set value, and an infrared image when the light intensity / visibility is less than or equal to the set value; The feature extraction module is used to extract features from the image information on the back of the target dam using a pre-trained deep neural network learning model, and extract the size of the pipe mouth, water head height, distance from the dam, flow rate, turbidity, and sand ring visibility of the piping; The dam piping identification module is used to establish a judgment matrix based on the extracted features to calculate the factor weight vector of each feature, and comprehensively classify the dam piping risk level according to the factor weight vector information and features.

[0016] As an optional implementation, the drone is equipped with an image acquisition device to acquire images of the back side of the target dam according to a preset inspection path; The image acquisition device includes visible light and infrared light image acquisition devices.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a method and system for identifying dam piping using visible light and infrared imaging carried by drones. Piping identification is achieved during the day and at night by using visible light and infrared imaging carried by drones. Piping feature information is extracted by using the Seg-Net network structure, including the size of the piping orifice, water head height, distance from the embankment, flow, turbidity, and sand ring visibility of the piping. The present invention uses a hierarchical analysis method based on the feature information to comprehensively judge the piping risk level, thereby achieving a more scientific and accurate positioning of the risk of piping disasters.

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0020] Figure 1 A flowchart of a method for identifying dam piping using visible light and infrared imaging by a drone according to an embodiment; Figure 2 A schematic diagram of a dam piping identification system using visible light and infrared imaging carried by a drone according to an embodiment; Figure 3 A schematic diagram of a Seg-Net neural network structure according to an embodiment; Figure 4This is a thermal imaging effect diagram of piping in a drone according to an embodiment; Figure 5 The following is a flowchart of the hierarchical analysis method according to an embodiment of the present invention.

[0021] Among them: 1. Dam; 2. Drone; 3. Infrared / visible light photography; 4. Pipe burst; 5. Infrared imaging. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0023] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0024] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0025] In the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other.

[0026] Embodiment 1 A method for identifying dam piping using visible light and infrared imaging carried out by drones is used to identify the dam piping hazards through drone projection. Figure 1 As shown, the specific steps include: Step (1): Import the location information of the dam inspection into the drone.

[0027] Step (2): Plan the flight path of UAV 2 according to the location information of dam 1. First, build a model based on dam 1 and its surrounding environment, and use a random sampling path planning algorithm, which is suitable for obstacle avoidance in a dynamic environment.

[0028] Step (3): Use the drone 2 to carry the detection equipment for inspection, take photos of the back of the dam 1, and perform infrared / visible light photography 3 to determine whether there is a pipe burst 4, such as Figure 2 shown.

[0029] Specifically, the drone can be equipped with visible light photography during the day and infrared light photography at night.

[0030] The pre-trained deep neural network learning model is based on a large collection of piping image data, which contains detailed information about pipe orifices, water heads, distances from dikes, flow rates, turbidity, and sand ring visibility.

[0031] The prepared dataset is labeled. The labeling task includes marking each pixel in the image as a corresponding piping feature category, such as pipe orifice, water head, distance from dike, flow, turbidity, sand ring, etc.

[0032] The Seg-Net image semantic segmentation network model is trained using a dataset with labeled information. During the training process, the model will learn how to extract features from the input image and correctly classify each pixel.

[0033] Step (4): Input the collected image information into the Seg-Net neural network, and use the Seg-Net neural network to extract the image information features, such as Figure 3 The results include the size of the pipe orifice, water head height, distance from the embankment, flow rate, turbidity and the degree of sand ring. Figure 4 As shown, the rectangular frame is the infrared imaging 5 of the pipe burst.

[0034] The Seg-Net network structure is an end-to-end deep learning semantic segmentation model, which realizes semantic segmentation output through image input, encoder network, decoder network, softmax classifier, and pixel segmentation result output.

[0035] The encoder network is responsible for extracting image features and generating feature maps, and is composed of convolutional layers and pooling layers.

[0036] The decoder network restores the feature map to the same spatial resolution as the original input image and generates a segmentation result.

[0037] The softmax classifier is used to predict the category of each pixel, which is the last layer of the decoder.

[0038] The loss function of the Seg-Net network structure is calculated using the cross entropy method, which is applicable to multi-classification problems. The calculation formula is as follows: ; Where: M is the number of categories; is a symbolic function, if the sample i If the true category of is equal to category c, it takes 1, otherwise it takes 0; For sample i Belongs to category c The predicted probability of .

[0039] Step (5): Based on the characteristic information, intelligent analysis is performed to classify the risk level of dam piping.

[0040] The nozzle size feature extraction is to calculate the area or perimeter of the nozzle area in the segmentation result; The water head height is obtained by converting the pixel height into the actual height using the image coordinate system; The distance from the dike is calculated by dividing the pipe opening and the dike area in the result of segmentation. The flow rate is the flow value directly predicted by the training model; The turbidity is estimated by analyzing the color and texture characteristics of the water area in the segmentation result; The sand ring conspicuity is calculated by calculating the area and shape characteristics of the sand ring region in the segmentation result.

[0041] In this embodiment, the characteristic information is divided, and a multi-scale, multi-level dam danger level evaluation method is established in combination with spatial information, and the analytic hierarchy process is used to determine the risk level of piping.

[0042] The analytic hierarchy process assigns different values ​​according to the weights of the factors affecting piping, and the weight function of the evaluation index is as follows: ; ; Among them: a ij is the weight of each influencing factor, n is the number of influencing factors, is the factor vector weight.

[0043] like Figure 5 As shown, establish the judgment matrix at each level Calculate factor weight vector .

[0044] In this embodiment, the risk level of dam piping is divided according to the weight vector information and the spatial information, including: Level I (major danger), Level II (relatively dangerous situation), and Level III (general danger).

[0045] Thresholds can be set for the analysis level classification standards: risk level thresholds can be set based on historical data and expert opinions. For example, risk scores between 0 and 30 are low risk, 31 to 60 are medium risk, and 61 or above are high risk.

[0046] Embodiment 2 A dam piping identification system using visible light and infrared imaging carried by an unmanned aerial vehicle, comprising: The drone is equipped with an image acquisition device for acquiring image information of the back side of the target dam taken by the drone, wherein the image information includes a visible light image when the light intensity / visibility is greater than a set value, and an infrared image when the light intensity / visibility is less than or equal to the set value; The feature extraction module is used to extract features from the image information on the back of the target dam using a pre-trained deep neural network learning model, and extract the size of the pipe mouth, water head height, distance from the dam, flow rate, turbidity, and sand ring visibility of the piping; The dam piping identification module is used to establish a judgment matrix based on the extracted features to calculate the factor weight vector of each feature, and comprehensively classify the dam piping risk level according to the factor weight vector information and features.

[0047] In this embodiment, the drone is equipped with an image acquisition device to acquire images of the back of the target dam according to a preset inspection path; The image acquisition device includes visible light and infrared light image acquisition devices.

[0048] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of one or more computer-usable storage media (including but not limited to disk storage, CD - ROM , optical storage, etc.).

[0049] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0050] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0051] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0052] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principle of the present invention without creative labor shall be included in the protection scope of the present invention.

Claims

1. A method for identifying dam piping using visible light and infrared imaging by using an unmanned aerial vehicle, characterized in that: The following steps are involved: Acquire image information of the back side of the target dam taken by the drone, the image information including visible light images when the light intensity / visibility is greater than a set value, and infrared images when the light intensity / visibility is less than or equal to the set value; The pre-trained deep neural network learning model is used to extract the features of the image information on the back of the target dam, and the size of the pipe mouth, head height, distance from the dam, flow rate, turbidity and sand ring visibility of the pipe burst are extracted; Based on the extracted features, a judgment matrix is ​​established to calculate the factor weight vector of each feature. According to the factor weight vector information and characteristics, the risk level of dam piping is divided in combination with multi-scale and multi-level spatial analysis.

2. The method for identifying dam piping by using visible light and infrared imaging with an unmanned aerial vehicle as claimed in claim 1, characterized in that: During the training process of the deep neural network learning model, historical dam hidden danger images are used as training samples for training and testing.

3. The method for identifying dam piping by using visible light and infrared imaging with an unmanned aerial vehicle as claimed in claim 1, characterized in that: The deep neural network learning model is a Seg-Net image semantic segmentation network model, which includes an image input layer, an encoder network, a decoder network, a softmax classifier and a pixel segmentation result output layer connected in sequence, wherein: The encoder network consists of a convolutional layer and a pooling layer, which is used to extract image features and generate feature maps; The decoder network is used to restore the feature map to the same spatial resolution as the original input image and generate a segmentation result; The softmax classifier is used to predict the category of each pixel, which is connected to the last layer of the decoder.

4. The method for identifying dam piping by using visible light and infrared imaging with an unmanned aerial vehicle as claimed in claim 3, characterized in that: The loss function of the Seg-Net image semantic segmentation network model is calculated using the cross entropy method, which is applicable to multi-classification problems. The calculation formula is as follows: ; Where M is the number of categories; is a symbolic function, if the sample i If the true category of is equal to category c, it takes 1, otherwise it takes 0; For sample i Belongs to category c The predicted probability of .

5. The method for identifying dam piping by using visible light and infrared imaging with an unmanned aerial vehicle as claimed in claim 1, characterized in that: Based on the extracted features, the process of establishing a judgment matrix to calculate the factor weight vector of each feature includes: using the hierarchical analysis method to assign different values ​​according to the weight of the piping influencing factors, and the evaluation index weight function is as follows: ; ; Among them: a ij is the weight of each influencing factor, n is the number of influencing factors, is the factor vector weight.

6. The method for identifying dam piping by using visible light and infrared imaging with an unmanned aerial vehicle as claimed in claim 1, characterized in that: The pre-training process of the deep neural network learning model includes: Collect historical piping image data, the data including information about pipe orifice, water head, distance from bank, flow, turbidity and sand ring visibility, to form a data set; Annotate the data set, including marking each pixel in the image as a corresponding piping feature category; The Seg-Net image semantic segmentation network model is trained using a dataset with labeled information. During the training process, the model will learn to extract features from the input image and correctly classify each pixel.

7. The method for identifying dam piping by using visible light and infrared imaging with an unmanned aerial vehicle as claimed in claim 1, characterized in that: The process of extracting features from the image information on the back of the target dam using the pre-trained deep neural network learning model includes: extracting the features of the orifice size by calculating the area or perimeter of the orifice region in the segmentation result; The water head height is obtained by converting the pixel height into the actual height using the image coordinate system; The distance from the dike is calculated by dividing the pipe opening and the dike area in the result of segmentation. The flow rate is directly predicted by the training model. The turbidity is estimated by analyzing the color and texture characteristics of the water area in the segmentation result; The degree of sand ring visibility is determined by calculating the area and shape characteristics of the sand ring region in the segmentation result.

8. The method for identifying dam piping by using visible light and infrared imaging with an unmanned aerial vehicle as claimed in claim 1, characterized in that: The multi-scale and multi-level spatial analysis includes macro-scale level, meso-scale level and local-scale level, wherein the macro-scale level is to analyze the hydrogeological conditions in the entire basin or region and the dangerous conditions in the entire basin, including flood risks and upstream water volume; The mesoscale level described is to analyze the structure and status of the entire dam; The local scale level is to extract characteristic information of dam piping based on drone photography to comprehensively analyze the development of piping hazards.

9. A dam piping identification system using visible light and infrared imaging carried by an unmanned aerial vehicle, characterized in that: include: The drone is equipped with an image acquisition device for acquiring image information of the back side of the target dam taken by the drone, wherein the image information includes a visible light image when the light intensity / visibility is greater than a set value, and an infrared image when the light intensity / visibility is less than or equal to the set value; The feature extraction module is used to extract features from the image information on the back of the target dam using a pre-trained deep neural network learning model, and extract the size of the pipe mouth, water head height, distance from the dam, flow rate, turbidity, and sand ring visibility of the piping; The dam piping identification module is used to establish a judgment matrix based on the extracted features to calculate the factor weight vector of each feature, and comprehensively classify the dam piping risk level according to the factor weight vector information and features.

10. The dam piping identification system using visible light and infrared imaging carried by an unmanned aerial vehicle as claimed in claim 9, characterized in that: The drone is equipped with an image acquisition device to acquire images of the back of the target dam according to a preset inspection path; The image acquisition device includes visible light and infrared light image acquisition devices.

Citation Information

Patent Citations

  • Unmanned aerial vehicle embankment piping patrol system and method based on deep learning

    CN114545965A

  • Urban dam piping danger advanced early warning method and system

    CN117787686A

Cited By

  • Unmanned aerial vehicle automatic inspection system for dam surface of extra-high arch dam

    CN119902561A

  • Piping disaster emergency management system, method and device and storage medium

    CN120416281A

  • Tunnel face water burst hidden danger identification method based on image fusion

    CN121545048A

  • Piping dangerous case troubleshooting method and system based on image recognition

    CN121998431A

  • A pipe gushing danger investigation method and system based on image recognition

    CN121998431B