Unmanned aerial vehicle-mounted automatic patrol and early warning method and device for leakage risk of earth and rockfill dam

Abstract

The application discloses a dam seepage danger unmanned aerial vehicle-mounted automatic patrol and early warning method and device, which comprises the following steps: continuously and with a certain overlap degree, an infrared image of a dam site is shot by using an unmanned flight platform; the infrared image is input into a deep convolutional neural network model obtained through training, and the deep convolutional neural network model predicts and outputs a dam seepage type. The dam seepage danger unmanned aerial vehicle-mounted automatic patrol and early warning method and device provided by the application adopt a distributed box-type ground station to obtain the dam seepage type, have strong flexibility, and solve the problem that a single unmanned flight platform is difficult to complete super-long dam patrol due to limited endurance time.

Description

TECHNICAL FIELD

[0001] The present application relates to a dam seepage danger unmanned aerial vehicle automatic patrol and early warning method and device, and belongs to the technical field of dam seepage danger early warning. BACKGROUND

[0002] River embankment and reservoir dam engineering, i.e. dam engineering, as an important part of the flood control system, is an important guarantee for people's life and social production. As of the end of 2018, 98,822 reservoirs of all types have been built in China, with a total length of 5-grade and above embankments exceeding 312,000 kilometers, of which the soil and stone embankments account for a large proportion. It is of great significance to ensure the safe operation of soil and stone embankments to maintain people's life and property safety and social stability.

[0003] The occurrence of soil and stone embankment seepage has the characteristics of spatial and temporal randomness and concealment, and timely discovery, accurate identification and positioning of seepage are the key to ensuring the safety of soil and stone embankments. Although some large soil and stone embankments are equipped with pressure measuring pipes, seepage pressure gauges and water measuring weirs, etc. monitoring facilities, they are usually only installed at a few key sections, which may miss the danger. In addition, most river embankments are not equipped with any seepage monitoring facilities. In addition, there are geophysical methods such as high-density electrical method, transient electromagnetic method and geological radar method, which are mainly used for detecting internal seepage hazards of embankments during non-flood season. The operation efficiency, coverage range and reliability of the existing technology cannot meet the needs of seepage danger detection and identification during flood season, so the discovery of soil and stone embankment seepage danger still mainly relies on manual netting patrol, which greatly restricts the risk control ability and emergency disposal level of China's water conservancy engineering. SUMMARY

[0004] The technical problem to be solved by the present application is to overcome the defects of the prior art, provide a soil and stone embankment seepage danger unmanned aerial vehicle automatic patrol and early warning method and device, realize full-coverage rapid detection, efficient automatic identification and early warning information release of soil and stone embankment seepage danger, and provide technical and equipment support for flood control and disaster reduction.

[0005] To achieve the above-mentioned purpose, the present application provides a soil and stone embankment seepage danger unmanned aerial vehicle automatic patrol and early warning method, comprising: using an unmanned flight platform to continuously and with a certain degree of overlap to shoot infrared images of the soil and stone embankment site.

[0006] The infrared image is input into a deep convolutional neural network model trained, and the deep convolutional neural network model predicts the output of the dam seepage type.

[0007] Preferably, the dam leakage type includes normal, low-temperature piping, high-temperature piping, low-temperature slope surface leakage or high-temperature slope surface leakage, and if the dam leakage type is low-temperature piping, high-temperature piping, low-temperature slope surface leakage or high-temperature slope surface leakage, the warning information is sent to the mobile terminal.

[0008] Preferably, the warning information includes the dam leakage type, the latitude and longitude of the unmanned aerial vehicle, the height of the unmanned aerial vehicle, the time when the infrared image is collected by the unmanned aerial vehicle, and the infrared image.

[0009] Preferably, the deep convolutional neural network model is trained, comprising:

[0010] building a deep convolutional neural network model;

[0011] training the deep convolutional neural network model using a dam leakage infrared image database, the dam leakage infrared image database including infrared images of a soil and rock dam site slope surface in a normal state and a water surface in a normal state, infrared images in a low-temperature piping state, infrared images in a high-temperature piping state, infrared images in a low-temperature slope surface leakage state, and infrared images in a high-temperature slope surface leakage state collected at different distances and different angles;

[0012] based on the solver Adam, the deep convolutional neural network model builds a mapping relationship between the infrared image as input and the dam leakage type as output, and iteratively trains the parameters of the deep convolutional neural network model;

[0013] after meeting the iteration stop condition, the final deep convolutional neural network model is obtained.

[0014] Preferably, before training, the initial learning rate, the small batch, the maximum number of iterations and the validation frequency of the deep convolutional neural network model are set.

[0015] after a set number of iterations, the learning rate is multiplied by a set reduction factor to obtain a new learning rate.

[0016] Preferably, the architecture of the deep convolutional neural network model includes a feature extractor AlexNet, an fc-replaced layer, a sof-replaced layer and an out-replaced layer, which are connected in sequence.

[0017] Preferably, before shooting infrared images and visible light images of the soil and rock dam site, the range of the soil and rock dam site is divided, and a plurality of distributed box ground stations are arranged;

[0018] each distributed box ground station is configured with a plurality of unmanned aerial vehicles;

[0019] determine the detection operation range, flight path and flight operation schedule of the unmanned flight platform.

[0020] The unmanned aerial automatic patrol and early warning device for dam leakage danger is used for executing the method, and comprises a plurality of distributed box ground stations, a central control station and a plurality of mobile terminals.

[0021] The distributed box ground station is arranged at the dam site, and comprises a storage battery, a data repeater, a wireless transceiver, an infrared thermal imager, a high-definition visible light camera, a processor and at least one unmanned flight platform.

[0022] The central control station comprises a wireless receiver, a wireless transmitter and a control processor, and is used for judging the dam leakage type of the dam site, storing the collected infrared image and visible light image and sending early warning information to the mobile terminal.

[0023] The mobile terminal is used for receiving and viewing the early warning information.

[0024] Preferably, the unmanned flight platform comprises a foot stand, an anode charging contact, a cathode charging contact, a cathode conductive plate and an anode conductive plate.

[0025] A computer readable storage medium, which stores a computer program, the computer program is executed by a processor to implement the steps of the method.

[0026] The present application has the following beneficial effects:

[0027] The present application provides an unmanned aerial automatic patrol and early warning method and device for dam leakage danger, which uses a distributed box ground station to obtain the dam leakage type, has strong flexibility, and solves the problem that a single unmanned flight platform cannot complete long dam patrol due to limited endurance time.

[0028] The unmanned aerial automatic patrol and early warning system for the earth and rock dam leakage danger provided by the application can realize full coverage patrol of the dam surface temperature field by adopting an unmanned aerial platform carrying an infrared thermal imager, and can work normally in the night, rainy days and other bad conditions, and has time and space continuity, and can realize automatic patrol and effectively avoid missing detection while reducing the huge human resource consumption of the current manual netting type patrol.

[0029] The unmanned aerial automatic patrol and early warning method for the earth and rock dam leakage danger provided by the application can realize automatic and efficient detection of the leakage danger, automatic identification and leakage early warning information release by comprehensively utilizing an unmanned aerial platform, thermal imaging technology and a deep convolutional neural network, and has much higher detection and identification efficiency and stability than manual detection.

[0030] The application solves the problems of low efficiency, easy missing of leakage danger and personal safety threat of the current manual patrol of the earth and rock dam, effectively makes up for the deficiency of the current 'point' type leakage monitoring method, is suitable for long-term monitoring of the earth and rock dam leakage danger, especially for the earth and rock dams and earth and rock embankment projects which lack leakage monitoring systems at present, can fill the safety monitoring blank, reduce the burden and risk of manual netting type investigation of the dam leakage danger in the flood season, provides support for long-term service and flood prevention and disaster reduction of the earth and rock dam, and has important social and economic value and application and popularization significance. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a system layout schematic diagram of the application;

[0032] Figure 2 is a schematic diagram of the dam leakage infrared image database in the first embodiment of the application;

[0033] Figure 3 is a schematic diagram of the dam leakage type predicted and output by the deep convolutional neural network model in the first embodiment of the application;

[0034] Figure 4 is a structure diagram of the deep convolutional neural network model in the embodiment of the application;

[0035] Figure 5 is a structure diagram of the unmanned aerial platform in the second embodiment of the application;

[0036] Figure 6 is a schematic diagram of the take-off and recovery of the unmanned aerial platform of the application;

[0037] Figure 7 is a framework diagram of the unmanned aerial automatic patrol and early warning device for the earth and rock dam leakage danger of the application;

[0038] Figure 8is a schematic diagram of the early warning information received by the mobile terminal in embodiment two of the present application;

[0039] Figure 9 is a schematic diagram of the early warning information received by the mobile terminal in embodiment two of the present application.

[0040] The meaning of the reference signs is as follows, 0 - earth and rock dam site; 1 - distributed box type ground station; 2 - unmanned flight platform; 3 - data relay; 4 - wireless transceiver; 5 - central control station; 6 - mobile terminal; 7 - infrared thermal imager; 8 - high-definition visible light camera; 9 - lighting device; 10 - anode charging contact; 11 - cathode charging contact; 12 - cathode conductive plate; 13 - anode conductive plate. DETAILED DESCRIPTION

[0041] The following examples are only used to more clearly illustrate the technical solutions of the present application, and cannot be used to limit the protection scope of the present application.

[0042] Embodiment one

[0043] As shown in Figure 1 The earth and rock dam leakage danger unmanned aerial automatic patrol and early warning method comprises:

[0044] Step 1, divide the range of the earth and rock dam site which is too long or positionally dispersed, and arrange a plurality of distributed box type ground stations at selected positions of the earth and rock dam site;

[0045] Step 2, clearly define the detection operation range, flight path and flight operation timetable of the unmanned flight platform controlled by each distributed box type ground station;

[0046] Step 3, use the unmanned flight platform to carry the infrared thermal imager and the high-definition visible light camera to continuously and with a certain degree of overlap, shoot and acquire the infrared images and visible light images of the earth and rock dam site within the monitoring range of the distributed box type ground station;

[0047] Step 4, transmit the infrared images and visible light images collected by the unmanned flight platform to the distributed box type ground station to which the unmanned flight platform belongs;

[0048] Step 5, the distributed box type ground station transmits the infrared images and visible light images to the deep convolutional neural network model;

[0049] Step 6, the deep convolutional neural network model identifies the dam leakage type: if the dam leakage type is slope leakage or piping, sends early warning information to the mobile terminal,

[0050] The early warning information comprises the dam leakage type, the latitude and longitude of the unmanned flight platform, the height of the unmanned flight platform, the time of collecting the infrared images by the unmanned flight platform, the infrared images and the corresponding visible light images.

[0051] Step 7, the engineering technician reviews whether the slope seepage occurs according to the seepage type, the infrared image and the visible light image received in the mobile terminal, and if it is confirmed that the slope seepage occurs, the seepage position is determined according to the latitude and longitude of the unmanned aerial platform, and emergency rescue is organized.

[0052] As shown in Figure 2 and Figure 3 , the dam seepage type includes normal, low-temperature piping, high-temperature piping, low-temperature slope seepage or high-temperature slope seepage, and high temperature and low temperature are relative, and high temperature means that the temperature of the seepage water is higher than the background (i.e. normal slope). The deep convolutional neural network model focuses on the contour and texture features of the seepage outlet. The infrared radiation temperature result in each window is mapped to a certain color space (such as gray space, HSV color space or RGB color space) according to its maximum value and minimum value, and the color scale value in the image represents the corresponding temperature value. According to the research results, each day is divided into two time periods: the river water temperature is higher than the slope / dam foot accumulated water; the river water temperature is lower than the slope / dam foot accumulated water. When the river water temperature is higher than the dam slope, a high-temperature abnormal area with a trailing contour feature will appear in the infrared image, which is called high-temperature slope seepage. Since the infrared image is mapped to the color scale value according to the maximum value and the minimum value, the high temperature in the infrared image is represented by a larger (greater than 128) color scale value, and the low temperature in the infrared image is represented by a smaller (less than 128) color scale value.

[0053] If the deep convolutional neural network model judges that the dam seepage type is low-temperature piping, high-temperature piping, low-temperature slope seepage or high-temperature slope seepage, and the corresponding classification score is greater than or equal to 0.80, the warning information is sent to the mobile terminal. The mathematical expression for calculating the classification score is:

[0054]

[0055] wherein x i is the output of the fc-replaced layer, i = 1, 2, 3, 4, 5.

[0056] As shown in Figure 3As shown, the first infrared image input deep convolutional neural network model is determined to be high-temperature piping, with a score of 0.93; the second infrared image input deep convolutional neural network model is determined to be low-temperature piping, with a score of 0.96; the third infrared image input deep convolutional neural network model is determined to be normal state, with a score of 1.00; the fourth infrared image input deep convolutional neural network model is determined to be high-temperature slope seepage, with a score of 1.00; the fifth infrared image input deep convolutional neural network model is determined to be low-temperature slope seepage, with a score of 0.99; and the sixth infrared image input deep convolutional neural network model is determined to be normal state, with a score of 0.94. The deep convolutional neural network model obtained by training includes:

[0057] constructing a deep convolutional neural network model;

[0058] training the deep convolutional neural network model using a dam seepage infrared image database, the dam seepage infrared image database including infrared images of a soil and rock dam site slope and a water surface in a normal state, infrared images in a low-temperature piping state, infrared images in a high-temperature piping state, infrared images in a low-temperature slope seepage state, and infrared images in a high-temperature slope seepage state collected at different distances and different angles;

[0059] based on the feature extractor AlexNet and the solver Adam, the deep convolutional neural network model constructs a mapping relationship between the infrared image as input and the dam seepage type as output, and iteratively trains the parameters of the deep convolutional neural network model;

[0060] after meeting the iteration stopping condition, the final deep convolutional neural network model is obtained.

[0061] The unmanned aerial vehicle-mounted automatic patrol and early warning device for soil and rock dam seepage danger is used to execute any one of the above methods, and includes a plurality of distributed box-type ground stations, a central control station, and a plurality of mobile terminals. The distributed box-type ground stations are in communication connection with the central control station, and the central control station and the mobile terminals are in communication connection;

[0062] The distributed box-type ground stations are arranged at the soil and rock dam site. The distributed box-type ground stations include a storage battery, a data repeater, a wireless transceiver, an infrared thermal imager, a high-definition visible light camera, a processor, and at least one unmanned flight platform. The processor is electrically connected to the storage battery, the data repeater, and the wireless transceiver. The infrared thermal imager and the high-definition visible light camera are installed on the unmanned flight platform, and are used to acquire infrared images and visible light images of the soil and rock dam site, and send them to the central control station.

[0063] The central control station comprises a wireless receiver, a wireless transmitter and a control processor for determining the type of dam leakage at the earth-rock dam site, storing the collected infrared images and visible light images and sending early warning information to the mobile terminal.

[0064] The mobile terminal is configured to receive and view the early warning information.

[0065] An electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of any one of the preceding claims when executing the program.

[0066] A computer readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the method of any one of the preceding claims when executed by a processor.

[0067] Before training, the initial learning rate, the mini-batch, the maximum number of iterations and the validation frequency of the deep convolutional neural network model are set.

[0068] After every set number of iterations, the learning rate is multiplied by a set reduction factor to obtain a new learning rate.

[0069] The dam leakage infrared image database comprises infrared images (1400) of the earth-rock dam slope in a normal state and the water surface in a normal state, infrared images (1200) of the low-temperature piping state, infrared images (1200) of the high-temperature piping state, infrared images (1200) of the low-temperature slope leakage state and infrared images (1200) of the high-temperature slope leakage state collected at different distances and different angles. The slope leakage state has a clear tailing profile feature in the infrared image; the piping state has the feature of spreading around the piping outlet in the infrared image; the temperature distribution of the slope in the normal state and the water surface in the normal state is uniform. The high temperature is represented by a large (greater than 128) color scale value in the infrared image, and the low temperature is represented by a small (less than 128) color scale value in the infrared image.

[0070] Before training, the initial learning rate, the mini-batch, the maximum number of iterations and the validation frequency of the deep convolutional neural network model are set.

[0071] After every set number of iterations, the learning rate is multiplied by a set reduction factor to obtain a new learning rate.

[0072] Specifically, the initial learning rate of the deep convolutional neural network model is set to 0.001, the mini-batch is set to 64, the maximum number of iterations is set to 60, and the validation frequency is set to 50; after every 10 iterations, the learning rate is multiplied by a reduction factor of 0.1 to reduce the learning rate. In addition, to improve the training efficiency, the learning rate of the fully connected layer is set to 5 times that of other layers.

[0073] As shown in Figure 4 , the deep convolutional neural network model is a deep convolutional neural network based on transfer learning and trained, and the feature extractor is AlexNet. Alternatively, the feature extractor can be one or more of SqueezeNet, ResNet, GoogLeNet or VGG-19.

[0074] The models of the wireless receiver, the wireless transmitting device, the control processor, the central control station, the plurality of mobile terminals, the battery, the data relay, the wireless transceiver, the infrared thermal imager, the high-definition visible light camera, the processor and the unmanned aerial vehicle are many in the prior art, and the person skilled in the art can select appropriate models according to actual needs. The present embodiment will not be exemplified one by one.

[0075] The framework of the deep convolutional neural network model includes the feature extractor AlexNet, the fc-replaced layer, the sof-replaced layer and the out-replaced layer, and the feature extractor AlexNet, the fc-replaced layer, the sof-replaced layer and the out-replaced layer are connected in sequence. For details, see Table 1.

[0076] Table 1

[0077]

[0078] The learning rate of the fully connected layer in Table 1 is set to 5 times that of other layers.

[0079] As shown in Figure 5 , the unmanned aerial vehicle carrying equipment has an infrared thermal imager, a high-definition visible light camera and a lighting device, which are used to collect infrared images and visible light images of the earth and rock dam and transmit the collected images to the box-shaped ground station to which they belong. The circuit of the lighting device is in a normally open state by default and is closed only when the unmanned aerial vehicle is in flight operation and the intensity of natural light is weak.

[0080] The model of the unmanned aerial vehicle is a multi-rotor unmanned aerial vehicle or a vertical take-off and landing fixed-wing unmanned aerial vehicle with vertical take-off and landing function. As Figure 6As shown, the distributed box-type ground station can serve as a storage for the unmanned flight platform. The distributed box-type ground station is laid out in the dam site in a complete initial state, and the unmanned flight platform is stored in the distributed box-type ground station. When the unmanned flight platform is started for a flight task, the box of the distributed box-type ground station is opened from the top to release the unmanned flight platform. After the unmanned flight platform completes the task, it automatically returns to the take-off point and enters the opened box of the distributed box-type ground station from the top. After the unmanned flight platform is recovered, the top of the box of the distributed box-type ground station is closed to protect the unmanned flight platform and other devices in the ground station.

[0081] The unmanned flight platform is provided with two foot supports, and the charging contacts of the unmanned flight platform are independently arranged at the bottom of the foot supports of the unmanned flight platform, that is, the anode charging contact 10 is arranged at the bottom of one foot support, and the cathode charging contact 11 is arranged at the bottom of the other foot support. The distributed box-type ground station includes n (n is an integer greater than or equal to 1) unmanned flight platforms, and the energy supply module is arranged at the bottom of the internal box of the box-type ground station and includes a charging cable, a photovoltaic charging device and a battery, which can directly contact the charging contacts of the unmanned flight platform. The power supply electrodes of the energy supply module are alternately arranged in the form of sheet-shaped anode conductive plates 13 and cathode conductive plates 12 and cover the bottom of the internal box of the box-type ground station. The charging connection mode is a "face-to-face" connection mode, and as long as the foot supports of the unmanned flight platform are located in the correct charging plate area when landing, the unmanned flight platform can be successfully charged. This eliminates the harsh requirements for the positioning of the unmanned flight platform in the current "point-to-point" plug-in charging connection mode.

[0082] As shown, Figure 7 The unmanned flight platform realizes flight control through the unmanned aerial vehicle flight control system, realizes load control through the unmanned aerial vehicle load control system, transmits the collected infrared images and visible light images to the distributed box-type ground station through the image transmission system, and transmits the infrared images and visible light images to the central control station through the data transmission module of the distributed box-type ground station. The collected infrared images and visible light images are stored in the data temporary storage module.

[0083] The mobile terminal includes a man-machine interaction module and a first communication module, the man-machine interaction module is used for viewing early warning information or retrieving infrared images and visible light images, and the first communication module is used for communicating with the distributed box-type ground station and the central control station.

[0084] The central control station comprises a data storage module, a man-machine interaction module, a deep convolutional neural network model identification module, an infrared image preprocessing module and a second communication module. The data storage module is used for storing infrared images, visible light images, the height of the unmanned aerial vehicle when collecting the infrared images and the time of the unmanned aerial vehicle when collecting the infrared images. The deep convolutional neural network model identification module is used for carrying and running a deep convolutional neural network model. The infrared image preprocessing module is used for preprocessing the infrared images to obtain images with appropriate sizes. The second communication module is used for realizing data interaction with a mobile terminal or a distributed box-type ground station.

[0085] The infrared thermal imager and the high-definition visible light camera carried by the unmanned flight platform automatically and continuously collect infrared images and visible light images of the predetermined range of the earth dam back slope and dam toe with a certain degree of overlap, so as to ensure that the collected infrared images and visible light images fully cover the predetermined range of the earth dam back slope and dam toe.

[0086] Further, the infrared images and the visible light images are collected separately and simultaneously, so as to ensure that the infrared images and the visible light images are collected and saved one by one in the form of image pairs. Each image is attached with information such as the longitude and latitude of the unmanned aerial vehicle, the height of the unmanned aerial vehicle when collecting the infrared images and the time of the unmanned aerial vehicle when collecting the infrared images.

[0087] Step 5: The dam infrared images and visible light images collected by the unmanned flight platform are transmitted to the data relay device 3 in the distributed box-type ground station to which the unmanned flight platform belongs through the real-time image transmission system of the unmanned flight platform.

[0088] Step 6: The infrared images and visible light images temporarily stored in the data relay device in the distributed box-type ground station are transmitted to the remote central control station 5 through the wireless transceiver device 4.

[0089] The warning information comprises the seepage type of the earth dam, the longitude and latitude of the unmanned flight platform, the flight height of the unmanned flight platform, the time of the unmanned flight platform when collecting the infrared images, the infrared images, the corresponding visible light images and the number of the infrared images. The unmanned flight platform is a multi-rotor unmanned aerial vehicle or a vertical take-off and landing fixed-wing unmanned aerial vehicle with a vertical take-off and landing function, so as to realize the vertical take-off and landing of the unmanned flight platform from the top of the box of the distributed box-type ground station.

[0090] The central control station can remotely control the flight and load of each unmanned flight platform.

[0091] Before inputting the deep convolutional neural network model, the size of the infrared image is scaled, and the infrared images collected by infrared thermal imagers of different models are uniformly scaled to the size required by the input layer of the deep convolutional neural network model. For example, the feature extractor of the deep convolutional neural network model selects AlexNet, and the received infrared image is uniformly scaled to 227x227x3.

[0092] The central control station stores the received infrared image in the form of a pair of image numbers, that is, two infrared images and a visible light image form an image pair, and share a number.

[0093] Embodiment two

[0094] The temperature field in the soil body without seepage flow is controlled by pure heat conduction. When there is seepage flow, the heat conduction intensity in the soil body will change due to the migration of water in the soil body. When the permeability coefficient of the dam soil body is greater than 10 -6 m / s, the soil conduction heat transfer will be obviously overtaken by the advection heat transfer caused by fluid motion, and even a little liquid flow will cause the soil temperature to adapt to the temperature of the seepage water, thereby causing local irregular changes in the original dam soil temperature field. Therefore, the leakage of the earth-rock dam can be indirectly monitored by temperature measurement.

[0095] Infrared thermal imaging breaks through the single-dimensional limitation of traditional point-type temperature measurement technology, can present a two-dimensional continuous temperature field of the sensed object, has the characteristics of intuitiveness, continuity and wide coverage, and is especially suitable for sensing large temperature fields. Moreover, the thermal imager can normally work at night without light, and therefore the measurement result has spatiotemporal continuity. The combination of infrared thermal imaging and low-altitude unmanned flight platform can realize efficient, flexible and comprehensive coverage of the earth-rock dam leakage danger patrol.

[0096] In view of the severe situation of earth-rock dam flood prevention and the urgent need for full-coverage leakage monitoring means, the present application comprehensively utilizes passive infrared imaging technology, unmanned flight platform, deep learning theory and method to develop an unmanned aerial vehicle-mounted automatic patrol and early warning method and system for earth-rock dam leakage danger, and provides technical support for guaranteeing the long-term service of earth-rock dams and flood prevention and disaster reduction.

[0097] The main parameters of the unmanned flight platform, the airborne infrared thermal imager and the airborne visible light camera used in the embodiments of the present application are shown in Table 2, Table 3 and Table 4, respectively.

[0098] Table 2

[0099]

[0100] Table 3

[0101]

[0102] Table 4

[0103]

[0104]

[0105] The unmanned aerial platform carries an infrared thermal imager and a visible light camera to patrol a certain section of the embankment backwater slope, the infrared thermal imager and the visible light camera are set at a fixed angle of 75°, the flight speed of the unmanned aerial platform is 1m / s, and the image sampling time interval is 2s. The leakage judgment module of the present application automatically identifies the leakage of the embankment based on the infrared image of the embankment. The mobile terminal finally receives the early warning information as shown in Figure 8 or Fig. 9, such as Figure 8 including infrared images and visible light images, the leakage type is low-temperature piping, the longitude of the unmanned aerial platform is 32°8'14.4467000000000412", the latitude of the unmanned aerial platform is 118°26'7.6474000000623343", the flight height of the unmanned aerial platform is 19m, the time is 2021-08-13 14:21, and the image pair number is 2021-08-13-6109;

[0106] As shown in Figure 9 , including infrared images and visible light images, the leakage type is low-temperature slope leakage, the longitude of the unmanned aerial platform is 32°9'49.5286999999952826", the latitude of the unmanned aerial platform is 118°25'56.14760000002576743", the flight height of the unmanned aerial platform is 13m, the time is 2021-07-18 13:07, and the image pair number is 2021-07-18-3691.

[0107] After the professional user / staff receives the leakage early warning information, more adjacent infrared images and visible light images received can be quickly retrieved through the image pair number, so as to better review the leakage judgment result.

[0108] The present application is described with reference to flowcharts and / or block diagrams according to the method, device (system) and computer program product of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be realized by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device realize the functions specified in the flowchart Figure 1one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart or multiple flows and / or blocks.

[0109] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart or multiple flows and / or blocks. Figure 1 one or more processes and / or blocks Figure 1 the functions specified in the flowchart or multiple flows and / or blocks.

[0110] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart or multiple flows and / or blocks. Figure 1 one or more processes and / or blocks Figure 1 the steps for performing the functions specified in the flowchart or multiple flows and / or blocks.

[0111] The above description is merely the preferred embodiment of the present application, it should be pointed out that for those skilled in the art, without departing from the technical principles of the present application, a number of improvements and modifications can be made, these improvements and modifications should also be considered as the protection scope of the present application.

Claims

1. An unmanned aerial vehicle-mounted automatic patrol and early warning method for earth and rockfill dam leakage danger, characterized in that, The method comprises the following steps: dividing the range of the super-long or position-dispersed earth-rock dam site, arranging a plurality of distributed box ground stations at selected positions of the earth-rock dam site, and configuring a plurality of unmanned aerial vehicles for each distributed box ground station; defining the detection operation range, flight path and flight operation time of the unmanned aerial vehicle controlled by each distributed box ground station; using the unmanned aerial vehicle to continuously and with a certain degree of overlap shoot infrared images of the earth-rock dam site; inputting the infrared images into a final deep convolutional neural network model, and the deep convolutional neural network model predicting and outputting the dam leakage type; training the deep convolutional neural network model, comprising: constructing the deep convolutional neural network model; training the deep convolutional neural network model using a dam leakage infrared image database, the dam leakage infrared image database comprising infrared images of the slope surface of the earth-rock dam site in a normal state and the water surface in a normal state, infrared images in a low-temperature piping state, infrared images in a high-temperature piping state, infrared images in a low-temperature slope leakage state and infrared images in a high-temperature slope leakage state collected at different distances and different angles; based on the feature extractor AlexNet and the solver Adam, the deep convolutional neural network model constructs a mapping relationship between the infrared image as input and the dam leakage type as output, and iteratively trains the parameters of the deep convolutional neural network model; after meeting the iteration stop condition, the final deep convolutional neural network model is obtained; wherein the architecture of the deep convolutional neural network model comprises a feature extractor AlexNet, an fc-replaced layer, a sof-replaced layer and an out-replaced layer, and the feature extractor AlexNet, the fc-replaced layer, the sof-replaced layer and the out-replaced layer are connected in sequence; wherein the dam leakage type includes normal, low-temperature piping, high-temperature piping, low-temperature slope leakage or high-temperature slope leakage, and if the dam leakage type is low-temperature piping, high-temperature piping, low-temperature slope leakage or high-temperature slope leakage, the mobile terminal is sent a warning information.

2. The unmanned aerial vehicle-mounted automatic patrol and early warning method for dam leakage danger according to claim 1, wherein the warning information comprises the dam leakage type, the latitude and longitude of the unmanned aerial vehicle, the height of the unmanned aerial vehicle, the time when the infrared image is collected by the unmanned aerial vehicle, the infrared image and the corresponding visible light image.

3. The unmanned aerial vehicle-mounted automatic patrol and early warning method for dam leakage danger according to claim 1, wherein before training, the initial learning rate, the small batch, the maximum number of iterations and the verification frequency of the deep convolutional neural network model are set; after every set number of iterations, the learning rate is multiplied by a set reduction factor to obtain a new learning rate. The method according to any one of claims 1-3 is executed, comprising a plurality of distributed box ground stations, a central control station and a plurality of mobile terminals, the distributed box ground stations being in communication connection with the central control station, and the central control station and the mobile terminals being in communication connection; ​ 4. The unmanned aerial vehicle-mounted automatic patrol and early warning device for leakage risk of earth and rockfill dam, characterized in that, ​ The distributed box-type ground station is deployed at the earth-rock dam site. The distributed box-type ground station includes a battery, a data repeater, a wireless transceiver, an infrared thermal imager, a high-definition visible light camera, a processor, and at least one unmanned aerial vehicle (UAV) platform. The processor is electrically connected to the battery, the data repeater, and the wireless transceiver. The infrared thermal imager and the high-definition visible light camera are installed on the UAV platform to acquire infrared and visible light images of the earth-rock dam site and send them to the central control station. The central control station includes a wireless receiver, a wireless transmitter, and a control processor, used to determine the type of seepage at the earth-rock dam site, store the collected infrared and visible light images, and send early warning information to mobile terminals. The mobile terminal is used to receive and view warning information.

5. The earth-rock dam leakage risk unmanned aerial automatic patrol and early warning device according to claim 4, characterized in that, It includes a tripod, an anode charging contact, a cathode charging contact, a cathode conductive plate, and an anode conductive plate. The tripod is installed on the unmanned flight platform. The anode charging contact and the cathode charging contact are independently installed at the bottom of the tripod. The sheet-shaped anode conductive plate and the cathode conductive plate are alternately arranged in the distributed box-type ground station.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 3.

Images