Unmanned aerial vehicle-based water conservancy facility patrol and early warning method and system

Abstract

The application discloses a water conservancy facility patrol early warning method and system based on a UAV, belongs to the technical field of water conservancy facility detection, and determines the shooting detection points of the UAV after dividing the sub-regions of the target dam; the shooting detection points of each sub-region ensure that all the detection images corresponding to the corresponding sub-region after the UAV completes shooting in the corresponding sub-region are covered in the case that the number is the least and the detection images are not lower than the preset definition, the UAV patrol efficiency is improved, the instability value of the target dam is comprehensively evaluated according to the abnormal values of each sub-region, the evaluation accuracy and comprehensiveness of the instability value of the dam are improved, early warning is performed according to the abnormal values of different sub-regions and the instability value of the target dam, early warning is comprehensively and timely performed, the abnormal sub-region is quickly positioned, and the dam safety maintenance efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of water conservancy facility detection technology, and particularly relates to a water conservancy facility patrol early warning method and system based on a UAV. BACKGROUND

[0002] A dam is an important water conservancy facility for controlling water flow, flood control and water storage, so it is very important to patrol and detect the dam. In the prior art, the patrol early warning method or system mainly detects a certain area of the dam, and the stability evaluated is limited to a single area, so that the evaluated stability is not accurate and comprehensive, and thus it is impossible to discover abnormalities in other areas of the dam in time. SUMMARY

[0003] The present application provides a water conservancy facility patrol early warning system based on a UAV to solve the technical problems in the prior art, which comprises a control center and a UAV carrying a camera device, the control center is in communication connection with the UAV, and the control center comprises a first image acquisition module, a region division module, a detection point determination module, a second image acquisition module, an abnormality analysis module, an instability evaluation module and an alarm module.

[0004] The first image acquisition module is used to acquire a dam panoramic image of a target dam shot by the UAV.

[0005] The region division module is used to divide sub-regions of the target dam according to the dam panoramic image.

[0006] The detection point determination module is used to determine a shooting detection point of the UAV in each sub-region of the target dam, and the shooting detection point of each sub-region ensures that all detection images corresponding to the sub-region are covered after the UAV completes shooting in the corresponding sub-region, with the number of detection images being the least and the detection images not being lower than a preset definition.

[0007] The second image acquisition module is used to acquire corresponding detection images shot by the UAV in each shooting detection point.

[0008] The abnormality analysis module is used to identify and analyze abnormal values of each sub-region according to each detection image.

[0009] The instability evaluation module is used to evaluate an instability value BW of the target dam according to the abnormal values of each sub-region.

[0010] The alarm module is used to send alarm information to a user according to the abnormal values of each sub-region and the instability value of the target dam.

[0011] Further, the target dam sub-regions include a dam foundation region, a dam body region and a slope region; and the detection images include a dam foundation region image, a dam body region image and a slope region image.

[0012] Further, the anomaly analysis module includes a dam foundation anomaly analysis module, a dam body anomaly analysis module and a slope anomaly analysis module.

[0013] The dam body anomaly analysis module is configured to analyze complete crack information of the dam body region according to the dam body region image, and calculate a dam body anomaly value BT according to the complete crack information.

[0014] The dam foundation anomaly analysis module is configured to analyze settlement depression information of the dam foundation region according to the dam foundation region image, and calculate a dam foundation anomaly value BJ according to the settlement depression information.

[0015] The slope anomaly analysis module is configured to analyze surface spalling information of the slope region according to the slope region image, and calculate a slope anomaly value BP according to the surface spalling information.

[0016] Further, the analysis of the complete crack information of the dam body region according to the dam body region image and the calculation of the dam body anomaly value BT according to the complete crack information are specifically as follows:

[0017] The suspected crack contours in the dam body region image are extracted by using an edge detection algorithm;

[0018] The real cracks are identified from the suspected crack contours by using a pre-trained first deep learning model;

[0019] The dam body region images are spliced to obtain a complete dam body region image according to the shooting detection points of the dam body region;

[0020] The real cracks that can be connected with the end points of the current real cracks are found in the complete dam body region image according to the end point positions of the real cracks, and are spliced continuously to obtain corresponding complete cracks;

[0021] It is judged whether the corresponding complete cracks pass through important regions of the dam body according to the end point positions of the complete cracks, and the number N1 of the complete cracks passing through the important regions of the dam body is recorded;

[0022] The total pixel grid number G of the complete dam body region image and the pixel grid number g occupied by each complete crack in the complete dam body region image are obtained, and if g / G is greater than or equal to a first preset threshold, the corresponding complete crack is judged as a serious crack, and the number N2 of the serious cracks is counted;

[0023] The dam body anomaly value BT is calculated, BT=k1*N1+k2*N2, wherein k1 represents a first preset coefficient, and k2 represents a second preset coefficient.

[0024] Further, according to the end point position of the real crack, a real crack capable of being connected with the end point of the current real crack is found in the complete dam body region image, and continuity splicing is performed to obtain a corresponding complete crack, specifically:

[0025] In the complete dam body region image, a real crack whose end point position coincides with the end point position of the current real crack or a real crack whose end point position is on the current real crack region is found, and splicing is performed with the current real crack to form a temporary complete crack;

[0026] Then, a real crack whose end point position coincides with the end point position of the current temporary complete crack or a real crack whose end point position is on the current real crack region is found and splicing is performed to form a new temporary complete crack, and the step is repeated until a real crack whose end point position coincides with the end point position of the new temporary complete crack and a real crack whose end point position is on the new temporary complete crack region cannot be found, and the latest temporary complete crack is taken as the complete crack.

[0027] Further, the settlement depression information of the dam foundation region is analyzed according to the dam foundation region image, and the dam foundation abnormal value is calculated according to the settlement depression information, specifically, the settlement depression region is identified from the dam foundation region image by using a second pre-trained deep learning model, the total number N3 of the settlement depression regions in all dam foundation region images and the pixel grid number N4 of each settlement depression region in the corresponding dam foundation region image are counted;

[0028] The dam foundation abnormal value BJ is calculated, BJ=k3*N3+k4*N4, wherein k3 represents a third preset coefficient, and k4 represents a fourth preset coefficient.

[0029] Further, the surface spalling information of the slope region is analyzed according to the slope region image, and the slope abnormal value is calculated according to the surface spalling information, specifically, the surface spalling region is identified from the slope region image by using a third pre-trained deep learning model, the total number N5 of the surface spalling regions in all slope region images and the pixel grid number N6 of each settlement depression region in the corresponding dam foundation region image are counted;

[0030] The dam foundation abnormal value BP is calculated, BP=k5*N5+k6*N6, wherein k5 represents a fifth preset coefficient, and k6 represents a sixth preset coefficient.

[0031] Further, the instability evaluation module evaluates the instability value BW of the target embankment according to the abnormal values of the sub-regions, specifically:

[0032] BW=r1*BT+r2*BJ+r3*BP, wherein BW represents an unstable value, r1 represents a first preset weight, r2 represents a second preset weight, and r3 represents a third preset weight.

[0033] Further, the alarm module sends alarm information to the user according to the abnormal value of each sub-region and the unstable value of the target dam, specifically:

[0034] If the dam body abnormal value is greater than a first preset abnormal threshold, dam body abnormal alarm information is sent to the user.

[0035] If the dam foundation abnormal value is greater than a second preset abnormal threshold, dam foundation abnormal alarm information is sent to the user.

[0036] If the slope abnormal value is greater than a third preset abnormal threshold, slope abnormal alarm information is sent to the user.

[0037] If the unstable value of the target dam is greater than a preset comprehensive threshold, dam instability alarm information is sent to the user.

[0038] The application further provides a water conservancy facility patrol and early warning method based on a UAV, comprising the following steps:

[0039] A dam panoramic image of a target dam is obtained by a UAV carrying a shooting device.

[0040] Sub-regions of the target dam are divided from the dam panoramic image, including a dam foundation region, a dam body region and a slope region.

[0041] Shooting detection points of each sub-region are analyzed.

[0042] Corresponding detection images are obtained by the UAV shooting at each shooting detection point, and the detection images are divided into dam foundation region images, dam body region images and slope region images.

[0043] According to each detection image, the abnormal value of each sub-region is analyzed.

[0044] According to the abnormal value of each sub-region, the unstable value of the target dam is evaluated.

[0045] According to the abnormal value of each sub-region and the unstable value of the target dam, alarm information is sent to the user.

[0046] Compared with the prior art, the application has the following beneficial effects:

[0047] After the sub-regions of the target dam are divided, the shooting detection points of the unmanned aerial vehicle are determined, the shooting detection points of each sub-region ensure that all detection images corresponding to the sub-region after the unmanned aerial vehicle completes shooting in the corresponding sub-region cover the corresponding sub-region with the least number and the detection images are not lower than the preset definition, the unmanned aerial vehicle patrol efficiency is improved, the instability value of the target dam is comprehensively evaluated according to the abnormal values of each sub-region, the evaluation accuracy and comprehensiveness of the instability value of the dam are improved, and the abnormal sub-region is quickly positioned according to the abnormal values of different sub-regions and the instability value of the target dam, the dam safety maintenance efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0048] The drawings incorporated into the specification and forming a part thereof illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows, and obviously, other drawings can be obtained by those skilled in the art without creative labor.

[0050] Figure 1 is a structural block diagram of the water conservancy facility patrol and early warning system based on the unmanned aerial vehicle of the present application;

[0051] Figure 2 is a structural block diagram of the control center of the water conservancy facility patrol and early warning system based on the unmanned aerial vehicle of the present application;

[0052] Figure 3 is a structural block diagram of the abnormality analysis module of the water conservancy facility patrol and early warning system based on the unmanned aerial vehicle of the present application;

[0053] Figure 4 is a flowchart of the water conservancy facility patrol and early warning method based on the unmanned aerial vehicle of the present application. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application, and obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0055] It should be noted that all direction indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative position relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the direction indications will also change accordingly.

[0056] In addition, the description involving "first", "second" and the like in the present application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of a person skilled in the art, and when the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist, and is not within the protection scope required by the present application.

[0057] Please refer to Figure 1 As shown in the figure, the unmanned aerial vehicle-based water conservancy facility patrol and early warning system provided by the present application comprises a control center and an unmanned aerial vehicle carrying a camera device, and the control center is in communication connection with the unmanned aerial vehicle.

[0058] Please refer to Figure 2 As shown in the figure, the control center comprises a first image acquisition module, a region division module, a detection point determination module, a second image acquisition module, an anomaly analysis module, an instability evaluation module and an alarm module, and a central processing unit in communication connection with the above-mentioned modules.

[0059] The first image acquisition module is used to acquire the dam panoramic image of the target dam shot by the unmanned aerial vehicle.

[0060] The region division module is used to divide the sub-regions of the target dam according to the dam panoramic image, specifically including the dam foundation region, the dam body region and the slope region.

[0061] The detection point determination module is used to determine the shooting detection points of the unmanned aerial vehicle in each sub-region of the target dam, and the shooting detection points of each sub-region ensure that all detection images corresponding to the corresponding sub-region are covered after the unmanned aerial vehicle completes shooting in the corresponding sub-region, with the number of detection images being the least and the detection images being not lower than the preset definition.

[0062] The second image acquisition module is used to acquire the corresponding detection images shot by the unmanned aerial vehicle in each shooting detection point, and the detection images are divided into dam foundation region images, dam body region images and slope region images.

[0063] The anomaly analysis module is used to identify and analyze the abnormal values of each sub-region according to each detection image.

[0064] An instability evaluation module is configured to evaluate an instability value of the target dam according to the abnormal value of each sub-region.

[0065] An alarm module is configured to send alarm information to a user according to the abnormal value of each sub-region and the instability value of the target dam.

[0066] The region division module divides the sub-regions of the target dam according to the panoramic image of the dam, which can be achieved by manual division or recognition division through a database pre-stored with the position data of the sub-regions of the target dam.

[0067] Referring to Figure 3 The abnormality analysis module includes a dam foundation abnormality analysis module, a dam body abnormality analysis module and a slope abnormality analysis module.

[0068] The dam body abnormality analysis module is configured to analyze the complete crack information of the dam body region according to the dam body region image and calculate the dam body abnormality value according to the complete crack information, specifically as follows.

[0069] A suspected crack contour in the dam body region image is extracted by using an edge detection algorithm;

[0070] A real crack is identified from the suspected crack contour by using a pre-trained first deep learning model;

[0071] Each dam body region image is spliced to obtain a complete dam body region image according to the shooting detection points of the dam body region;

[0072] A real crack that can be connected with the end point of the current real crack is found in the complete dam body region image according to the end point position of the real crack, and continuity splicing is performed to obtain a corresponding complete crack;

[0073] It is judged whether the corresponding complete crack passes through an important region of the dam body according to the end point position of the complete crack, and the number N1 of the complete cracks passing through the important region of the dam body is recorded.

[0074] The total number G of pixel grids of the complete dam body region image and the number g of pixel grids occupied by each complete crack in the complete dam body region image are obtained, and if g / G is greater than or equal to a first preset threshold, the corresponding complete crack is determined as a serious crack, and the number N2 of the serious cracks is counted.

[0075] The dam body abnormality value BT is calculated, BT=k1*N1+k2*N2, wherein k1 represents a first preset coefficient and k2 represents a second preset coefficient.

[0076] The edge detection algorithm can select the Canny edge detection algorithm. The Canny edge detection algorithm usually converts a color image into a grayscale image at present, because the Canny algorithm is designed based on a single-channel grayscale image, and in order to reduce noise in the image and maintain the integrity of the edge, a Gaussian filter is used to smooth the grayscale image. The Gaussian filter is a low-pass filter that can effectively remove high-frequency noise in the image. Subsequently, on the Gaussian smoothed image, a gradient operator (selecting the Sobel operator) is used to calculate the gradient size and direction of each pixel. In order to eliminate false responses generated in gradient calculation, the gradient direction of each pixel is checked. If the gradient intensity of the pixel is not the local maximum along the gradient direction, the gradient value of the pixel is suppressed. Strong edge points are detected by a high threshold, and weak edge points are detected by a low threshold. When the edges are connected, it is checked whether each weak edge point is adjacent to any strong edge point. If so, the weak edge point is retained. Therefore, the Canny algorithm can ensure the continuity and integrity of the edge points, thereby producing clear and continuous edge contours.

[0077] The first deep learning model can select a U-Net model or a convolutional neural network model, and train a large number of dam crack images as a training set.

[0078] The dam region images are spliced to obtain a complete dam region image according to the shooting detection points of the dam region. For example, in the dam region, the shooting detection point A is adjacent to the shooting detection point B, and the point A is on the right side of the point B. Therefore, the dam region image A corresponding to the shooting detection point A and the dam region image B corresponding to the shooting detection point B can be spliced, and the dam region image A is spliced on the right side of the dam region image B.

[0079] According to the end point position of the real crack, a real crack that can be connected with the end point of the current real crack is found in the complete dam region image, and continuity splicing is performed to obtain a corresponding complete crack. Specifically,

[0080] In the complete dam region image, a real crack whose end point position coincides with the end point position of the current real crack, or a real crack whose end point position is on the current real crack region, is found, and then spliced with the current real crack to form a temporary complete crack.

[0081] Then, a real crack whose end point position coincides with the end point position of the current temporary complete crack, or a real crack whose end point position is on the current real crack region, is found, and then spliced to form a new temporary complete crack. This step is repeated until a real crack whose end point position coincides with the end point position of the new temporary complete crack and a real crack whose end point position is on the new temporary complete crack region cannot be found. The latest temporary complete crack is the complete crack.

[0082] The dam body important region is pre-divided by the staff according to the dam body structure of the target dam.

[0083] The dam foundation anomaly analysis module is configured to analyze settlement depression information of the dam foundation region according to the dam foundation region image, and calculate a dam foundation anomaly value according to the settlement depression information, specifically, identify settlement depression regions from the dam foundation region image through a pre-trained second deep learning model, and count a total number N3 of the settlement depression regions in all dam foundation region images and a pixel grid number N4 of each settlement depression region in the corresponding dam foundation region image.

[0084] The dam foundation anomaly value BJ is calculated as BJ=k3*N3+k4*N4, where k3 represents a third preset coefficient and k4 represents a fourth preset coefficient.

[0085] The second deep learning model can be a U-Net model or a convolutional neural network model, and is trained with a large number of dam foundation settlement depression images as a training set.

[0086] The slope anomaly analysis module is configured to analyze surface spalling information of the slope region according to the slope region image, and calculate a slope anomaly value according to the surface spalling information, specifically, identify surface spalling regions from the slope region image through a pre-trained third deep learning model, and count a total number N5 of the surface spalling regions in all slope region images and a pixel grid number N6 of each settlement depression region in the corresponding dam foundation region image.

[0087] The dam foundation anomaly value BP is calculated as BP=k5*N5+k6*N6, where k5 represents a fifth preset coefficient and k6 represents a sixth preset coefficient.

[0088] The third deep learning model can be a U-Net model or a convolutional neural network model, and is trained with a large number of slope surface spalling images as a training set.

[0089] The instability evaluation module evaluates an instability value of the target dam according to the anomaly values of the sub-regions, specifically:

[0090] BW=r1*BT+r2*BJ+r3*BP, where BW represents the instability value, r1 represents a first preset weight, r2 represents a second preset weight, and r3 represents a third preset weight. In this embodiment, r1+r2+r3=1.

[0091] The alarm module sends alarm information to the user according to the anomaly values of the sub-regions and the instability value of the target dam, specifically:

[0092] If the dam body anomaly value is greater than a first preset anomaly threshold, the dam body anomaly alarm information is sent to the user.

[0093] If the dam foundation abnormal value is greater than a second preset abnormal threshold, dam foundation abnormality alarm information is sent to the user;

[0094] If the slope abnormal value is greater than a third preset abnormal threshold, slope abnormality alarm information is sent to the user;

[0095] If the instability value of the target embankment is greater than a preset comprehensive threshold, embankment instability alarm information is sent to the user.

[0096] In another embodiment, the first image acquisition module is further configured to acquire a second facility image captured by the UAV, the second facility image including an embankment image, a revetment image, and a water gate image.

[0097] The second facility image is acquired by the UAV after acquiring dam foundation region images, dam body region images, and slope region images according to the detection points for shooting.

[0098] The embankment image is an image of an embankment region that has an impact on the stability of the target embankment, and the embankment region that has an impact on the stability of the target embankment is manually divided or identified and divided through the database; the revetment image is an image of a revetment region that has an impact on the stability of the target embankment, and the revetment region that has an impact on the stability of the target embankment is manually divided or identified and divided through the database.

[0099] The database pre-stores embankment region position data that has an impact on the stability of the target embankment and revetment region position data that has an impact on the stability of the target embankment.

[0100] The abnormality analysis module further includes an embankment abnormality analysis module, a revetment abnormality analysis module, a water gate abnormality analysis module, and a pump station abnormality analysis module.

[0101] The embankment abnormality analysis module is configured to analyze embankment crack information of the embankment according to the embankment image, and calculate an embankment abnormal value DF according to the embankment crack information, specifically by identifying embankment cracks from the embankment image through a pre-trained fourth deep learning model, counting the total number N7 of embankment cracks and the number N8 of pixel grids occupied by the embankment cracks in the embankment image;

[0102] DF=k7*N7+k8*N8, wherein k7 represents a seventh preset coefficient and k8 represents an eighth preset coefficient;

[0103] The fourth deep learning model can be a U-Net model or a convolutional neural network model, and is trained with a large number of embankment crack images as a training set.

[0104] The revetment anomaly analysis module is configured to analyze revetment crack information of the revetment according to the revetment image, and calculate a revetment anomaly value HA according to the revetment crack information, specifically, by identifying the revetment cracks from the revetment image through a pre-trained fifth deep learning model, counting a total number N9 of the revetment cracks and a pixel grid number N10 of the revetment cracks in the revetment image;

[0105] HA=k9*N9+k10*N10, wherein k9 represents a ninth preset coefficient, and k10 represents a tenth preset coefficient.

[0106] The fifth deep learning model can be a U-Net model or a convolutional neural network model, and is trained with a large number of revetment crack images as a training set.

[0107] The sluice anomaly analysis module is configured to analyze sluice crack information and sluice corrosion information of the sluice according to the sluice image, and calculate a sluice anomaly value SZ according to the sluice crack information and the sluice corrosion information, specifically, by identifying the sluice cracks from the sluice image through a pre-trained sixth deep learning model, counting a total number N11 of the sluice cracks and a pixel grid number N12 of the sluice cracks in the sluice image, identifying a sluice corrosion area from the sluice image through a pre-trained seventh deep learning model, and counting a pixel grid number N13 of the sluice corrosion area in the sluice image.

[0108] SZ=k11*N11+k12*N12+k13*N13, wherein k11 represents an eleventh preset coefficient, k12 represents a twelfth preset coefficient, and k13 represents a thirteenth preset coefficient.

[0109] The sixth deep learning model can be a U-Net model or a convolutional neural network model, and is trained with a large number of sluice crack images as a training set, and the seventh deep learning model can be a U-Net model or a convolutional neural network model, and is trained with a large number of sluice corrosion images as a training set.

[0110] The instability evaluation module further evaluates an instability value of the target dike according to the dike anomaly value, the revetment anomaly value, the sluice anomaly value, and the anomaly values of the sub-regions, specifically:

[0111] BW=r1*BT+r2*BJ+r3*BP+r4*DF+r5*HA+r6*SZ, wherein r4 represents a fourth preset weight, r5 represents a fifth preset weight, and r6 represents a sixth preset weight, and in this embodiment, r1+r2+r3+r4+r5+r6=1.

[0112] The warning module further sends warning information to the user according to the dike anomaly value, the revetment anomaly value, and the sluice anomaly value, specifically:

[0113] If the dike abnormal value is greater than a fourth preset abnormal threshold, a dike abnormality alarm information is sent to the user;

[0114] If the revetment abnormal value is greater than a fifth preset abnormal threshold, a revetment abnormality alarm information is sent to the user;

[0115] If the water gate abnormal value is greater than a sixth preset abnormal threshold, a water gate abnormality alarm information is sent to the user.

[0116] Embodiment two

[0117] Referring to Figure 4 The present application also provides a water conservancy facility patrol and early warning method based on a UAV, which specifically comprises the following steps:

[0118] S1, obtaining a dike panoramic image of a target dike through a UAV carrying a shooting device;

[0119] S2, dividing a sub-region of the target dike through the dike panoramic image, including a dam foundation region, a dam body region and a slope region;

[0120] S3, analyzing a shooting detection point of each sub-region;

[0121] S4, obtaining a corresponding detection image of each shooting detection point of the UAV, the detection image being divided into a dam foundation region image, a dam body region image and a slope region image;

[0122] S5, analyzing an abnormal value of each sub-region according to each detection image;

[0123] S6, evaluating an instability value of the target dike according to the abnormal value of each sub-region;

[0124] S7, sending an alarm information to the user according to the abnormal value of each sub-region and the instability value of the target dike.

[0125] For step S3, the shooting detection point of each sub-region ensures that all detection images corresponding to the corresponding sub-region after the UAV completes shooting in the corresponding sub-region cover the corresponding sub-region with the least number of detection images and the detection images being not lower than a preset definition.

[0126] For step S5, the abnormal value of each sub-region includes analyzing a dam foundation abnormal value, a dam body abnormal value and a slope abnormal value.

[0127] T1, analyzing the dam body abnormal value, specifically comprising the following steps:

[0128] T2, extracting a suspected crack contour in the dam body region image by using an edge detection algorithm;

[0129] T3, identifying a real crack from the suspected crack contour by using a first pre-trained deep learning model.

[0130] T4, splicing each dam body region image to obtain a complete dam body region image according to the shooting detection points of the dam body region;

[0131] T5, finding a real crack that can be connected with the end point of the current real crack in the complete dam body region image according to the end point position of the real crack, and performing continuity splicing to obtain a corresponding complete crack;

[0132] T6, judging whether the corresponding complete crack passes through the important region of the dam body according to the end point position of the complete crack, and recording the number N1 of complete cracks passing through the important region of the dam body;

[0133] T7, obtaining the total pixel grid number G of the complete dam body region image and the pixel grid number g occupied by each complete crack in the complete dam body region image, and if g / G is greater than or equal to a first preset threshold, judging that the corresponding complete crack is a serious crack, and counting the number N2 of serious cracks;

[0134] T8, calculating a dam body anomaly value BT, BT=k1*N1+k2*N2, wherein k1 represents a first preset coefficient, and k2 represents a second preset coefficient.

[0135] The first deep learning model can select a U-Net model or a convolutional neural network model, and a large number of dam body crack images are used as a training set for training.

[0136] For step T5, a real crack that can be connected with the end point of the current real crack is found in the complete dam body region image according to the end point position of the real crack, and continuity splicing is performed to obtain a corresponding complete crack, which is specifically:

[0137] In the complete dam body region image, a real crack whose end point position coincides with the end point position of the current real crack, or a real crack whose end point position is on the current real crack region, is found, and then splicing is performed with the current real crack to form a temporary complete crack;

[0138] Then, a real crack whose end point position coincides with the end point position of the current temporary complete crack, or a real crack whose end point position is on the current real crack region, is found, and then splicing is performed to form a new temporary complete crack, and the step is repeated until a real crack whose end point position coincides with the end point position of the new temporary complete crack and a real crack whose end point position is on the new temporary complete crack region cannot be found, and the latest temporary complete crack is the complete crack.

[0139] The important region of the dam body is pre-divided by the staff according to the dam body structure of the target dam.

[0140] Analyzing the dam foundation anomaly value, specifically including the following steps:

[0141] identify the settlement depression regions from the dam foundation region images through the pre-trained second deep learning model, count the total number N3 of the settlement depression regions in all the dam foundation region images and the pixel grid number N4 of each settlement depression region in the corresponding dam foundation region image;

[0142] calculate the dam foundation abnormal value BJ, BJ=k3*N3+k4*N4, wherein k3 represents the third preset coefficient and k4 represents the fourth preset coefficient.

[0143] The second deep learning model can be a U-Net model or a convolutional neural network model and is trained with a large number of dam foundation settlement depression images as a training set.

[0144] analyze the slope abnormal value, specifically including the following steps:

[0145] identify the surface spalling regions from the slope region images through the pre-trained third deep learning model, count the total number N5 of the surface spalling regions in all the slope region images and the pixel grid number N6 of each settlement depression region in the corresponding dam foundation region image;

[0146] calculate the dam foundation abnormal value BP, BP=k5*N5+k6*N6, wherein k5 represents the fifth preset coefficient and k6 represents the sixth preset coefficient.

[0147] The third deep learning model can be a U-Net model or a convolutional neural network model and is trained with a large number of slope surface spalling images as a training set.

[0148] For step S6, evaluate the instability value of the target embankment according to the abnormal values of the sub-regions, specifically:

[0149] BW=r1*BT+r2*BJ+r3*BP, wherein BW represents the instability value, r1 represents the first preset weight, r2 represents the second preset weight, and r3 represents the third preset weight.

[0150] For step S7, send an alarm information to the user according to the abnormal values of the sub-regions and the instability value of the target embankment, specifically:

[0151] if the dam body abnormal value is greater than the first preset abnormal threshold, send a dam body abnormal alarm information to the user;

[0152] if the dam foundation abnormal value is greater than the second preset abnormal threshold, send a dam foundation abnormal alarm information to the user;

[0153] if the slope abnormal value is greater than the third preset abnormal threshold, send a slope abnormal alarm information to the user;

[0154] If the instability value of the target dam is greater than the preset comprehensive threshold value, a dam instability warning information is sent to a user.

[0155] Embodiment three

[0156] The application further provides an electronic device, comprising a processor, a sending device, an input device, an output device and a memory, the memory is used for storing computer program code, the computer program code comprises computer instructions, when the processor executes the computer instructions, the electronic device executes the method in any one of the possible implementation manners described above.

[0157] Embodiment four

[0158] The application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program comprises program instructions, when the program instructions are executed by a processor of an electronic device, the processor executes the method in any one of the possible implementation manners described above.

[0159] The application has the following beneficial effects:

[0160] By dividing the target dam into sub-regions, the shooting detection points of the unmanned aerial vehicle are determined, the shooting detection points of each sub-region ensure that all detection images corresponding to the sub-region after the unmanned aerial vehicle completes shooting in the corresponding sub-region cover the corresponding sub-region with the least number of detection images and the detection images are not lower than the preset definition, the unmanned aerial vehicle patrol efficiency is improved, the instability value of the target dam is comprehensively evaluated according to the abnormal values of each sub-region, the evaluation accuracy and comprehensiveness of the instability value of the dam are improved, and the dam safety maintenance efficiency is improved

[0161] In the description of the specification, the description of the terms "one embodiment", "example", "specific example" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0162] The above is only a specific embodiment of the application, which enables those skilled in the art to understand or implement the application. Various modifications of these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A drone-based water conservancy facility inspection and early warning system, comprising a control center and a drone equipped with a camera device, wherein the control center is communicatively connected to the drone, characterized in that, The control center includes: a first image acquisition module, a region division module, a detection point determination module, a second image acquisition module, an anomaly analysis module, an instability assessment module, and an alarm module; The first image acquisition module is used to acquire a panoramic image of the target dam taken by the drone; The region division module is used to divide the target dam into sub-regions based on the panoramic image of the dam. The target dam's sub-regions include the dam foundation region, the dam body region, and the slope region; the detected images include images of the dam foundation region, the dam body region, and the slope region. The detection point determination module is used to determine the shooting detection points of the UAV in each sub-region of the target dam. The shooting detection points in each sub-region ensure that all detection images corresponding to the UAV after shooting in the corresponding sub-region cover the corresponding sub-region with the minimum number of images and the detection images having a resolution not lower than the preset resolution. The second image acquisition module is used to acquire the corresponding detection images captured by the UAV at each shooting detection point; The anomaly analysis module is used to identify and analyze the abnormal values ​​of each sub-region based on each detected image. The anomaly analysis module includes a dam anomaly analysis module, used to analyze the complete crack information of the dam area based on the dam area image, and calculate the dam anomaly value BT based on the complete crack information, specifically: The algorithm extracts the outlines of suspected cracks in the dam area image using an edge detection algorithm; identifies real cracks from the suspected crack outlines using a pre-trained first deep learning model; stitches together the images of each dam area based on the detection points of the dam area to obtain a complete dam area image; finds real cracks in the complete dam area image that can connect to the endpoints of the current real cracks based on the endpoint positions of the real cracks, and stitches them continuously to obtain the corresponding complete cracks; determines whether the corresponding complete crack passes through an important area of ​​the dam based on the endpoint positions of the complete cracks, and records the number N1 of complete cracks passing through the important area of ​​the dam; obtains the total number of pixels G in the complete dam area image and the number of pixels g occupied by each complete crack in the complete dam area image; if g / G is greater than or equal to a first preset threshold, the corresponding complete crack is determined to be a serious crack, and the number of serious cracks N2 is counted; calculates the dam anomaly value BT, BT=k1*N1+k2*N2, where k1 represents the first preset coefficient and k2 represents the second preset coefficient. The process involves finding real cracks in the complete dam area image that can be connected to the endpoints of the current real cracks based on their endpoint locations, and then continuously stitching them together to obtain the corresponding complete cracks. Specifically: In the image of the complete dam area, find a real crack whose endpoint position coincides with the endpoint position of the current real crack, or a real crack whose endpoint position is on the current real crack area, and then stitch it with the current real crack to form a temporary complete crack; then find a real crack whose endpoint position coincides with the endpoint position of the current temporary complete crack, or a real crack whose endpoint position is on the current real crack area, and then stitch it together to form a new temporary complete crack. Repeat this step until no real crack whose endpoint position coincides with the endpoint position of the new temporary complete crack or a real crack whose endpoint position is on the new temporary complete crack area can be found. The latest temporary complete crack is then taken as the complete crack. The anomaly analysis module includes a dam foundation anomaly analysis module and a slope anomaly analysis module. The dam foundation anomaly analysis module is used to analyze the settlement and depression information of the dam foundation area based on the dam foundation area image, and calculate the dam foundation anomaly value BJ based on the settlement and depression information. Specifically, it identifies the settlement and depression areas from the dam foundation area image through a pre-trained second deep learning model, counts the total number N3 of settlement and depression areas in all dam foundation area images and the number of pixels N4 occupied by each settlement and depression area in the corresponding dam foundation area image; and calculates the dam foundation anomaly value BJ, BJ=k3*N3+k4*N4, where k3 represents the third preset coefficient and k4 represents the fourth preset coefficient. The slope anomaly analysis module is used to analyze the surface spalling information of the slope area based on the slope area image, and calculate the slope anomaly value BP based on the surface spalling information. Specifically, it identifies the surface spalling area from the slope area image through a pre-trained third deep learning model, counts the total number of surface spalling areas N5 in all slope area images and the number of pixels N6 occupied by each settlement depression area in the corresponding dam foundation area image; and calculates the dam foundation anomaly value BP, BP=k5*N5+k6*N6, where k5 represents the fifth preset coefficient and k6 represents the sixth preset coefficient. The instability assessment module is used to assess the instability value BW of the target dam based on the outliers in each sub-region; The alarm module is used to issue alarm information to the user based on the abnormal values ​​of each sub-region and the instability value of the target dam. Specifically, if the abnormal value of the dam body is greater than a first preset abnormal threshold, an alarm information for the dam body abnormality is issued to the user; if the abnormal value of the dam foundation is greater than a second preset abnormal threshold, an alarm information for the dam foundation abnormality is issued to the user; if the abnormal value of the slope is greater than a third preset abnormal threshold, an alarm information for the slope abnormality is issued to the user; and if the instability value of the target dam is greater than a preset comprehensive threshold, an alarm information for the dam instability is issued to the user. The first image acquisition module is also used to acquire second facility images captured by the UAV, the second facility images including dike images, revetment images, and sluice gate images; The anomaly analysis module also includes an anomaly analysis module for dikes, anomaly analysis module for revetments, anomaly analysis module for sluice gates, and anomaly analysis module for pumping stations; The levee anomaly analysis module is used to identify levee cracks from levee images using a pre-trained fourth deep learning model, count the total number of levee cracks N7 and the number of pixel grids occupied by levee cracks in the levee image N8, and calculate the levee anomaly value DF, DF=k7*N7+k8*N8, where k7 represents the seventh preset coefficient and k8 represents the eighth preset coefficient. The bank protection anomaly analysis module is used to identify bank protection cracks from bank protection images using a pre-trained fifth deep learning model, count the total number of bank protection cracks N9 and the number of pixel grids occupied by bank protection cracks in the bank protection image N10, and calculate the bank protection anomaly value HA, HA=k9*N9+k10*N10, where k9 represents the ninth preset coefficient and k10 represents the tenth preset coefficient. The sluice gate anomaly analysis module is used to identify sluice gate cracks from sluice gate images using a pre-trained sixth deep learning model, count the total number of sluice gate cracks N11 and the number of pixels occupied by sluice gate cracks in the sluice gate image N12, identify sluice gate corrosion areas from sluice gate images using a pre-trained seventh deep learning model, count the number of pixels occupied by sluice gate corrosion areas in the sluice gate image N13, and calculate the sluice gate anomaly value SZ, SZ=k11*N11+k12*N12+k13*N13, where k11 represents the eleventh preset coefficient, k12 represents the twelfth preset coefficient, and k13 represents the thirteenth preset coefficient; BW = r1*BT + r2*BJ + r3*BP + r4*DF + r5*HA + r6*SZ, where r1 represents the first preset weight, r2 represents the second preset weight, r3 represents the third preset weight, r4 represents the fourth preset weight, r5 represents the fifth preset weight, and r6 represents the sixth preset weight.

2. A method for unmanned aerial vehicle (UAV)-based inspection and early warning of water conservancy facilities, applied to the UAV-based inspection and early warning system for water conservancy facilities as described in claim 1, characterized in that, Includes the following steps: A panoramic image of the target dam was obtained using a drone equipped with a camera. The target dam is divided into sub-regions by using panoramic images of the dam, including the dam foundation region, the dam body region, and the slope region; The detection points for each sub-region were analyzed; The detection images captured by the drone at each detection point are obtained. The detection images are divided into images of the dam foundation area, the dam body area, and the slope area. Based on the detected images, analyze the outliers in each sub-region: The process involves analyzing the complete crack information of the dam area based on the dam area image and calculating the dam outlier BT based on the complete crack information. Specifically, this includes: extracting suspected crack contours from the dam area image using an edge detection algorithm; identifying real cracks from the suspected crack contours using a pre-trained first deep learning model; stitching together the images of each dam area based on the detection points of the dam area to obtain a complete dam area image; and finding real cracks in the complete dam area image that can be connected to the endpoints of the current real cracks based on the endpoint positions of the real cracks, and stitching them together continuously to obtain the corresponding complete cracks. Determine whether the corresponding complete crack passes through an important area of ​​the dam body based on the endpoint position of the complete crack, and record the number N1 of complete cracks that pass through the important area of ​​the dam body; Obtain the total number of pixels G in the image of the complete dam area and the number of pixels g occupied by each complete crack in the image of the complete dam area. If g / G is greater than or equal to a first preset threshold, the corresponding complete crack is determined to be a severe crack, and the number of severe cracks N2 is counted. Calculate the dam body anomaly value BT, BT=k1*N1+k2*N2, where k1 represents the first preset coefficient and k2 represents the second preset coefficient; The process involves finding real cracks in the complete dam area image that can connect to the endpoints of the current real cracks, based on the endpoint positions of the real cracks, and continuously splicing them to obtain the corresponding complete cracks. Specifically, in the complete dam area image, after finding real cracks whose endpoint positions coincide with the endpoint positions of the current real cracks, or whose endpoint positions are on the current real crack area, splicing them with the current real cracks to form temporary complete cracks; then finding real cracks whose endpoint positions coincide with the endpoint positions of the current temporary complete cracks, or whose endpoint positions are on the current real crack area, and splicing them again to form new temporary complete cracks. This step is repeated until no real cracks whose endpoint positions coincide with the endpoint positions of the new temporary complete cracks or whose endpoint positions are on the new temporary complete crack area can be found. The latest temporary complete crack is then taken as the complete crack. The settlement and depression information of the dam foundation area is analyzed based on the images of the dam foundation area, and the dam foundation anomaly value BJ is calculated based on the settlement and depression information. Specifically, the settlement and depression areas are identified from the dam foundation area images by a pre-trained second deep learning model. The total number of settlement and depression areas N3 in all dam foundation area images and the number of pixels N4 occupied by each settlement and depression area in the corresponding dam foundation area image are counted. The dam foundation anomaly value BJ is calculated as BJ=k3*N3+k4*N4, where k3 represents the third preset coefficient and k4 represents the fourth preset coefficient. The surface spalling information of the slope area is analyzed based on the slope area images, and the slope anomaly value BP is calculated based on the surface spalling information. Specifically, the surface spalling areas are identified from the slope area images by a pre-trained third deep learning model. The total number of surface spalling areas N5 in all slope area images and the number of pixels N6 occupied by each settlement depression area in the corresponding dam foundation area image are counted. The dam foundation anomaly value BP is calculated as BP = k5 * N5 + k6 * N6, where k5 represents the fifth preset coefficient and k6 represents the sixth preset coefficient. Acquire second facility images captured by the drone, the second facility images including images of dikes, revetments, and sluice gates; The fourth deep learning model, pre-trained, identifies dike cracks from dike images, counts the total number of cracks (N7) and the number of pixels occupied by cracks in the dike image (N8), and calculates the dike anomaly value DF: DF = k7*N7 + k8*N8, where k7 represents the seventh preset coefficient and k8 represents the eighth preset coefficient. The fifth deep learning model, pre-trained, identifies revetment cracks from revetment images, counts the total number of revetment cracks (N9) and the number of pixels occupied by cracks in the revetment image (N10), and calculates the revetment anomaly value HA: HA = k9*N9 + k10*N10, where k9 represents the ninth preset coefficient. The number, k10, represents the tenth preset coefficient; it is used to identify sluice gate cracks from sluice gate images using the pre-trained sixth deep learning model, count the total number of sluice gate cracks N11 and the number of pixel grids occupied by sluice gate cracks in sluice gate images N12, identify sluice gate corrosion areas from sluice gate images using the pre-trained seventh deep learning model, count the number of pixel grids occupied by sluice gate corrosion areas in sluice gate images N13, and calculate the sluice gate outlier value SZ, SZ=k11*N11+k12*N12+k13*N13, k11 represents the eleventh preset coefficient, k12 represents the twelfth preset coefficient, and k13 represents the thirteenth preset coefficient; The instability value of the target dam is assessed based on the outliers in each sub-region, specifically as follows: BW=r1*BT+r2*BJ+r3*BP+r4*DF+r5*HA+r6*SZ, where BW represents the instability value, r1 represents the first preset weight, r2 represents the second preset weight, r3 represents the third preset weight, r4 represents the fourth preset weight, r5 represents the fifth preset weight, and r6 represents the sixth preset weight. Based on the outliers in each sub-region and the instability value of the target dam, alarm messages are issued to the user. Specifically: if the outlier of the dam body is greater than a first preset outlier threshold, an outlier alarm message is issued to the user; if the outlier of the dam foundation is greater than a second preset outlier threshold, an outlier alarm message is issued to the user; if the outlier of the slope is greater than a third preset outlier threshold, an outlier alarm message is issued to the user; if the instability value of the target dam is greater than a preset comprehensive threshold, an instability alarm message is issued to the user.

Images