Deformable body crack automatic monitoring and early warning method based on unmanned aerial vehicle airport

Abstract

The invention discloses a deformation body crack automatic monitoring and early warning method based on an unmanned aerial vehicle airport, and the method comprises the steps: S1, carrying out the interpretation of an initial feature value of a crack; S2, planning and testing an inspection route; S3, arranging an unmanned aerial vehicle hangar, executing an automatic inspection task, and collecting an inspection photo through the inspection of an unmanned aerial vehicle; s4, automatically uploading the patrolled photo to a data center through a network, carrying out automatic modeling by adopting an open source modeling algorithm, generating an orthoimage, and numbering the image; s5, a deep learning model is adopted to recognize the orthoimage after inspection, the recognized crack is automatically repaired, and the contour of the crack is extracted and output from the repaired and recognized result in a vector form; s6, performing multi-stage comparison and early warning on the crack; the problems of low deformation body crack inspection efficiency, poor effect and high labor cost in the prior art are solved, and full-automatic inspection operation and abnormity early warning are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of geological disaster monitoring and early warning, and in particular relates to an automatic monitoring and early warning method for deformable body cracks based on an unmanned aerial vehicle airport. Background Art

[0002] Deformed bodies are a type of geological disaster body widely distributed in the western high mountain and canyon areas, which have different degrees of impact on the safe operation of hydropower stations and other projects. In the operation and management of power stations, in addition to the use of instrument monitoring and other means, regular inspections of deformed bodies are also an important task in the daily operation and management of geological disasters. Traditional inspections mainly rely on manual inspections, which have problems such as high safety risks and difficulty in personnel arrival in high mountain and canyon areas. In addition, the inspection method for deformed bodies is mainly to observe the changes in cracks on the deformed bodies, which is also greatly affected by subjective factors, and it is difficult to fully and accurately grasp the actual changes in cracks. In the prior art, there is a method of collecting image data by taking photos or videos by drones, and then manually judging and evaluating. Due to the different clarity and resolution of the image and video image data directly obtained by drone shooting data, the accuracy of manual judgment is difficult to check, and the flight collection path of the drone is chaotic during flight, and there are a large number of missed areas and blind spots in the shooting detection; therefore, it is necessary to design an automatic monitoring and early warning method for deformed body cracks based on drone airports to solve the above problems. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an automatic monitoring and early warning method for deformable body cracks based on a drone airport. The method aims to solve the problems of low efficiency, poor effect and high labor cost in the prior art for deformable body crack inspection, and realize fully automated inspection operations and abnormal early warning.

[0004] In order to achieve the above technical effects, the technical solution adopted by the present invention is: A method for automatic monitoring and early warning of cracks in a deformable body based on an unmanned aerial vehicle airport comprises the following steps: S1, interpretation of initial crack eigenvalues: The deformable body is photographed by close-up aerial photography to establish a high-precision 3D real scene model and 3D point cloud model of the deformable body; Interpret geological hazards based on the model, determine the cracks that need to be inspected, and read the crack characteristic values; S2, inspection route planning and testing: Crack inspection route planning is carried out based on the established high-precision 3D real-scene model and 3D point cloud model, and the planned route is field tested; S3, arranges the drone hangar, performs automated inspection tasks, and collects inspection photos through drone inspections; S4, automatically uploads the photos after inspection to the data center through the network, uses open source modeling algorithms for automatic modeling, generates orthophotos, and numbers the images; S5, using a deep learning model to identify the orthophotos after inspection, automatically repairing the identified cracks, and extracting and outputting the contours of the cracks in the form of vectors from the repaired and identified results; S6, multi-period comparison and early warning of cracks: The current identification result is compared with the previous one. If the change exceeds the set threshold △E, an abnormal warning is issued; the comparison indicators include characteristic indicators of the extension length, average width and maximum width of the crack.

[0005] Preferably, the interpretation of the initial characteristic value of the crack in step S1 includes: The geological disaster interpretation is carried out based on the Southern CASS software. The established 3D real scene model is imported into the Southern CASS software, and the crack boundary is outlined in the 3D real scene model to form a vector file of the crack; The crack characteristic values ​​include the crack extension length L, average width , maximum width B max And the crack area A: ; ; Where L1 and L2 represent the lengths of the edges on both sides of the crack.

[0006] Preferably, in step S2, the inspection route planning and testing includes: Set the path planning parameters, including the inspection route flight altitude H and the ground sampling resolution GSD, and calculate as follows: ; ; Where W i is the image width in pixels; F is the real focal length in mm; H is the flight height in m; W s is the actual sensor width in mm; GSD stands for ground sampling resolution, which refers to the size of the range represented by each pixel during aerial photography.

[0007] Preferably, in the inspection route planning and testing of step S2, the route planning is performed using professional route planning software based on the established three-dimensional point cloud model; the route planning draws multiple lines along the direction of the crack in the three-dimensional point cloud model, and offsets Hm upward in the vertical direction; the shooting angle of the route planning camera is vertically downward, and the overlap rate of the side lines of the captured image is not less than 75%; after the route planning is completed, a collision detection is performed with the three-dimensional point cloud to ensure that the route does not intersect with the point cloud or the distance is not less than the set limit distance; after the route planning is completed, an on-site field investigation and flight test are carried out to ensure that the drone can fly normally.

[0008] Preferably, in step S3, arranging the drone hangar includes: Arrange a drone hangar around the crack, import the inspection route tested in step S2 into the hangar, set the inspection frequency, and use a drone equipped with a high-definition camera to carry out automated inspections; The layout of the drone hangar is at a distance from the crack: ; ; In the formula, is the maximum flight speed of the drone, in m / s; is the maximum cruising time of the UAV, in seconds; is the time required for drone inspection, in seconds; is the crack extension length, in m; Check the flight speed of the drone; The inspection frequency is determined based on the deformation rate and importance of the cracks, generally once every 1 to 2 weeks; The inspection drone carries an RTK module for positioning to ensure flight safety and subsequent modeling accuracy.

[0009] Preferably, the open source modeling algorithm in step S4 includes OpenDroneMap, and OpenDroneMap is used to perform aerial triangulation on the aerial images to generate point clouds, orthophotos and elevation models, and perform three-dimensional reconstruction to generate a three-dimensional model.

[0010] Preferably, in step S4, the orthophoto is in tif format, including actual coordinates, and the coordinate system selects CGCS2000 projection coordinates; the purpose is to facilitate rapid retrieval of the inspection results of each period and multi-period comparison. The numbering rule of the orthophoto number is geological disaster object-inspection element-inspection number-date, for example, pond deformation body-crack 1-2-20240205.

[0011] Preferably, in step S5, using a deep learning model to recognize the orthophoto after inspection includes: S501, constructing a data set based on historical inspection orthophotos, and marking cracks in the data set; dividing the constructed data set into a training set, a test set, and a validation set; S502, preprocessing the orthophoto in the data set and converting it into a grayscale image; S503, constructing a deformable body crack recognition network based on a four-level symmetric encoder-decoder architecture of Transformer; S504, adding a nonlinear non-activation network to the constructed four-stage symmetric encoder-decoder network structure; S505, add a mixed-scale feed-forward network to Transformer; S506, training the network structure to obtain a trained deformable body crack recognition network; S507, inputting the orthophoto obtained by the drone inspection into the trained deformable body crack recognition network, and outputting the crack recognition result.

[0012] Preferably, in step S5, when performing crack repair, the noise data in the image is repaired, and the noise data includes noise data such as branches or boulders in the image that are highly similar to the crack image and densely distributed; the crack repair is performed by the morphological closing operation pixel repair method in the OpenCV open source library to fill the small gaps in the foreground object, while the overall position and shape remain unchanged, and the morphological closing operation is used to repair part of the real crack data that is predicted incorrectly.

[0013] Preferably, in step S5, the contour of the crack is extracted and output in vector form, including performing boundary scanning on the raster image of the crack image to obtain upper and lower boundary lines respectively; the contour extraction is completed using the contour extraction function in the OpenCV open source library.

[0014] The beneficial effects of the present invention are as follows: 1. The present invention can realize fully automated inspection and monitoring of cracks in deformed bodies, reduce inspection operation risks, improve inspection efficiency, and make daily inspection of geological disasters in mountainous and canyon areas possible; 2. The present invention can realize automatic identification and early warning of cracks. For areas with a large number of cracks and heavy inspection tasks, it can greatly reduce the workload of manual inspection and improve the effect of geological disaster inspection.

[0015] 3. The present invention plans the inspection route of UAV through scientific methods, identifies cracks in the collected data through deep learning models, and performs repair and early warning judgment through software, which greatly improves the accuracy of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of a flow chart of the present invention; Figure 2 It is a schematic diagram of the crack structure in the present invention; Figure 3 The original image for performing initial characteristic value interpretation of cracks in the embodiment of the present invention; Figure 4 A vector image for performing initial eigenvalue interpretation of cracks in an embodiment of the present invention; Figure 5 Schematic diagram of the structure of the Transformer model in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] Embodiment 1: like Figure 1 As shown, a method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport comprises the following steps: S1, interpretation of initial crack eigenvalues: The deformable body is photographed by close-up aerial photography to establish a high-precision 3D real scene model and 3D point cloud model of the deformable body; Interpret geological hazards based on the model, determine the cracks that need to be inspected, and read the crack characteristic values; S2, inspection route planning and testing: Crack inspection route planning is carried out based on the established high-precision 3D real-scene model and 3D point cloud model, and the planned route is field tested; S3, arranges the drone hangar, performs automated inspection tasks, and collects inspection photos through drone inspections; S4, automatically uploads the photos after inspection to the data center through the network, uses open source modeling algorithms for automatic modeling, generates orthophotos, and numbers the images; S5, using a deep learning model to identify the orthophotos after inspection, automatically repairing the identified cracks, and extracting and outputting the contours of the cracks in the form of vectors from the repaired and identified results; S6, multi-period comparison and early warning of cracks: The current identification result is compared with the previous one. If the change exceeds the set threshold △E, an abnormal warning is issued; the comparison indicators include characteristic indicators of the extension length, average width and maximum width of the crack.

[0018] like Figure 2 As shown, the initial characteristic value interpretation of the fracture in step S1 includes: The geological disaster interpretation is carried out based on the Southern CASS software. The established 3D real scene model is imported into the Southern CASS software, and the crack boundary is outlined in the 3D real scene model to form a vector file of the crack; like Figure 3 and Figure 4As shown, the crack characteristic values ​​include the crack extension length L, the average width , maximum width B max And the crack area A: ; ; Where L1 and L2 represent the lengths of the edges on both sides of the crack.

[0019] Preferably, in step S2, the inspection route planning and testing includes: Set the path planning parameters, including the inspection route flight altitude H and the ground sampling resolution GSD, and calculate as follows: ; ; Where W i is the image width in pixels; F is the real focal length in mm; H is the flight height in m; W s is the actual sensor width in mm; GSD stands for ground sampling resolution, which refers to the size of the range represented by each pixel during aerial photography.

[0020] Preferably, in the inspection route planning and testing of step S2, the route planning is performed using professional route planning software based on the established three-dimensional point cloud model; the route planning draws multiple lines along the direction of the crack in the three-dimensional point cloud model, and offsets Hm upward in the vertical direction; the shooting angle of the route planning camera is vertically downward, and the overlap rate of the side lines of the captured image is not less than 75%; after the route planning is completed, a collision detection is performed with the three-dimensional point cloud to ensure that the route does not intersect with the point cloud or the distance is not less than the set limit distance; after the route planning is completed, an on-site field investigation and flight test are carried out to ensure that the drone can fly normally.

[0021] Preferably, in step S3, arranging the drone hangar includes: Arrange a drone hangar around the crack, import the inspection route tested in step S2 into the hangar, set the inspection frequency, and use a drone equipped with a high-definition camera to carry out automated inspections; The layout of the drone hangar is at a distance from the crack: ; ; In the formula, is the maximum flight speed of the drone, in m / s; is the maximum cruising time of the UAV, in seconds; is the time required for drone inspection, in seconds; is the crack extension length, in m; Check the flight speed of the drone; The inspection frequency is determined based on the deformation rate and importance of the cracks, generally once every 1 to 2 weeks; The inspection drone carries an RTK module for positioning to ensure flight safety and subsequent modeling accuracy.

[0022] Preferably, the open source modeling algorithm in step S4 includes OpenDroneMap, and OpenDroneMap is used to perform aerial triangulation on the aerial images to generate point clouds, orthophotos and elevation models, and perform three-dimensional reconstruction to generate a three-dimensional model.

[0023] Preferably, in step S4, the orthophoto is in tif format, including actual coordinates, and the coordinate system selects CGCS2000 projection coordinates; the purpose is to facilitate rapid retrieval of the inspection results of each period and multi-period comparison. The numbering rule of the orthophoto number is geological disaster object-inspection element-inspection number-date, for example, pond deformation body-crack 1-2-20240205.

[0024] Preferably, in step S5, using a deep learning model to recognize the orthophoto after inspection includes: S501, constructing a data set based on historical inspection orthophotos, and marking cracks in the data set; dividing the constructed data set into a training set, a test set, and a validation set; S502, preprocessing the orthophoto in the data set and converting it into a grayscale image; S503, constructing a deformable body crack recognition network based on a four-level symmetric encoder-decoder architecture of Transformer; S504, adding a nonlinear non-activation network to the constructed four-stage symmetric encoder-decoder network structure; S505, add a mixed-scale feed-forward network to Transformer; S506, training the network structure to obtain a trained deformable body crack recognition network; S507, inputting the orthophoto obtained by the drone inspection into the trained deformable body crack recognition network, and outputting the crack recognition result.

[0025] Preferably, in step S5, when performing crack repair, the noise data in the image is repaired, and the noise data includes noise data such as branches or boulders in the image that are highly similar to the crack image and densely distributed; the crack repair is performed by the morphological closing operation pixel repair method in the OpenCV open source library to fill the small gaps in the foreground object, while the overall position and shape remain unchanged, and the morphological closing operation is used to repair part of the real crack data that is predicted incorrectly.

[0026] Preferably, in step S5, the contour of the crack is extracted and output in vector form, including performing boundary scanning on the raster image of the crack image to obtain upper and lower boundary lines respectively; the contour extraction is completed using the contour extraction function in the OpenCV open source library.

[0027] Embodiment 2: The present invention provides a method for identifying an orthophoto after inspection using a deep learning model, comprising: S501, constructing a data set based on historical inspection orthophotos, and marking cracks in the data set; dividing the constructed data set into a training set, a test set, and a validation set; S502, preprocessing the orthophoto in the data set and converting it into a grayscale image; S503, constructing a deformable body crack recognition network based on a four-level symmetric encoder-decoder architecture of Transformer; S504, adding a nonlinear non-activation network to the constructed four-stage symmetric encoder-decoder network structure; S505, add a mixed-scale feed-forward network to Transformer; S506, training the network structure to obtain a trained deformable body crack recognition network; S507, inputting the orthophoto obtained by the drone inspection into the trained deformable body crack recognition network, and outputting the crack recognition result.

[0028] Furthermore, the preprocessing operations for the orthophotos in the dataset include: Adaptive median filtering is performed on the original images in the data set to remove noise; Then, the image is opened to eliminate possible interference points in filtering; Finally, the weighted average method is used to process the grayscale of the image.

[0029] like Figure 5 As shown, further, constructing a four-level symmetric encoder-decoder network structure based on Transformer includes: Use convolutional layers to obtain shallow features of the preprocessed image; Four sets of Transformers are used as encoders to obtain crack feature information; The low-resolution latent features are input through four sets of symmetrical Transformers as decoders, gradually restoring the image features to high resolution. The symmetrical encoder-decoder at each level, that is, the two symmetrical groups of Transformers, are connected using residual connections, and the number of Transformers in each level of the encoder-decoder increases from top to bottom.

[0030] Furthermore, the specific steps of adding a nonlinear non-activated network are: In nonlinear non-activated networks, a simplified channel attention mechanism is used to replace the multi-head attention mechanism with linear complexity in traditional Transformer; the module structure is further simplified by deleting or using a simple GLU variation structure to replace the attention mechanism and the activation function in the feedforward network; the specific formula is: ; ; ; ; Among them, Gate( ) and SimpleGate( ) represent the traditional GLU structure and the simplified GLU structure respectively, CA( ) and SCA( ) represent the traditional channel attention mechanism and the simplified channel attention mechanism, X and Y are feature maps of the same size, f and g is a linear Transformer; σ represents a nonlinear activation function; pool represents the global average pooling operation that aggregates spatial information into channels; W, W1, and W2 are fully connected layers.

[0031] Furthermore, the specific steps of adding a mixed-scale feedforward network are as follows: After layer normalization, 1×1 convolution is first used to expand the channel dimension in a certain ratio, and then it is input into two parallel branches; in the feature transformation process, 3×3 and 5×5 deep convolution methods are used to enhance the extraction of multi-scale local information; the specific formula is: ; ; ; ; ; ; in represents a 1×1 convolution, and It represents 3×3 and 5×5 depth convolution, and [ ] represents channel cascade.

[0032] Furthermore, the network structure is trained to obtain a trained deformable body crack recognition network including: Using the training set images in the data set, the images are input into the designed Transformer-based deformable body crack recognition network according to the number of images required for each iteration. After a predetermined number of rounds of training, the weights with the best effect on the validation set are saved to complete the deformable body crack recognition network training.

Claims

1. A method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport, characterized in that: The following steps are involved: S1, interpretation of initial crack eigenvalues: The deformable body is photographed by close-up aerial photography to establish a high-precision 3D real scene model and 3D point cloud model of the deformable body; Interpret geological hazards based on the model, determine the cracks that need to be inspected, and read the crack characteristic values; S2, inspection route planning and testing: Crack inspection route planning is carried out based on the established high-precision 3D real-scene model and 3D point cloud model, and the planned route is field tested; S3, arranges drone hangars, performs automated inspection tasks, and collects inspection photos through drone inspections; S4, automatically uploads the photos after inspection to the data center through the network, uses open source modeling algorithms for automatic modeling, generates orthophotos, and numbers the images; S5, using a deep learning model to identify the orthophotos after inspection, automatically repairing the identified cracks, and extracting and outputting the contours of the cracks in the form of vectors from the repaired and identified results; S6, multi-period comparison and early warning of cracks: The current identification result is compared with the previous one. If the change exceeds the set threshold △E, an abnormal warning is issued; the comparison indicators include characteristic indicators of the extension length, average width and maximum width of the crack.

2. The method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport according to claim 1 is characterized in that: The interpretation of the initial characteristic values ​​of the fracture in step S1 includes: The geological disaster interpretation is carried out based on the Southern CASS software. The established 3D real scene model is imported into the Southern CASS software, and the crack boundary is outlined in the 3D real scene model to form a vector file of the crack; The crack characteristic values ​​include the crack extension length L, the average width , maximum width B max And the crack area A: ； ； Where L1 and L2 represent the lengths of the edges on both sides of the crack.

3. The method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport according to claim 1 is characterized in that: In step S2, inspection route planning and testing includes: Set the path planning parameters, including the inspection route flight altitude H and the ground sampling resolution GSD, and calculate as follows: ； ； Where W i is the image width in pixels; F is the real focal length in mm; H is the flight height in m; W s is the actual sensor width in mm; GSD stands for ground sampling resolution, which refers to the size of the range represented by each pixel during aerial photography.

4. The method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport according to claim 3 is characterized in that: In the inspection route planning and testing of step S2, the route planning is carried out using professional route planning software based on the established three-dimensional point cloud model; the route planning draws multiple lines along the direction of the crack in the three-dimensional point cloud model, and offsets Hm upward in the vertical direction; the route planning camera shooting angle is vertically downward, and the overlap rate of the side lines of the captured image is not less than 75%; after the route planning is completed, a collision detection is performed with the three-dimensional point cloud to ensure that the route does not intersect with the point cloud or the distance is not less than the set limit distance; after the route planning is completed, an on-site field investigation and flight test are carried out to ensure that the UAV can fly normally.

5. The method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport according to claim 1 is characterized in that: In step S3, arranging the drone hangar includes: Arrange a drone hangar around the crack, import the inspection route tested in step S2 into the hangar, set the inspection frequency, and use a drone equipped with a high-definition camera to carry out automated inspections; The layout of the drone hangar is at a distance from the crack: ； ； In the formula, is the maximum flight speed of the drone, in m / s; is the maximum cruising time of the UAV, in seconds; is the time required for drone inspection, in seconds; is the crack extension length, in m; Check the flight speed of the drone; The inspection frequency is determined based on the deformation rate and importance of the cracks; The inspection drone carries an RTK module for positioning.

6. The method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport according to claim 1 is characterized in that: The open source modeling algorithm in step S4 includes OpenDroneMap, which performs aerial triangulation on the aerial images to generate point clouds, orthophotos, and elevation models, and performs three-dimensional reconstruction to generate a three-dimensional model.

7. The method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport according to claim 1 is characterized in that: In step S4, the orthophoto is in tif format, including actual coordinates, and the coordinate system selects CGCS2000 projection coordinates; the numbering rule of the orthophoto is geological disaster object-inspection element-inspection number-date.

8. The method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport according to claim 1 is characterized in that: In step S5, using a deep learning model to recognize the orthophoto after inspection includes: S1, construct a dataset based on historical inspection orthophotos and annotate the cracks in the dataset; divide the constructed dataset into training set, test set and validation set; S2, preprocessing the orthophotos in the data set and converting them into grayscale images; S3, construct a deformable body crack recognition network based on a four-level symmetric encoder-decoder architecture of Transformer; S4, adding a nonlinear non-activation network to the constructed four-level symmetric encoder-decoder network structure; S5, adding a mixed-scale feedforward network to the Transformer; S6, training the network structure to obtain a trained deformable body crack recognition network; S7, input the orthophoto obtained by the UAV inspection into the trained deformable body crack recognition network, and output the crack recognition result.

9. The method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport according to claim 1 is characterized in that: In step S5, when performing crack repair, the noise data in the image is repaired, and the noise data includes noise data such as branches or boulders in the image that are highly similar to the crack image and densely distributed; the crack repair is performed through the morphological closing operation pixel repair method in the OpenCV open source library to fill the small gaps in the foreground object, while the overall position and shape remain unchanged, and the morphological closing operation is used to repair some of the real crack data that were predicted incorrectly.

10. The method for automatic monitoring and early warning of cracks in a deformable body based on a drone airport according to claim 1 is characterized in that: In step S5, the contour of the crack is extracted and output in vector form, including performing boundary scanning on the raster image of the crack image to obtain upper and lower boundary lines respectively; the contour extraction is completed using the contour extraction function in the OpenCV open source library.

Images