Rockfill material gradation detection method and device based on unmanned aerial vehicle photogrammetry and instance segmentation

Abstract

The application discloses a rockfill material gradation detection method and device based on unmanned aerial vehicle aerial survey and instance segmentation, and comprises the following steps: establishing a conversion relationship between surface gradation and overall gradation by relying on local surface rockfill material images before density pit excavation and rockfill material screening results of the density pit; obtaining a rockfill dam filling bin surface image by using unmanned aerial vehicle close-range photogrammetry technology; generating an orthographic image of the filling bin surface; making an image dataset based on the orthographic image; training an instance segmentation model jointly constructed by YOLOv8 and Unet to obtain an instance segmentation model for rockfill material segmentation; obtaining an orthographic image of a rockfill dam filling bin surface to be predicted and cutting it into small blocks, using the instance segmentation model to perform prediction, realizing pixel-level instance segmentation and morphology extraction; splicing the predicted blocks, performing morphology quantitative analysis on the rockfill material instances obtained by prediction, and obtaining overall bin surface rockfill gradation through gradation conversion. The application realizes accurate and efficient gradation detection of overall bin surface rockfill.

Description

Technical Field

[0001] This invention belongs to the field of water conservancy engineering technology, specifically relating to a method and device for detecting the gradation of rockfill based on UAV aerial surveying and instance segmentation. Background Technology

[0002] Rockfill dams play a vital role in water conservancy and hydropower resource development, serving as a major dam type. The construction and safety stability of rockfill dams are closely related to the gradation curve of the dam-building materials used. The gradation curve affects the degree of particle filling within the rockfill, thus influencing its density and deformation modulus. Poor gradation results in inadequate particle filling, leading to insufficient density and modulus to meet design requirements. Under high head gradient conditions, this can cause seepage failure and other problems, threatening the safety and stability of the rockfill dam. Therefore, effectively controlling the gradation of the rockfill during the construction phase is a significant challenge in rockfill dam engineering, directly impacting the project's quality and safety.

[0003] Currently, sieve analysis is a commonly used method in engineering practice to determine the gradation of rockfill. The sieve analysis method involves using sieves of different mesh sizes to sieve the rockfill, calculating the percentage of residual rockfill on each mesh size, and plotting the corresponding gradation curve. While this method is technically mature, it has some significant limitations. First, because gradation testing is only conducted on a small number of test points, it lacks representativeness and cannot reflect the overall gradation of the fill surface. Second, manual sampling and sieving are time-consuming and labor-intensive, potentially impacting the overall dam construction progress. Furthermore, the accuracy and reliability of gradation results obtained through manual sieving heavily depend on the skill level of the testing personnel.

[0004] Digital image processing technology provides a reliable means for the detection of gradation in riprap, and some scholars have conducted related research. Traditional digital image processing techniques, when applied to riprap segmentation, exhibit insufficient accuracy, with significant errors in image segmentation and gradation calculation. With the development of deep learning algorithms and the improvement of computer performance, new progress has been made in methods for extracting the contours of target objects in images. Some instance segmentation algorithms can achieve pixel-level segmentation of object contours in images, providing technical support for riprap gradation detection based on UAV aerial surveying. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of current methods for detecting the gradation of rockfill material in rockfill dam filling surfaces by providing a method and apparatus for detecting the gradation of rockfill material based on UAV aerial surveying and instance segmentation. This method utilizes UAV close-range photogrammetry technology, maneuvering the UAV to a certain height above the rockfill dam filling surface to capture photos of the surface with a certain overlap rate. Furthermore, Agisoft Metashape aerial surveying image processing software is used to generate an orthophoto of the entire filling surface. Then, a deep learning instance segmentation algorithm is used to segment the rockfill material particles and extract their particle size, achieving accurate and efficient gradation detection of the rockfill material across the entire filling surface of the rockfill dam.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A method for detecting the gradation of rockfill based on UAV aerial surveying and instance segmentation includes the following steps:

[0008] S1: Based on the local surface rockfill images before the density pit excavation and the rockfill screening results of the density pit, establish the conversion relationship between surface gradation and overall gradation;

[0009] S2: Based on the conversion relationship between the surface gradation and the overall gradation, use UAV close-range photogrammetry technology to obtain images of the rockfill dam filling surface;

[0010] S3: Based on the image of the rockfill dam filling surface, generate an orthophoto of the filling surface;

[0011] S4: The orthophoto image is cropped into a series of tiles using a sliding window, and the tiles are labeled with instances using SAM point selection auxiliary annotation to create a riprap image dataset.

[0012] S5: Based on the riprap image dataset, train an instance segmentation model jointly constructed by YOLOv8 and Unet to obtain an instance segmentation model for riprap segmentation;

[0013] S6: Obtain the orthophoto of the fill surface of the rockfill dam to be predicted and crop it into small tiles. Use the trained instance segmentation model to make predictions, and achieve pixel-level instance segmentation and morphology extraction.

[0014] S7: Piece together the predicted blocks, perform morphological quantitative analysis on the predicted rockfill instances, and obtain the full-area rockfill gradation through gradation conversion.

[0015] Preferably, in step S1, the method for establishing the conversion relationship between surface gradation and overall gradation based on the local surface rockfill image before the density pit excavation and the rockfill screening results of the density pit includes:

[0016] Photographs were taken of local rockfill samples above several density pits before excavation, and the rockfill gradation on the surface of the density pits was extracted using image recognition algorithms.

[0017] The sieving results of the rockfill after the density pit excavation were used as the overall rockfill gradation of the density pit;

[0018] Based on the gradation of the riprap on the surface of the density pit and the overall riprap gradation of the density pit, the Weibull-Rosin-Rammler particle size distribution equation is introduced. The correlation model of the characteristic equation parameters of the overall riprap and the characteristic equation parameters of the surface riprap is established using the multiple linear regression method, so as to obtain the conversion relationship between the surface riprap gradation and the overall riprap gradation.

[0019] Preferably, in step S2, the method for acquiring images of the rockfill dam fill surface using close-range photogrammetry technology based on the conversion relationship between the surface gradation and the overall gradation includes:

[0020] Operate a drone to fly and take pictures above the entire rockfill dam filling surface; the shooting time is before static rolling; the rockfill material is evenly and quantitatively sprayed with water before shooting; more than 4 phase control points are set up around the perimeter of the dam surface; the drone takes off from the horizontal plane where the dam surface is located, and the flight plane is at a fixed height within the range of 3-6 meters. The angle between the normal of the lens and the normal of the dam surface should not exceed 5°; when the drone moves in a zigzag pattern within the flight plane, the overlap rate of adjacent pictures along the flight direction should not be less than 60%, and the overlap rate of adjacent pictures perpendicular to the flight direction should not be less than 40%.

[0021] Preferably, in step S4, the method for cropping the orthophoto image into a series of tiles using a sliding window and using SAM point selection auxiliary annotation to perform instance annotation on the tiles to create a riprap image dataset includes:

[0022] The orthophoto of the filling silo surface is cropped using a sliding window with overlapping pixels. The size of the sliding window is m×n, which is determined according to the requirements of model training. A certain proportion of the regions overlap between the sliding windows is set, and the proportion is not less than 25%.

[0023] The Vit-H model of the Segment Anything Model is used for point-selection-assisted interactive annotation to obtain a polygonal mask of the image;

[0024] Based on the polygonal mask of the image, the dataset is expanded by translation, rotation, cropping, scaling, flipping, HSV transformation, and Mosaic data augmentation techniques to obtain a standard dataset for model training.

[0025] Preferably, in step S5, the method for training an instance segmentation model jointly constructed by YOLOv8 and Unet based on the riprap image dataset to obtain an instance segmentation model for riprap segmentation includes:

[0026] The standard dataset is divided into a training set and a validation set. A deep learning instance segmentation model is trained in the training set and validated in the validation set to obtain an instance segmentation model for riprap segmentation.

[0027] The deep learning instance segmentation model is a two-stage instance segmentation model, which is jointly constructed by the YOLOv8 object detection branch and the Unet semantic segmentation branch. YOLOv8 can identify and locate the rock pile instances and generate rectangular detection boxes. The detection boxes are then input into the Unet branch, and Unet can generate pixel-level masks for the rock pile instances in each detection box.

[0028] Preferably, in step S6, the method of acquiring an orthophoto of the rockfill dam fill surface to be predicted and cropping it into small patches, and using the trained instance segmentation model for prediction to achieve pixel-level instance segmentation and morphology extraction includes:

[0029] The sample patches to be detected are input in batches into the trained instance segmentation model, and the detection boxes and masks of the pile instances are predicted in sequence. The predicted patches are then stitched back into the original image using a sliding window. Overlapping detection boxes in overlapping areas are removed using the non-maximum suppression algorithm (NMS), thus achieving pixel-level instance segmentation and morphology extraction.

[0030] Preferably, in step S7, the method for stitching together the predicted map blocks, performing morphological quantitative analysis on the predicted rockfill instances, and obtaining the full-area rockfill gradation through gradation conversion includes:

[0031] For the extracted contour features of the riprap particles, the predicted pixel size is converted into the actual size through size calibration;

[0032] Calculate the equivalent particle size and equivalent ellipsoidal volume of the particles based on the shape of the mask;

[0033] The mass percentage of each particle size group is replaced by the volume percentage. The volume percentage of rockfill particles smaller than the standard value of each gradation particle size is calculated on the entire surface of the rockfill to obtain the gradation curve of the surface rockfill.

[0034] The gradation curve was corrected using the Weibull-Rosin-Rammler particle size characteristic equation, and the overall gradation of the full-surface rockfill was finally obtained.

[0035] The formulas for calculating the equivalent particle size and equivalent ellipsoidal volume of the particles, based on the shape of the mask, are as follows:

[0036]

[0037]

[0038] Where a and b are the major and minor axes of the equivalent ellipse, respectively, and A is the area of ​​the equivalent ellipse;

[0039] The Weibull-Rosin-Rammler particle size distribution equation is:

[0040]

[0041] Where R is the percentage of particles smaller than d by mass; d is the standard value of particle size; e is the base of the natural logarithm; b is a parameter related to particle size; and n is a parameter related to material properties.

[0042] The present invention also provides a rockfill gradation detection device based on UAV aerial survey and instance segmentation, comprising: a processor and a memory;

[0043] The processor is used to operate according to instructions to implement any of the above-described methods for detecting the gradation of rockfill based on UAV aerial surveying and instance segmentation.

[0044] The memory stores instructions that can be executed by the processor to implement any of the methods for detecting the gradation of rockfill based on UAV aerial surveying and instance segmentation.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] This invention utilizes UAV close-range photogrammetry technology to quickly acquire high-quality images of the entire rockfill dam surface, providing a reliable basis for the study of the gradation of the rockfill surface.

[0047] This invention uses Agisoft Metashape aerial survey image processing software to generate an orthophoto of the entire cabin surface, and correlates the coordinates of the orthophoto image with the real world coordinates.

[0048] The deep learning instance segmentation model proposed in this invention has high accuracy and improves detection efficiency while maintaining detection accuracy compared to traditional instance segmentation models.

[0049] This invention introduces the Weibull-Rosin-Rammler particle size characteristic equation, establishes a conversion model between the overall characteristic equation parameters and the surface characteristic equation parameters, and obtains the corrected full-surface rockfill gradation curve.

[0050] This invention can calculate the gradation of rockfill across the entire dam surface without damage or loss, enabling gradation testing throughout the construction process. This provides a reliable basis for monitoring the construction quality of rockfill dams and ensures the safety of dam construction and operation. Attached Figure Description

[0051] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0053] Figure 2 This is a partial image of the rockfill material taken above the density pit according to the present invention;

[0054] Figure 3 This is a schematic diagram of the UAV aerial survey of the present invention;

[0055] Figure 4 This invention uses the Agisoft Metashape aerial survey image processing software to generate orthophotos of the warehouse surface.

[0056] Figure 5 This is a schematic diagram of the instance segmentation result of the present invention;

[0057] Figure 6 This is a schematic diagram of the gradation curve of the rockfill material according to the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Example 1

[0061] like Figures 1-6 As shown, this invention provides a method for detecting the gradation of riprap based on UAV aerial surveying and instance segmentation, comprising the following steps:

[0062] S1: Based on the local surface rockfill images before the density pit excavation and the rockfill screening results of the density pit, establish the conversion relationship between surface gradation and overall gradation;

[0063] S2: Based on the conversion relationship between surface gradation and overall gradation, use UAV close-range photogrammetry technology to obtain images of the fill surface of the rockfill dam;

[0064] S3: Generate an orthophoto of the fill surface based on the image of the rockfill dam fill surface;

[0065] S4: Crop the orthophoto into a series of tiles using a sliding window, and use SAM point selection auxiliary annotation to perform instance annotation on the tiles to create a riprap image dataset;

[0066] S5: Based on the riprap image dataset, train the instance segmentation model jointly constructed by YOLOv8 and Unet to obtain the instance segmentation model for riprap segmentation;

[0067] S6: Obtain the orthophoto of the fill surface of the rockfill dam to be predicted and crop it into small tiles. Use the trained instance segmentation model to make predictions, and achieve pixel-level instance segmentation and morphology extraction.

[0068] S7: Piece together the predicted blocks, perform morphological quantitative analysis on the predicted rockfill instances, and obtain the full-area rockfill gradation through gradation conversion.

[0069] In this embodiment, S1: Local images of the riprap above multiple pre-excavation density pits are acquired. The diameter D of each density pit is approximately 1.6 meters. The riprap gradation on the surface of the density pits is obtained using traditional image recognition methods. The riprap after excavation is manually sieved to obtain the overall riprap gradation of the density pits. The conversion relationship between the riprap gradation on the density pit surface and the overall riprap gradation is established using the Weibull-Rosin-Rammler particle size distribution equation. The form of this equation is:

[0070]

[0071] Where R is the percentage of particles smaller than d by mass; d is the standard value of particle size; e is the base of the natural logarithm; b is a parameter related to particle size; and n is a parameter related to material properties.

[0072] First, based on the gradation data of the local surface riprap in the density pit, the parameters b and n are fitted using the nonlinear least squares algorithm Levenberg-Marquardt. The fitting effect can be reflected by RMSE and R-squared. Then, a correlation model is established using multiple linear regression to represent the characteristic equation parameters (b' and n') of the overall riprap in the density pit and the characteristic equation parameters (b and n) of the surface riprap, thus obtaining a transformation model between the overall characteristic equation parameters and the surface characteristic equation parameters.

[0073] In this embodiment, S2: Data acquisition. For large scenes such as the entire surface of a rockfill dam, close-up shooting cannot capture the entire scene at once. Although long-distance shooting can capture the entire scene, the clarity cannot meet the requirements for subsequent instance segmentation. By using UAV close-range photogrammetry technology, multiple images can be stitched together into a large image, thereby solving the above problems.

[0074] The shooting method involves uniformly and quantitatively spraying water onto the silo surface after filling and laying the material. This watering washes away dust from the silo surface, improving image quality. At least four phased array sensors are positioned around the silo to be photographed. The DJI Mavic 2 Zoom drone is then flown to a height of 3 meters above the silo surface. The lens is adjusted to point vertically downwards, ensuring the angle between the lens's normal and the silo surface's normal does not exceed 5°. A 48mm focal length is selected for image capture. During shooting, the drone's flight path must be controlled to ensure that adjacent images along the flight direction have an overlap of at least 60% and adjacent images perpendicular to the flight direction have an overlap of at least 40%.

[0075] The DJI Mavic 2 Zoom drone has a camera resolution of 4000×3000 pixels, with an aspect ratio of 4:3. At a focal length of 48mm and a flight altitude of 3m, a single image captures a rectangular area approximately 4.5m long and 3.375m wide, with a pixel size of approximately 1.1mm. In this example, 350 images were captured to generate an orthophoto of the entire cabin surface.

[0076] In this embodiment, S3: Generate orthophotos by importing the acquired images into a computer with Agisoft Metashape software installed, and using Agisoft Metashape aerial survey image processing software to generate orthophotos.

[0077] Agisoft Metashape is a professional aerial survey image processing software used to process image data from drone aerial photography, satellite remote sensing, and aerial photography. After opening the image to be processed in Agisoft Metashape, create a new batch processing workflow. The entire workflow includes aligning photos, generating dense point clouds, creating digital elevation models, and generating orthophotos. Once the workflow starts, the computer will automatically complete each step, and the orthophoto will be saved to a custom path upon completion.

[0078] The images captured by the drone record the latitude, longitude, and altitude (i.e., world coordinates) of the shooting point. When generating an orthophoto, the world coordinates of any location in the image are transformed into image coordinates in the orthophoto, thus obtaining the correspondence between the image coordinates in the orthophoto and the corresponding world coordinates. The specific transformation process is as follows:

[0079]

[0080] Where (X,Y,Z) are world coordinates, and x, y, z are transition values.

[0081]

[0082] Where x' and y' are transition values, K1, K2, K3, and K4 are radial distortion coefficients, and P1, P2, P3, and P4 are tangential distortion coefficients.

[0083]

[0084] Where (u,v) are the coordinates of the projection point in the orthophoto image coordinate system (in pixels); w and h are the width and height of the image pixels; c y c x B1 is the principal point offset; B2 are the orthogonality and obliqueness coefficients; f is the camera focal length. After the above transformation process, the world coordinates of any point on the orthophoto can be displayed.

[0085] In this embodiment, S4: Create an image dataset. The generated orthophoto is much larger than the size of a regular image. Due to limitations in computer capacity and computing power, deep learning model training cannot be based on such a large image. Therefore, the image needs to be cropped into smaller patches suitable for model training. This example uses a sliding window with overlapping pixels to crop the orthophoto. The sliding window size is set; it can be m×n, mainly determined by the model training requirements. The sliding window is moved zigzag across the orthophoto from left to right and from top to bottom. The areas before and after the sliding window movement overlap by a set percentage, which can be set according to needs, but should not be less than 25%. This effectively avoids the problem of checkerboard-style cropping splitting the edge pile particles in half, making them unsuitable as single instances for model training. In this embodiment, the sliding window size is 512×512, and the overlap rate between adjacent sliding windows is 50%. Instance annotation is performed on the cropped patches, i.e., closed polygons are drawn along the contour points of each pile particle in the patch. This example uses the Vit-H model of the Segment Anything Model (SAM) for point selection-assisted annotation. SAM can use interactive points and boxes for annotation, which is more accurate and faster than manually drawing polygons. For some pile particles that SAM cannot accurately annotate, the traditional method of manually drawing polygons is used. After the mask is annotated, the algorithm generates rectangular target detection boxes for the pile particles. After all the pile particles in the map are annotated, the dataset is expanded by data augmentation techniques, thus constructing the standard dataset for model training.

[0086] In this embodiment, S5: Model training. The standard dataset created in step S4 is divided into a training set and a validation set. The training set is used to train the deep learning model, and the validation set is used for validation. The constructed deep learning model is an instance segmentation model, mainly including two branches: YOLOv8 and Unet. YOLOv8, as an object detection model, obtains the bounding box for each pile of stones in the image. Unet, as a semantic segmentation model, generates pixel-level masks for the pile instances in the object detection boxes, separating the contour of each pile of stones from the background. YOLOv8, as a recent deep learning-based object detection algorithm, has advantages such as high speed, high accuracy, and ease of use. Its detection speed is more than twice that of traditional object detection algorithms such as Mask R-CNN and SSD, while its detection accuracy is comparable to these algorithms. YOLOv8 adopts the algorithm framework of YOLOv5, using CSP-Darknet53 as the backbone. The neck section removes two Conv layers, and the C3 module is replaced by the C2f module. The head includes three detectors using a decoupled header. Unet, as a semantic segmentation algorithm, has a typical Encoder-Decoder structure, fusing low-level and high-level semantic information through skip connections, resulting in excellent segmentation performance.

[0087] The original YOLOv8 model writes the detection results directly to disk as text files, including the detected bounding boxes and the corresponding images. In the second stage of semantic segmentation using Unet, the model needs to read these images and bounding boxes from the disk, which requires a lot of I / O time. Therefore, this example directly embeds Unet into the YOLOv8 architecture.

[0088] The instance segmentation model in this example is developed in Ubuntu 20.04LTS, and the model is built on the PyTorch framework. The GPU is an NVIDIA GeForce RTX 3090 with 24GB of video memory.

[0089] In this embodiment, S6: Model prediction, acquiring images of the piled stone material on the silo surface that needs to be graded, generating orthophotos according to steps S3 and S4 and cropping them with a sliding window containing overlapping pixels, the model prediction predicts the mask of the piled stone particles based on these cropped small patches; using the sliding window to stitch the small patches back to the original image, there will be overlapping areas between the small patches during patch stitching, resulting in overlapping detection boxes appearing in the overlapping areas in the model prediction, which can be removed by the non-maximum suppression (NMS) algorithm.

[0090] In this embodiment, S7: Grading calculation. The contour features of the riprap particles in the image can be obtained from the results of step S6. After size calibration, the predicted pixel size is converted into the actual size. Based on this, morphological quantification analysis is performed on the riprap particles to further convert them into gradation data. This example uses the equivalent particle size empirical formula proposed by Kemeny et al. to calculate the equivalent particle size based on the particle contour information, and then calculates the equivalent ellipsoidal volume of the particles accordingly. The formulas for equivalent particle size and equivalent ellipsoidal volume are as follows:

[0091]

[0092]

[0093] Where a and b are the major and minor axes of the equivalent ellipse, respectively, and A is the area of ​​the equivalent ellipse.

[0094] Assuming that the same batch of riprap particles has a fixed density, the mass percentage of particles in each size group can be replaced by the volume percentage. This allows for the calculation of the mass percentage of riprap particles smaller than the standard value for each gradation size across the entire silo surface. Due to resolution limitations, some fine particles cannot be accurately identified, causing errors in the mass percentage calculation. Therefore, the number of fine particles can be estimated based on the surface area occupied by these particles in the image, assuming a specific size distribution. This allows the gradation curve of the riprap surface across the entire silo to be derived.

[0095] The gradation curve of the surface rockfill in the entire bin is corrected by using the Weibull-Rosin-Rammler particle size characteristic equation established based on multiple density pits in step S1, so as to obtain the overall gradation curve of the rockfill in the entire bin.

[0096] Example 2

[0097] This invention also provides a riprap gradation detection device based on UAV aerial surveying and instance segmentation, comprising:

[0098] At least one processor operates according to the instructions to implement the method for detecting the gradation of rockfill based on UAV aerial survey and instance segmentation;

[0099] The memory stores instructions that can be executed by the processor to implement the method for detecting the gradation of rockfill based on UAV aerial surveying and instance segmentation.

[0100] This invention proposes an efficient and accurate solution for full-surface gradation detection of rockfill dams by using UAV aerial surveying and deep learning instance segmentation technology, filling the gap in rapid detection of full-surface rockfill gradation. This method can be used to monitor the entire construction process, provide reliable gradation data support, and ensure the construction and operation safety of rockfill dams.

[0101] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for detecting the gradation of riprap based on UAV aerial surveying and instance segmentation, characterized in that, Includes the following steps: S1: Based on the local surface rockfill images before the density pit excavation and the rockfill screening results of the density pit, establish the conversion relationship between surface gradation and overall gradation; S2: Based on the conversion relationship between the surface gradation and the overall gradation, use UAV close-range photogrammetry technology to obtain images of the rockfill dam filling surface; S3: Based on the image of the rockfill dam filling surface, generate an orthophoto of the filling surface; S4: The orthophoto image is cropped into a series of tiles using a sliding window, and the tiles are labeled with instances using SAM point selection auxiliary annotation to create a riprap image dataset. S5: Based on the riprap image dataset, train an instance segmentation model jointly constructed by YOLOv8 and Unet to obtain an instance segmentation model for riprap segmentation; S6: Obtain the orthophoto of the fill surface of the rockfill dam to be predicted and crop it into small tiles. Use the trained instance segmentation model to make predictions, and achieve pixel-level instance segmentation and morphology extraction. S7: Piece together the predicted blocks, perform morphological quantitative analysis on the predicted rockfill instances, and obtain the full-area rockfill gradation through gradation conversion.

2. The method for detecting the gradation of riprap based on UAV aerial surveying and instance segmentation according to claim 1, characterized in that, In step S1, the method for establishing the conversion relationship between surface gradation and overall gradation based on the local surface rockfill image before the density pit excavation and the rockfill screening results of the density pit includes: Photographs were taken of local rockfill samples above several density pits before excavation, and the rockfill gradation on the surface of the density pits was extracted using image recognition algorithms. The sieving results of the rockfill after the density pit excavation were used as the overall rockfill gradation of the density pit; Based on the gradation of the riprap on the surface of the density pit and the overall riprap gradation of the density pit, the Weibull-Rosin-Rammler particle size distribution equation is introduced. The correlation model of the characteristic equation parameters of the overall riprap and the characteristic equation parameters of the surface riprap is established using the multiple linear regression method, so as to obtain the conversion relationship between the surface riprap gradation and the overall riprap gradation.

3. The method for detecting the gradation of riprap based on UAV aerial surveying and instance segmentation according to claim 1, characterized in that, In step S2, the method for acquiring images of the fill surface of a rockfill dam using close-range photogrammetry technology based on the conversion relationship between surface gradation and overall gradation includes: Operate a drone to fly and take pictures above the entire rockfill dam filling surface; the shooting time is before static rolling; the rockfill material is evenly and quantitatively sprayed with water before shooting; more than 4 phase control points are set up around the perimeter of the dam surface; the drone takes off from the horizontal plane where the dam surface is located, and the flight plane is at a fixed height within the range of 3-6 meters. The angle between the normal of the lens and the normal of the dam surface should not exceed 5°; when the drone moves in a zigzag pattern within the flight plane, the overlap rate of adjacent pictures along the flight direction should not be less than 60%, and the overlap rate of adjacent pictures perpendicular to the flight direction should not be less than 40%.

4. The method for detecting the gradation of riprap based on UAV aerial surveying and instance segmentation according to claim 1, characterized in that, In step S4, the orthophoto image is cropped into a series of tiles using a sliding window, and the tiles are labeled using SAM point selection auxiliary annotation to create a riprap image dataset. The orthophoto of the filling silo surface is cropped using a sliding window with overlapping pixels. The size of the sliding window is m×n, which is determined according to the requirements of model training. A certain proportion of the regions overlap between the sliding windows is set, and the proportion is not less than 25%. The Vit-H model of the Segment Anything Model is used for point-selection-assisted interactive annotation to obtain a polygonal mask of the image; Based on the polygonal mask of the image, the dataset is expanded by translation, rotation, cropping, scaling, flipping, HSV transformation, and Mosaic data augmentation techniques to obtain a standard dataset for model training.

5. The method for detecting the gradation of riprap based on UAV aerial surveying and instance segmentation according to claim 4, characterized in that, In step S5, the method for training an instance segmentation model jointly constructed by YOLOv8 and Unet based on the riprap image dataset to obtain an instance segmentation model for riprap segmentation includes: The standard dataset is divided into a training set and a validation set. A deep learning instance segmentation model is trained in the training set and validated in the validation set to obtain an instance segmentation model for riprap segmentation. The deep learning instance segmentation model is a two-stage instance segmentation model, which is jointly constructed by the YOLOv8 object detection branch and the Unet semantic segmentation branch. YOLOv8 can identify and locate the rock pile instances and generate rectangular detection boxes. The detection boxes are then input into the Unet branch, and Unet can generate pixel-level masks for the rock pile instances in each detection box.

6. The method for detecting the gradation of riprap based on UAV aerial surveying and instance segmentation according to claim 1, characterized in that, In step S6, the method of acquiring an orthophoto of the fill surface of the rockfill dam to be predicted and cropping it into small patches, and using the trained instance segmentation model for prediction, to achieve pixel-level instance segmentation and morphology extraction, includes: The sample patches to be detected are input in batches into the trained instance segmentation model, and the detection boxes and masks of the pile instances are predicted in sequence. The predicted patches are then stitched back into the original image using a sliding window. Overlapping detection boxes in overlapping areas are removed using the non-maximum suppression algorithm (NMS), thus achieving pixel-level instance segmentation and morphology extraction.

7. The method for detecting the gradation of riprap based on UAV aerial surveying and instance segmentation according to claim 6, characterized in that, In step S7, the method of stitching together the predicted map blocks, performing morphological quantitative analysis on the predicted rockfill instances, and obtaining the full-surface rockfill gradation through gradation conversion includes: For the extracted contour features of the riprap particles, the predicted pixel size is converted into the actual size through size calibration; Calculate the equivalent particle size and equivalent ellipsoidal volume of the particles based on the shape of the mask; The mass percentage of each particle size group is replaced by the volume percentage. The volume percentage of rockfill particles smaller than the standard value of each gradation particle size is calculated on the entire surface of the rockfill to obtain the gradation curve of the surface rockfill. The gradation curve was corrected using the Weibull-Rosin-Rammler particle size characteristic equation, and the overall gradation of the full-surface rockfill was finally obtained. The formulas for calculating the equivalent particle size and equivalent ellipsoidal volume of the particles, based on the shape of the mask, are as follows: Where a and b are the major and minor axes of the equivalent ellipse, respectively, and A is the area of ​​the equivalent ellipse; The Weibull-Rosin-Rammler particle size distribution equation is: R / 100=1-e -(d / b)n Where R is the percentage of particles smaller than d by mass; d is the standard value of particle size; e is the base of the natural logarithm; b is a parameter related to particle size; and n is a parameter related to material properties.

8. A rockfill gradation detection device based on UAV aerial surveying and instance segmentation, characterized in that, include: Processor and memory; The processor is used to operate according to instructions to implement the rockfill gradation detection method based on UAV aerial survey and instance segmentation as described in any one of claims 1-7; The memory stores instructions that can be executed by the processor to implement the rockfill gradation detection method based on UAV aerial survey and instance segmentation as described in any one of claims 1-7.

Images