Rockfill dam surface particle size distribution detection method and system based on gnss-vision fusion

Abstract

This invention discloses a method and system for detecting particle size distribution on the surface of rockfill dams based on GNSS-visual fusion, belonging to the field of hydraulic engineering construction quality inspection technology. The method acquires images of the rockfill dam surface and corresponding spatial coordinate information; for a single image, a deep learning instance segmentation model is used to identify particle contours and calculate equivalent particle sizes; adjacent image sequences are registered to match and fuse rockfill particles in theoretically overlapping areas, eliminating redundant data; through coordinate system transformation, the rockfill particle contours and equivalent particle sizes are mapped to a unified global coordinate system, generating a particle spatial distribution dataset covering the entire rockfill dam surface; finally, the overall particle size distribution curve and the spatial distribution of rockfill particles are obtained. This invention overcomes the limitation of traditional single-image detection range, achieving continuous, efficient, and high-precision detection of particle size distribution on a large surface area of ​​rockfill dams, providing a reliable technical means for monitoring and managing the construction quality of rockfill dams.

Description

Technical Field

[0001] This invention relates to the field of water conservancy engineering construction quality inspection technology, and in particular to a method and system for detecting particle size distribution on the surface of rockfill dams based on GNSS-visual fusion. Background Technology

[0002] Rockfill dams are a common type of civil engineering structure, and their construction quality directly affects the stability and impermeability of the dam body. The particle size distribution of the rockfill material is one of the core control indicators for evaluating the construction quality of rockfill dams, directly affecting key mechanical properties such as the dam's density, permeability, and shear strength. However, traditional testing methods mainly rely on manual sieve analysis or fixed-point sampling photogrammetry, which are inefficient, lack precision, have poor representativeness, and cannot achieve full coverage of the dam surface.

[0003] In recent years, with the development of computer vision and positioning technologies, image-based particle gradation analysis methods have been gradually applied to the field of engineering inspection. However, single-vision techniques are limited by local observation range and it is difficult to establish the correlation between particle spatial location and gradation distribution. Although traditional GNSS (Global Navigation Satellite System) positioning technology can provide global coordinate information, it cannot directly obtain particle morphological characteristics.

[0004] Therefore, how to overcome the limitations of a single technology and achieve continuous, efficient, and high-precision detection of the surface particle size distribution of rockfill dams over a wide range has become a key issue that urgently needs to be addressed in the intelligent management and control of construction quality in water conservancy projects. Summary of the Invention

[0005] To address the limitations of traditional single-image analysis, large cumulative errors in large-scale image stitching, repeated particle counting, and lack of absolute spatial information, this invention provides a GNSS-visual fusion-based method and system for detecting particle size distribution on the surface of rockfill dams. This method enables continuous, efficient, and high-precision detection of particle size distribution on a large scale of rockfill dam surfaces. Specifically, it is achieved through the following technologies.

[0006] This invention provides a method for detecting particle size distribution on the surface of rockfill dams based on GNSS-visual fusion, comprising the following steps:

[0007] Collect sequential images of the rockfill dam surface and its corresponding spatial coordinate information;

[0008] The optimal instance segmentation model is obtained through training. The optimal instance segmentation model is used to identify the contour of the rock pile particles in each of the sequence images and to calculate the equivalent particle size of each rock pile particle.

[0009] The initial positional relationship between two adjacent image sequences is determined using the spatial coordinate information of the two sequences. The theoretical overlapping area of ​​the images is determined using the initial positional relationship. Reliable edge feature matching point pairs of the pile particles are extracted and retained to complete the spatial registration of two adjacent image sequences.

[0010] By utilizing the equivalent particle size of each rockfill particle and the spatial registration relationship of two adjacent sequence images, the same rockfill particles that are repeatedly identified within the theoretical overlapping area of ​​the images are identified and fused, and redundant rockfill particle data are eliminated.

[0011] Using the spatial registration relationship between two adjacent image sequences and the fusion result of rockfill particles, the rockfill particle contour information and equivalent particle size retained in each image sequence are mapped to a unified global coordinate system, and a dataset of rockfill particle spatial distribution and particle size covering the entire surface of the rockfill dam is generated.

[0012] Based on the spatial distribution and particle size dataset of the rockfill particles, statistical analysis is used to generate an overall particle size distribution curve that reflects the particle size distribution of all rockfill particles on the surface of the entire rockfill dam.

[0013] Furthermore, sequential images of the rockfill dam surface and corresponding spatial coordinate information are acquired using spatial positioning equipment and image acquisition equipment; the spatial positioning equipment is a GNSS receiver that supports RTK or PPK.

[0014] Furthermore, the method for training and obtaining the optimal instance segmentation model is as follows: polygonal annotation is performed on the contours of the rubble particles in the sequence images to obtain a polygonal mask of the rubble particles;

[0015] Using the polygonal mask of the boulders (i.e., the sequence of images collected on-site and annotated with polygons) as the basic dataset, the basic dataset is enhanced through preprocessing operations such as parallelization, rotation, cropping, scaling, and flipping to obtain a standard dataset, which is then divided into a training set and a validation set.

[0016] A basic instance segmentation model based on deep learning is constructed. The training set is input into the basic instance segmentation model for iterative training. The total loss function is calculated, and the weights are updated until the accuracy requirement is met, thereby obtaining the optimal instance segmentation model.

[0017] Furthermore, the total loss function includes classification loss, bounding box regression loss, and mask prediction loss.

[0018] Furthermore, the method for identifying the contours of the rockfill particles in each of the sequence images using the optimal instance segmentation model, and calculating the equivalent particle size of each rockfill particle, is as follows:

[0019] The sequence of images to be identified, collected on-site, is input into the optimal instance segmentation model, and the output is an image containing a mask, confidence score, and category label for each rockfill particle.

[0020] The contour boundary of a single rockfill particle is extracted using a mask, and the equivalent particle size is calculated using the following formula:

[0021] ;

[0022] ;

[0023] Where x and y are the coordinates of a point on a single particle profile; β is the projection direction angle of a single particle profile, ranging from 0 to π; d pixel The maximum diameter pixel value is obtained by traversing the projection direction angle of the particle; d is the equivalent particle diameter, s is the pixel size, Z is the distance from the camera to the particle, and f is the camera focal length.

[0024] Furthermore, the method for determining the initial positional relationship between two adjacent sequence images using their spatial coordinate information is as follows:

[0025] For two adjacent sequence images I k and I k+1 The corresponding GNSS center coordinates are as follows:

[0026] ;

[0027] ;

[0028] Where T is the matrix transpose symbol, and x, y, z are the sequence images I, respectively. k and I k+1 Rectangular coordinates in space;

[0029] The initial positional relationship between two adjacent image sequences is as follows:

[0030] ;

[0031] The spatial distance l and azimuth angle θ between two adjacent image sequences are as follows:

[0032] ;

[0033] .

[0034] Furthermore, the method for determining the theoretical overlapping region of the image using the initial positional relationship is as follows:

[0035] ;

[0036] In the formula, w overlap w is the width of the overlapping region. image α is the image width in pixels; Z is the horizontal field of view of the camera; l is the distance from the camera to the particle; and l is the spatial distance between two adjacent images in the sequence.

[0037] The method for determining the theoretical overlapping region of the images using the initial positional relationship, extracting and retaining reliable edge feature matching point pairs of the rubble particles, and completing the spatial registration of two adjacent image sequences is as follows:

[0038] Within the theoretically overlapping region of two adjacent sequence images, for the rock pile particle mask output by the optimal instance segmentation model, extract the rock pile particle edge feature points and calculate the feature descriptor for each edge feature point;

[0039] A similarity threshold is set, and the similarity of the feature descriptors between two adjacent sequence images is calculated using a feature matching algorithm to obtain initial edge feature matching point pairs;

[0040] Remove erroneous edge feature matching point pairs and retain reliable edge feature matching point pairs; calculate the homography matrix H between two adjacent sequence images:

[0041] ;

[0042] ;

[0043] Where (a,b) and (a′,b′) are two adjacent sequence images I, respectively. k and I k+1 The coordinates of the matching point in h; 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 These are the elements of the homography matrix H;

[0044] Using the homography matrix H, points in one of the sequence images are mapped to another adjacent sequence image.

[0045] Furthermore, by utilizing the equivalent particle size of each rockfill particle and the spatial registration relationship of two adjacent sequence images, the method for identifying and fusing repeatedly identified rockfill particles within the theoretically overlapping area of ​​the images, and for eliminating redundant rockfill particle data, is as follows:

[0046] For the rockfill particles A and B within the theoretically overlapping region of the image, with centroid C... A C B The equivalent particle size is d A d B ;

[0047] If the centroid distance is less than the equivalent particle size, that is:

[0048] ;

[0049] Then, the intersection-union ratio determination will be performed according to the following formula;

[0050] ;

[0051] in, Let be the intersection area of ​​the sequence images of boulders A and B. The area of ​​union of the sequence images of boulders A and B is given.

[0052] If IOU AB If the similarity threshold is reached, the particles are determined to be from the same pile of stones.

[0053] For the same rockfill particle that is successfully matched, retain the rockfill particle profile and corresponding equivalent particle size with higher confidence, and delete redundant rockfill particle data.

[0054] Furthermore, the method for mapping the retained contour information and equivalent particle size of the rockfill particles in each of the sequence images to a unified global coordinate system, using the spatial registration relationship between two adjacent image sequences and the fusion result of the rockfill particles, is as follows:

[0055] Establish a global coordinate system for the rockfill dam, a coordinate system for the image acquisition equipment, and a coordinate system for the image pixels.

[0056] Based on the position coordinates (x) of the spatial positioning device gnss ,y gnss ,z gnss ), and the horizontal distance L between the spatial positioning device and the image acquisition device. offset The position of the center of the image acquisition device in the global coordinate system of the rockfill dam is calculated as follows:

[0057] ;

[0058] Where θ is the azimuth angle of the device's direction of travel, and Δh is the vertical installation height difference between the phase center of the GNSS antenna and the optical center of the image acquisition device;

[0059] For the centroid pixel coordinates (u,v) of each rock pile particle in the sequence image, convert them to the coordinate system of the image acquisition device according to the following formula;

[0060] ;

[0061] Where f is the camera focal length; c x and c y The coordinates of the principal point of the camera image;

[0062] Transform the coordinates of the image acquisition device to the global coordinate system using the following formula;

[0063]

[0064] Where R is the rotation matrix and t is the translation vector;

[0065] The spatial distribution and particle size dataset of the rockfill particles are then fused to generate the dataset.

[0066] Furthermore, the method for generating the overall particle size distribution curve through statistical analysis is as follows:

[0067] Statistical analysis of spatial distribution and particle size of rockfill particles. Equivalent particle size d in the data. i The cumulative distribution of is given by the following formula;

[0068] ;

[0069] Where P(D) represents the percentage of particles smaller than a certain particle size, and d i The equivalent particle size.

[0070] In the above method, the present invention adopts a technical route that combines GNSS high-precision positioning with computer vision. It synchronously acquires images of the rockfill dam surface and centimeter-level position data through mobile acquisition equipment, identifies the contours and equivalent particle sizes of rockfill particles using the optimal instance segmentation model, and realizes the stitching of adjacent images based on the image spatial registration method with GNSS initial position constraints. It matches and fuses rockfill particles in the theoretically overlapping areas of the images to eliminate duplicate counting, and finally constructs a dataset of particle gradation distribution on the surface of the rockfill dam with global spatial coordinates to achieve overall particle gradation detection.

[0071] The present invention also provides a GNSS-visual fusion-based particle size distribution detection system for rockfill dam surfaces, comprising a mobile platform and hardware devices mounted on the mobile platform; the mobile platform is used to move on the surface of the rockfill dam.

[0072] The hardware device includes:

[0073] Spatial positioning equipment used to acquire positioning data from sequential images of the surface of a rockfill dam;

[0074] An image acquisition device is used to acquire a sequence of images of the surface of the rockfill dam, and the image acquisition device is synchronized with the timestamp of the spatial positioning device;

[0075] The data processing module is used to identify rockfill particles in the sequence images, calculate the equivalent particle size of the rockfill particles, spatially register two adjacent sequence images, fuse rockfill particles in the overlapping area of ​​two adjacent sequence images, generate a dataset of spatial distribution and particle size of rockfill particles, and output the final analysis results.

[0076] The results output module is used to display the final analysis results.

[0077] Compared with the prior art, the advantages of the present invention are:

[0078] 1. This invention fully leverages the dual advantages of GNSS high-precision positioning and computer vision fine perception, breaking through the limitation of single-image-level recognition range. Through mobile acquisition and image sequence fusion, it achieves large-scale, full-coverage particle detection on the rockfill dam surface.

[0079] 2. This invention utilizes GNSS to provide absolute centimeter-level reference coordinates, effectively suppressing the accumulation of errors in pure visual registration and ensuring the accuracy of global particle position.

[0080] 3. This invention innovatively proposes a particle matching and fusion algorithm for overlapping regions in image theory, ensuring that each particle is uniquely and optimally represented only once in the global space, resulting in accurate and unbiased data.

[0081] 4. The spatial distribution dataset proposed in this invention can not only statistically analyze the overall particle size distribution curve, but also obtain the spatial distribution of rockfill particles, revealing the spatial variability of the size distribution and assisting in refined management.

[0082] 5. The method proposed in this invention is highly efficient and automated. It utilizes deep learning to automatically identify particles and processes the entire process automatically, with detection efficiency far exceeding that of manual methods. Attached Figure Description

[0083] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0084] Figure 2 This is a schematic diagram of the mobile data acquisition device of the present invention.

[0085] Figure 3 This is a single image of the surface of a rockfill dam collected according to the present invention.

[0086] Figure 4 This is a schematic diagram of adjacent image matching and particle fusion process according to the present invention.

[0087] Figure 5 This is a schematic diagram of the overall surface structure of the rockfill dam according to the present invention.

[0088] Figure 6 This is a schematic diagram of the GNSS positioning trajectory recorded in this invention.

[0089] Figure 7 This is a schematic diagram of the global gradation distribution curve of the rockfill dam surface identified by the present invention. Detailed Implementation

[0090] The technical solution of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0091] Example 1

[0092] This embodiment provides a GNSS-visual fusion-based method for detecting particle size distribution on the surface of a rockfill dam. The monitored area is a compaction test site at a hydropower station, with an effective area of ​​480m². 2 (24 m long, 20 m wide). The overall surface structure of the rockfill dam is as follows: Figure 5 As shown. This method uses a mobile acquisition device to move along the surface of the rockfill dam and acquire a sequence of images of the dam surface and corresponding spatial coordinate information with centimeter-level precision. For example... Figure 1 As shown, the specific steps include the following.

[0093] Step 1: Collect a sequence of images of the rockfill dam surface and the corresponding spatial coordinate information.

[0094] Mobile data acquisition devices (such as...) Figure 2 (As shown) The image moves along the surface of the rockfill dam. During the movement, a sequence of images of the rockfill dam surface and its corresponding spatial coordinate information with centimeter-level precision are acquired simultaneously.

[0095] Specifically, the mobile data acquisition device used in this embodiment is a customized four-wheeled vehicle, such as... Figure 2 As shown, the mobile data acquisition device integrates an industrial camera with a high-precision GNSS (Global Navigation Satellite System) data acquisition unit.

[0096] Optionally, the high-precision GNSS acquisition device can be a receiver that supports RTK (Real-Time Kinematic) positioning, such as the Hi-Target V60, or a receiver that uses PPK (Post-Processing Kinematic) technology; the planar positioning accuracy is ±(10+1×10). - 6 D) mm, elevation positioning accuracy is ±(20+1×10 -6 D) mm.

[0097] Optionally, the industrial camera is equipped with a high-resolution lens, such as a 20-megapixel device. The industrial camera is rigidly connected to the GNSS receiver and mounted on a stable bracket, ensuring a fixed spatial relationship between the camera's optical center and the GNSS antenna's phase center. The industrial camera is positioned perpendicular to the dam surface, at a horizontal distance of 0.2 m, and the distance H from the camera to the dam surface is... c =1.1 m, then the horizontal angle of view α and the field of view range of the camera can be calculated as w×h.

[0098] In the data acquisition route planning, nine parallel survey lines were set along the length direction, spaced 2 meters apart. The equipment moved along a pre-set path on the surface of the rockfill dam; images were taken at 1-meter intervals. During image acquisition, the equipment should remain stationary and stable, synchronously acquiring a sequence of images of the rockfill dam surface and their corresponding spatial coordinate information with centimeter-level accuracy. The GNSS positioning trajectory is as follows: Figure 6 As shown.

[0099] Finally, a series of images covering the surface of the rockfill dam were acquired (e.g. Figure 3 The image shown is a single surface image of a rockfill dam collected by this invention, along with its corresponding spatial coordinate information with centimeter-level precision.

[0100] Step 2: Train to obtain the optimal instance segmentation model. Use the optimal deep learning instance segmentation model to identify the contours of the rock pile particles in each sequence image and calculate the equivalent particle size of each rock pile particle.

[0101] Using the sequence of single images of the rockfill dam surface obtained in step one above, the deep learning-based instance segmentation model is iteratively trained to identify the contours of each rockfill particle in the sequence of images, and the equivalent particle size of each rockfill particle is calculated based on the contours of the rockfill particles. The specific steps are as follows.

[0102] (1) Data annotation: Based on the sequence of images of rock pile particles collected on site, the outlines of the rock pile particles in the sequence of images are annotated with polygons using image annotation software such as Labelme, and a polygon mask label corresponding to each rock pile particle is generated.

[0103] (2) Data augmentation and dataset partitioning: The basic dataset is augmented by preprocessing operations such as parallelization, rotation, cropping, scaling and flipping to obtain a standard dataset for model training, and then randomly divided into training set and validation set according to the ratio of training set: validation set = 8:2.

[0104] (3) Model selection and construction: The basic instance segmentation model is trained using the training set and the validation set to obtain the optimal instance segmentation model for rubble particle identification.

[0105] Specifically, this embodiment preferably employs the Mask R-CNN instance segmentation model, which includes a backbone network for feature extraction, a feature pyramid network, and an instance segmentation detection head. In other alternative embodiments, YOLCAT, SOLO architecture, or their derivatives and improvements may also be used.

[0106] In the specific training process, the transfer learning strategy is preferred, and the pre-trained weights loaded on the open source dataset are used as initialization parameters to accelerate the convergence of the instance segmentation model.

[0107] First, the training set data is input into the basic instance segmentation model in batches. The forward propagation of the instance segmentation model outputs the classification, bounding box, and mask prediction results.

[0108] Secondly, the total loss function between the predicted result and the true label is calculated (preferably obtained by adding the classification loss, bounding box regression loss and mask prediction loss), and the network weights are updated using the backpropagation algorithm combined with the optimizer.

[0109] For example, an initial learning rate of 0.0001 can be set, along with a learning rate decay strategy, and a two-stage fine-tuning strategy can be adopted: first, freeze the underlying parameters to train the detector head, and then fine-tune globally.

[0110] Finally, mean accuracy (mAP) can be used as the evaluation metric on the validation set. After training, the weight file with the highest validation set score is saved as the final deployed model.

[0111] (4) Input the images of the sequence to be identified collected on site into the trained optimal instance segmentation model, and the output result is an image containing the mask, confidence score and category label of each rock pile particle.

[0112] (5) The Canny operator is used to extract the contour boundary of a single rockfill particle from the mask of the rockfill particles, and the equivalent particle size is calculated. The formula for calculating the equivalent particle size is as follows.

[0113] ;

[0114] ;

[0115] Where x and y are the coordinates of a point on a single particle profile; β is the projection direction angle of a single particle profile, ranging from 0 to π; d pixel The maximum diameter pixel value is obtained by traversing the projection direction angle of the particle; d is the equivalent particle diameter, s is the pixel size, Z is the distance from the camera to the particle, and f is the camera focal length.

[0116] Step 3: Based on the spatial coordinate information of two adjacent image sequences from Step 1, spatial registration is achieved using an image feature matching algorithm.

[0117] For the two adjacent image sequences acquired in step one above, the initial spatial relationship is determined based on the spatial coordinate information of the two adjacent image sequences. Using the initial spatial relationship and an image feature matching algorithm, high-precision spatial registration between the two adjacent image sequences is achieved. The specific steps are as follows.

[0118] (1) Two adjacent sequence images are I k and I k+1 The corresponding GNSS center coordinates are as follows:

[0119] ;

[0120] ;

[0121] Where T is the matrix transpose symbol, and x, y, z are the images I, I, and Z, respectively. k and I k+1 Rectangular coordinates in space.

[0122] Therefore, the initial positional relationship between two adjacent image sequences is as follows:

[0123] ;

[0124] The spatial distance l and azimuth angle θ between two adjacent image sequences are as follows:

[0125] ;

[0126] ;

[0127] The image theory estimates the overlapping region as follows:

[0128] ;

[0129] In the formula, w overlap w is the width of the overlapping region. image α is the image width in pixels; Z is the horizontal field of view of the camera; l is the distance from the camera to the particle; and l is the spatial distance between two adjacent images in the sequence.

[0130] (2) Within the estimated theoretical overlapping region of the image, generate feature descriptors.

[0131] This embodiment preferably uses the SIFT (Scale Invariant Feature Transform) algorithm for feature extraction. The specific process includes:

[0132] (1) By constructing a Gaussian difference (DoG) multi-scale space, local extreme points are detected at different scales as key points to ensure the scale invariance of the features; (2) Then, the gradient magnitude and direction of the pixels in the neighborhood of the key point are calculated to determine the main direction of the key point to give the feature rotation invariance; (3) Finally, based on the gradient information of the area around the key point, a multi-dimensional feature descriptor vector is generated.

[0133] After obtaining the feature descriptors, the similarity between the feature descriptors of two adjacent sequence images is calculated using a feature matching algorithm (such as calculating the Euclidean distance). Preliminary matching is performed by setting a similarity threshold to obtain initial edge feature matching point pairs.

[0134] It is understood that, in other alternative embodiments, those skilled in the art may also use SURF, ORB, or deep learning-based local feature extraction algorithms to replace the SIFT algorithm to achieve the same purpose.

[0135] (3) To improve matching accuracy, the RANSAC algorithm is used to remove erroneous edge feature matching point pairs, retain reliable edge feature matching point pairs, and calculate the homography matrix H between two adjacent sequence images. The specific method is as follows:

[0136] From the initial set of edge feature matching point pairs, four non-collinear matching point pairs are randomly selected as interior points, and the remaining matching points are exterior points. The homography matrix is ​​calculated based on the four randomly selected interior points.

[0137] Other matching points are tested based on the homography matrix. The projection error between these points and the model is calculated, and a threshold is set. If the error is less than a threshold, the point is considered a new inlier; otherwise, it is considered an outlier (i.e., a mismatch). Therefore, by using the calculated homography matrix, mismatches can be eliminated in one step.

[0138] The inlier statistics are updated, and the above steps are repeated again. This process is repeated a preset number of times until the model with the most inliers is obtained. At this point, the model is the optimal model, and the final high-precision homography matrix H is calculated.

[0139] ;

[0140] ;

[0141] Where (a,b) and (a′,b′) are two adjacent sequence images I, respectively. k and I k+1 The coordinates of the matching point in h; 11 h 12 h 13 h 21 h 22 h23 h 31 h 32 h 33 These are the elements of the homography matrix.

[0142] Using the homography matrix H described above, points in one sequence of images can be mapped to another adjacent sequence of images, achieving high-precision spatial registration.

[0143] Step 4: Using the equivalent particle size of each rockfill particle obtained in Step 2 and the spatial registration relationship of two adjacent sequence images in Step 3, identify the same rockfill particles that are repeatedly identified in the theoretical overlapping area of ​​two adjacent sequence images; fuse the identification results of the same rockfill particles in two adjacent sequence images and remove redundant rockfill particle data.

[0144] (1) Project the rock particles from different images into the same reference plane within the image theory overlap area.

[0145] Specifically, the centroids of boulders A and B within the overlapping region of the image theory are C. A C B The equivalent particle size is d A d B If the centroid distance is less than the equivalent particle size, that is:

[0146] ;

[0147] Then, the intersection-union ratio determination is performed according to the following formula.

[0148] ;

[0149] in, Let be the intersection area of ​​the sequence images of boulders A and B. This represents the area of ​​union of sequential images of boulders A and B.

[0150] In this embodiment, the similarity threshold is set to 0.75, that is: if IOU AB If the value is greater than 0.75, it is determined that they are from the same pile of stone particles.

[0151] (2) For the same pile of quarry particles that are successfully matched, retain the quarry particle outline and corresponding equivalent particle size with higher confidence in instance segmentation identification in the optimal instance segmentation model, and delete redundant quarry particle data; to ensure that each particle retains only unique and optimal representation data in the global space, such as Figure 4 As shown.

[0152] Step 5: Spatial registration relationship between two adjacent sequence images in Step 3, and the fusion result of rockfill particles in Step 4, through coordinate system transformation, map the rockfill particle contour information and equivalent particle size retained in each sequence image to a unified global coordinate system, and fuse them to generate a dataset of rockfill particle spatial distribution and particle size covering the entire surface of the rockfill dam.

[0153] (1) Establish the global coordinate system (X) of the rockfill dam. g ,Y g Z g The coordinate system of the image acquisition device (i.e., the camera coordinate system) (X) c ,Y c Z c ) and image pixel coordinate system (u,v).

[0154] Based on the position coordinates (x) of the spatial positioning device gnss ,y gnss ,z gnss ), combined with the known rigid connection relationship (horizontal distance L) offset The position of the camera center in the global coordinate system of the rockfill dam is calculated as follows:

[0155] ;

[0156] Where θ is the azimuth angle of the device's direction of travel, and Δh is the vertical installation height difference between the phase center of the GNSS antenna and the optical center of the image acquisition device.

[0157] (2) For the centroid pixel coordinates (u,v) of each rock pile in the sequence image, transform them to the coordinate system of the image acquisition device (i.e., the camera coordinate system).

[0158] ;

[0159] Where f is the camera focal length, in this embodiment f = 6 mm; c x and c y Let c be the coordinates of the principal point of the camera image. x =1000, c y =1500.

[0160] (3) Transform the coordinate system of the image acquisition device (i.e., the camera coordinate system) to the global coordinate system.

[0161] ;

[0162] Where R is the rotation matrix and t is the translation vector.

[0163] In this embodiment, the camera is perpendicular to the dam surface, and the camera's direction of travel is perpendicular to the dam axis. Therefore, the rotation matrix R in this embodiment is:

[0164] ;

[0165] t represents the position of the origin of the image acquisition device's coordinate system (i.e., the camera's coordinate system) relative to the origin of the global coordinate system. In this embodiment, the translation vector t is:

[0166] ;

[0167] The global coordinates, equivalent particle size, and other related data of all fused rockfill particles are transformed into a unified global coordinate system, forming a dataset containing the global coordinates, equivalent particle size, and other information of each rockfill particle, namely, the spatial distribution and particle size dataset of rockfill particles.

[0168] In this embodiment, a total of 216 images were acquired, which, after processing, covered an area of ​​480 m. 2 All particles within the area.

[0169] Step 6: Based on the spatial distribution and particle size dataset of rockfill particles generated in Step 5, statistical analysis is performed to generate an overall particle size distribution curve that reflects the particle size distribution of all rockfill particles on the surface of the entire rockfill dam.

[0170] Statistical analysis of spatial distribution and particle size of rockfill particles. Equivalent particle size d in the data. i The cumulative distribution of is given by the following formula.

[0171] ;

[0172] Where P(D) represents the percentage of particles smaller than a certain particle size, and d i The equivalent particle size.

[0173] The final obtained global gradation distribution curve of the rockfill dam surface is as follows: Figure 7 As shown.

[0174] Example 2

[0175] This embodiment provides a system for detecting particle size distribution on the surface of rockfill dams based on GNSS-visual fusion, such as... Figure 2 As shown, the detection method of Embodiment 1 is implemented. Specifically, it includes a mobile platform for mounting various hardware devices and for moving across the surface of the rockfill dam. The hardware devices include:

[0176] A spatial positioning module (GNSS receiver) is a high-precision GNSS receiving device installed on a mobile platform and rigidly connected to an industrial camera; it is used to acquire centimeter-level accurate spatial coordinate information of the industrial camera's shooting position in real time.

[0177] Image acquisition equipment: an industrial camera mounted on a mobile platform to acquire image information of the rockfill dam surface; a high-precision GNSS receiver is synchronized with the timestamp of the industrial camera;

[0178] Data processing module: Based on instructions, it identifies rockfill particles in sequence images, calculates the equivalent particle size of rockfill particles, performs high-precision spatial registration of adjacent sequence images, fuses rockfill particles in overlapping areas of adjacent sequence images, generates a dataset of spatial distribution and particle size of rockfill particles, and outputs the final analysis results (e.g., overall particle size distribution curve); in order to realize the GNSS-visual fusion-based particle size distribution detection method for rockfill dam surfaces.

[0179] Results output module: Used to display the final analysis results (such as overall particle size distribution curve, visual distribution map, etc.).

[0180] Optionally, the data processing module can be deployed on a local computing terminal or connected to a cloud server via a wireless communication module.

[0181] The above detailed embodiments describe the implementation of the present invention; however, the present invention is not limited to the specific details described in the above embodiments. Within the scope of the claims and technical concept of the present invention, various simple modifications and changes can be made to the technical solution of the present invention, and these simple modifications all fall within the protection scope of the present invention.

Claims

1. A method for detecting particle size distribution on the surface of a rockfill dam based on GNSS-visual fusion, characterized in that, Includes the following steps: Collect sequential images of the rockfill dam surface and its corresponding spatial coordinate information; The optimal instance segmentation model is obtained through training. The optimal instance segmentation model is used to identify the contour of the rock pile particles in each of the sequence images and to calculate the equivalent particle size of each rock pile particle. The initial positional relationship between two adjacent image sequences is determined using the spatial coordinate information of the two sequences. The theoretical overlapping area of ​​the images is determined using the initial positional relationship. Reliable edge feature matching point pairs of the pile particles are extracted and retained to complete the spatial registration of two adjacent image sequences. By utilizing the equivalent particle size of each rockfill particle and the spatial registration relationship of two adjacent sequence images, the same rockfill particles that are repeatedly identified within the theoretical overlapping area of ​​the images are identified and fused, and redundant rockfill particle data are eliminated. Using the spatial registration relationship between two adjacent image sequences and the fusion result of rockfill particles, the rockfill particle contour information and equivalent particle size retained in each image sequence are mapped to a unified global coordinate system, and a dataset of rockfill particle spatial distribution and particle size covering the entire surface of the rockfill dam is generated. Based on the spatial distribution and particle size dataset of the rockfill particles, statistical analysis is used to generate an overall particle size distribution curve that reflects the particle size distribution of all rockfill particles on the surface of the entire rockfill dam.

2. The method for detecting particle size distribution on the surface of a rockfill dam based on GNSS-visual fusion according to claim 1, characterized in that, The rockfill dam surface is captured using a spatial positioning device and an image acquisition device, along with corresponding spatial coordinate information. The spatial positioning device is a GNSS receiver that supports RTK or PPK.

3. The method for detecting particle size distribution on the surface of a rockfill dam based on GNSS-visual fusion according to claim 1, characterized in that, The method for training a deep learning instance segmentation model is as follows: polygonal annotation is performed on the contours of the rubble particles in the sequence images to obtain a polygonal mask of the rubble particles. Using the polygonal mask of the piled-up particles as the basic dataset, the basic dataset is enhanced through preprocessing operations such as parallelization, rotation, clipping, scaling, and flipping to obtain a standard dataset, which is then divided into a training set and a validation set. Construct a basic instance segmentation model based on deep learning, input the training set into the basic instance segmentation model for iterative training, calculate the total loss function, update the weights until the accuracy requirement is met, and obtain the optimal instance segmentation model. Furthermore, the total loss function includes classification loss, bounding box regression loss, and mask prediction loss.

4. The method for detecting particle size distribution on the surface of a rockfill dam based on GNSS-visual fusion according to claim 1, characterized in that, The method for identifying the contours of rubble particles in each of the sequence images using the optimal instance segmentation model and calculating the equivalent particle size of each rubble particle is as follows: The sequence of images to be identified, collected on-site, is input into the optimal instance segmentation model, and the output is an image containing a mask, confidence score, and category label for each rockfill particle. The contour boundary of a single rockfill particle is extracted using a mask, and the equivalent particle size is calculated using the following formula: ； ； Where x and y are the coordinates of a point on a single particle profile; β is the projection direction angle of a single particle profile, ranging from 0 to π; d pixel The maximum diameter pixel value is obtained by traversing the projection direction angle of the particle; d is the equivalent particle diameter, s is the pixel size, Z is the distance from the camera to the particle, and f is the camera focal length.

5. The method for detecting particle size distribution on the surface of a rockfill dam based on GNSS-visual fusion according to claim 1, characterized in that, The method for determining the initial positional relationship between two adjacent sequence images using their spatial coordinate information is as follows: For two adjacent sequence images I k and I k+1 The corresponding GNSS center coordinates are as follows: ； ； Where T is the matrix transpose symbol, and x, y, z are the sequence images I, respectively. k and I k+1 Rectangular coordinates in space; The initial positional relationship between two adjacent image sequences is as follows: ； The spatial distance l and azimuth angle θ between two adjacent image sequences are as follows: ； 。 6. The method for detecting particle size distribution on the surface of a rockfill dam based on GNSS-visual fusion according to claim 5, characterized in that, The method for determining the theoretical overlapping region of an image using the initial positional relationship is as follows: ； In the formula, w overlap w is the width of the overlapping region. image α is the image width in pixels; Z is the horizontal field of view of the camera; l is the distance from the camera to the particle; and l is the spatial distance between two adjacent images in the sequence. The method for determining the theoretical overlapping region of the images using the initial positional relationship, extracting and retaining reliable edge feature matching point pairs of the rubble particles, and completing the spatial registration of two adjacent image sequences is as follows: Within the theoretically overlapping region of two adjacent sequence images, for the rock pile particle mask output by the optimal instance segmentation model, extract the rock pile particle edge feature points and calculate the feature descriptor for each edge feature point; A similarity threshold is set, and the similarity of the feature descriptors between two adjacent sequence images is calculated using a feature matching algorithm to obtain initial edge feature matching point pairs; Remove erroneous edge feature matching point pairs and retain reliable edge feature matching point pairs; calculate the homography matrix H between two adjacent sequence images: ； ； Where (a,b) and (a′,b′) are two adjacent sequence images I, respectively. k and I k+1 The coordinates of the matching point in h; 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 These are the elements of the homography matrix H; Using the homography matrix H, points in one of the sequence images are mapped to another adjacent sequence image.

7. The method for detecting particle size distribution on the surface of a rockfill dam based on GNSS-visual fusion according to claim 1, characterized in that, The method for identifying and fusing repeatedly identified rockfill particles within the theoretically overlapping area of ​​the images, using the equivalent particle size of each rockfill particle and the spatial registration relationship of two adjacent sequence images, and removing redundant rockfill particle data, is as follows: For the rockfill particles A and B within the theoretically overlapping region of the image, with centroid C... A C B The equivalent particle size is d A d B ; If the centroid distance is less than the equivalent particle size, that is: ； Then, the intersection-union ratio determination will be performed according to the following formula; ； in, Let be the intersection area of ​​the sequence images of boulders A and B. The area of ​​union of the sequence images of boulders A and B is given. If IOU AB If the similarity threshold is reached, the particles are determined to be from the same pile of stones. For the same rockfill particle that is successfully matched, retain the rockfill particle profile and corresponding equivalent particle size with higher confidence, and delete redundant rockfill particle data.

8. The method for detecting particle size distribution on the surface of a rockfill dam based on GNSS-visual fusion according to claim 1, characterized in that, The method for mapping the retained contour information and equivalent particle size of the rockfill particles in each of the image sequences to a unified global coordinate system, using the spatial registration relationship between two adjacent image sequences and the fusion result of the rockfill particles, is as follows: Establish the global coordinate system of the rockfill dam, the coordinate system of the image acquisition equipment, and the coordinate system of the image pixels; Based on the position coordinates (x) of the spatial positioning device gnss ,y gnss ,z gnss ), and the horizontal distance L between the spatial positioning device and the image acquisition device. offset The position of the center of the image acquisition device in the global coordinate system of the rockfill dam is calculated as follows: ； Where θ is the azimuth angle of the device's direction of travel, and Δh is the vertical installation height difference between the phase center of the GNSS antenna and the optical center of the image acquisition device; For the centroid pixel coordinates (u,v) of each rock pile particle in the sequence image, convert them to the coordinate system of the image acquisition device according to the following formula; ； Where f is the camera focal length; c x and c y The coordinates of the principal point of the camera image; Transform the coordinates of the image acquisition device to the global coordinate system using the following formula; ； Where R is the rotation matrix and t is the translation vector; The spatial distribution and particle size dataset of the rockfill particles are then fused together.

9. The method for detecting particle size distribution on the surface of a rockfill dam based on GNSS-visual fusion according to claim 1, characterized in that, The method for generating the overall particle size distribution curve through statistical analysis is as follows: Statistical analysis of spatial distribution and particle size of rockfill particles. Equivalent particle size d in the data. i The cumulative distribution of is given by the following formula; ； Where P(D) represents the percentage of particles smaller than a certain particle size, and d i The equivalent particle size.

10. A GNSS-visual fusion-based particle size distribution detection system for rockfill dam surfaces, characterized in that, Includes a mobile platform and hardware devices mounted on the mobile platform; the mobile platform is used to move on the surface of the rockfill dam. The hardware device includes: Spatial positioning equipment used to acquire positioning data from sequential images of the surface of a rockfill dam; An image acquisition device is used to acquire a sequence of images of the surface of the rockfill dam, and the image acquisition device is synchronized with the timestamp of the spatial positioning device; The data processing module is used to identify rockfill particles in the sequence images, calculate the equivalent particle size of the rockfill particles, spatially register two adjacent sequence images, fuse rockfill particles in the overlapping area of ​​two adjacent sequence images, generate a dataset of spatial distribution and particle size of rockfill particles, and output the final analysis results. The results output module is used to display the final analysis results.

Images