Three-dimensional particle size analysis method, system and device based on unmanned aerial vehicle image and multi-view identification, and storage medium

Abstract

The application discloses a three-dimensional particle granularity analysis method, system and device based on unmanned aerial vehicle image and multi-view identification and a storage medium, and belongs to the technical field of geological disaster investigation and geotechnical engineering. The method comprises the following steps: obtaining image data of a target accumulation area; generating a three-dimensional analysis frame; automatically cutting out corresponding two-dimensional image segments on effective view angle images; accurately identifying the boundaries of particles on each two-dimensional image segment to obtain target particles; reversely projecting the boundaries of the particles on all two-dimensional image segments back to a three-dimensional space to reconstruct three-dimensional geometric feature data of the target particles; and obtaining a three-dimensional particle group. Through the "two-dimensional segmentation + three-dimensional reverse projection" strategy, the fine contours of the particles segmented on the multi-view two-dimensional images are intersected in the three-dimensional space, the "fine three-dimensional entity" is successfully reconstructed from the "sparse data source", and the unclear tiny boundaries in the point cloud can be easily identified.

Description

Technical Field

[0001] This invention belongs to the field of geological disaster investigation and geotechnical engineering technology, and in particular relates to a three-dimensional particle size analysis method, system, equipment and storage medium based on UAV imagery and multi-view recognition. Background Technology

[0002] Currently, the main methods for obtaining the particle size distribution of loose sediments include: manual contact measurement, two-dimensional image analysis, and three-dimensional laser scanning. Manual contact measurement involves on-site workers using calipers, sieves, and other tools to directly measure and sieve the particles in the sediment. This is the most traditional method, requiring on-site work, resulting in a large workload, low efficiency, and high safety risks, especially for newly formed and unstable sediments. Furthermore, the measurement process disturbs the original state of the sediment. Two-dimensional image analysis involves taking two-dimensional photographs of the sediment and using image processing software to identify particle boundaries and estimate particle size. This method achieves non-contact measurement, improving efficiency and safety. Three-dimensional laser scanning uses equipment such as a ground-based three-dimensional laser scanner (LiDAR) to obtain a high-precision three-dimensional point cloud model of the sediment, and then performs particle segmentation and size measurement on the point cloud model. This method can obtain accurate three-dimensional information. Summary of the Invention

[0003] The purpose of this invention is to provide a method, system, device, and storage medium for three-dimensional particle size analysis based on UAV imagery and multi-view recognition, so as to solve the problems of insufficient accuracy of two-dimensional image analysis methods and high cost and difficulty of point cloud segmentation of three-dimensional laser scanning methods in the prior art.

[0004] The embodiments of this application are implemented as follows: a three-dimensional particle size analysis method based on UAV imagery and multi-view recognition, including: S01. Obtain image data of the target accumulation area, process the image data, and generate a three-dimensional sparse point cloud model of the target accumulation area and camera pose parameters for each image data. S02. Select an image from the image data and define it as the source image. Obtain the region of interest from the source image. Use the camera pose parameters and the 3D sparse point cloud model to back-project the region of interest to generate a 3D analysis box. S03. Traverse all images in the image data, automatically calculate and filter out the effective view images that observe the three-dimensional analysis frame, and then automatically crop out the corresponding two-dimensional image segments from the effective view images. S04. On each two-dimensional image segment, accurately identify the boundary of the particle, obtain the target particle, and extract the two-dimensional detection result of the target particle; S05. Based on the accurate identification of particle boundaries on all two-dimensional image fragments, project them back into three-dimensional space to reconstruct the three-dimensional geometric feature data of the target particle. S06. Compare the three-dimensional geometric feature data of the target particles with the two-dimensional detection results, construct feature vectors, and input them into a pre-trained machine learning classification model based on random forest to obtain a three-dimensional particle population.

[0005] Optionally, in some embodiments of this application, a common viewpoint index matching method is used to automatically calculate and filter the effective viewpoint images observed by the three-dimensional analysis frame. The method for automatically calculating and filtering the effective viewpoint images observed by the three-dimensional analysis frame using the common viewpoint index matching method includes: S031. Obtain all three-dimensional sparse points within the three-dimensional analysis frame; S032. Inquire which images observed the above three-dimensional sparse points; S033. If the number of sparse 3D points within the 3D analysis frame observed in an image exceeds a preset threshold, then the image is determined to be a valid viewpoint image for observing the 3D analysis frame.

[0006] Optionally, in some embodiments of this application, a high-precision dual adaptive thresholding and watershed algorithm is performed in parallel on all generated two-dimensional image segments, wherein the two-dimensional detection results include two-dimensional centroid coordinates and two-dimensional major and minor axis lengths; and / or A three-dimensional particle population is obtained, and the output results include various size parameters of the three-dimensional particle population, and finally, a particle gradation curve and a statistical table of particle size analysis results are generated.

[0007] Optionally, in some embodiments of this application, a high-precision dual adaptive thresholding and watershed algorithm is performed in parallel on all generated two-dimensional image segments, including the following steps: S041. Divide the two-dimensional image segment into several local micro-blocks, calculate the Otsu threshold independently for each local micro-block, and construct an initial threshold matrix. S042. Apply Gaussian filtering to the initial threshold matrix for spatial smoothing to obtain the reference threshold field; S043. Based on the benchmark threshold field, two different weighting coefficients are applied to perform weighting processing to generate a strict threshold map and a tolerance threshold map. S044. The region defined by the strict threshold map is taken as the explicit foreground, i.e. the initial seed; the region outside the tolerance threshold map is taken as the explicit background; and the region between the explicit foreground and the explicit background is defined as the unknown region. S045. Starting from the initial seed, a watershed algorithm is introduced in the unknown region. Based on the pixel gradient, edge optimization and boundary competition are performed to accurately identify the boundary of the particle, obtain the target particle, and automatically calculate the two-dimensional detection result of the target particle.

[0008] Optionally, in some embodiments of this application, the respective camera pose parameters are used to project back into three-dimensional space, and a clustering algorithm is used to aggregate multiple two-dimensional detection results from different viewpoints but pointing to the same target particle. This process is repeated to aggregate all target particles, thereby reconstructing the three-dimensional geometric feature data of all target particles. The three-dimensional geometric feature data includes the three-dimensional centroid and the three-dimensional major and minor axes.

[0009] Optionally, in some embodiments of this application, the method for obtaining the pre-trained random forest-based machine learning classification model includes: S061. Using the three-dimensional geometric feature data of the target particle as the evaluation object, the two-dimensional detection result corresponding to the target particle is retrieved back as the verification benchmark. S062. Calculate the geometric consistency statistical index between the projection of the evaluation object onto the two-dimensional image segment and the verification benchmark, and construct a feature vector; S063. Assign corresponding classification labels to the feature vectors to construct the training dataset; S064. Based on the training dataset, train a machine learning classification model based on random forest to obtain a pre-trained machine learning classification model based on random forest.

[0010] Optionally, in some embodiments of this application, the feature vector includes one or more of the following: multi-view centroid dispersion, axis length consistency, and reprojection viewpoint coverage; and / or The reconstructed three-dimensional geometric feature data of each target particle is parsed and projected into a two-dimensional contour in three-dimensional space. Then, it is precisely superimposed on the corresponding original two-dimensional image in a semi-transparent or wireframe form. By observing the boundary matching degree between the two-dimensional contour and the soil texture in the original two-dimensional image, the candidate samples are classified and labeled in two categories. The one-to-one correspondence between the two-dimensional labels and the feature vectors is recorded to generate a training dataset. If the 2D contour matches the boundary of the soil texture in the original 2D image, it is a real particle and is marked as a positive sample; if the 2D contour covers vegetation texture, water surface, shadow area or area without clear entity texture, it is a disturbance and is marked as a negative sample.

[0011] Accordingly, embodiments of this application also provide a three-dimensional particle size analysis system based on UAV imagery and multi-view recognition, including: The data acquisition and modeling module is used to obtain image data of the target accumulation area, process the image data, and generate a three-dimensional sparse point cloud model of the target accumulation area and camera pose parameters for each image data. The 3D analysis box acquisition module is used to select an image from the image data as the source image, obtain the region of interest from the source image, and back-project the region of interest using the camera pose parameters and the 3D sparse point cloud model to generate a 3D analysis box. The two-dimensional image fragment module is used to traverse all images in the image data, automatically calculate and filter out the effective view images that observe the three-dimensional analysis frame, and then automatically crop out the corresponding two-dimensional image fragments from the effective view images. The two-dimensional detection result module is used to accurately identify the boundaries of particles in each two-dimensional image segment, obtain the target particles, and extract the two-dimensional detection results of the target particles. The 3D geometric feature data acquisition module accurately identifies the boundaries of particles on all 2D image fragments, projects them back into 3D space, and reconstructs the 3D geometric feature data of the target particles. The 3D particle swarm module is used to compare the 3D geometric feature data of the target particles with the 2D detection results, construct feature vectors, and input them into a pre-trained machine learning classification model based on random forest to obtain a 3D particle swarm.

[0012] Accordingly, embodiments of this application also provide a computer device, including a storage device and a processor, wherein the storage device stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0013] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0014] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: In this application: This application abandons the traditional approach of "hard segmentation" directly on 3D point clouds, and instead utilizes sparse point clouds as "spatial localization indexes" (i.e., only used to tell the system which images the particles are on), transferring the core segmentation task to high-resolution 2D raw images. Since the pixel density of 2D images is much higher than that of 3D point clouds, and they have rich texture and edge information, combined with the improved high-precision dual adaptive threshold segmentation and watershed algorithm of this application, it is possible to easily identify the blurry and indistinct small boundaries in the point cloud; This application does not rely on expensive LiDAR to acquire segmentable dense point clouds, but only requires ordinary consumer-grade UAV imagery. By employing a "2D segmentation + 3D backprojection" strategy, it utilizes the fine contours of particles segmented from multi-view 2D images to find their intersection in 3D space, successfully reconstructing "fine 3D entities" from "sparse data sources," fundamentally solving the technical challenge of the trade-off between low cost and high accuracy in traditional methods. Attached Figure Description

[0015] Figure 1 This is a flowchart of the three-dimensional particle size analysis method based on UAV imagery and multi-view recognition according to the present invention; Figure 2 This is a schematic diagram illustrating the interactive selection of a region of interest (ROI) on a source image according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the effect of projecting the three-dimensional geometric feature data of particles onto a two-dimensional image to generate a colored axis, as described in an embodiment of the present invention. Figure 4 This is a schematic diagram of the effective soil and rock particle identification results described in the embodiments of the present invention (displayed as a semi-transparent geometric patch based on axial length features). Figure 5 This is a schematic diagram of the particle size distribution curve described in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] The technical solution of this application is as follows: Please see Figure 1 This application provides a three-dimensional particle size analysis method based on UAV imagery and multi-view recognition, including: S01. Obtain image data of the target accumulation area, process the image data, and generate a three-dimensional sparse point cloud model of the target accumulation area and camera pose parameters for each image data. S02. Select an image from the image data and define it as the source image. Obtain the region of interest from the source image. Use the camera pose parameters and the 3D sparse point cloud model to back-project the region of interest to generate a 3D analysis box. S03. Traverse all images in the image data, automatically calculate and filter out the effective view images that observe the three-dimensional analysis frame, and then automatically crop out the corresponding two-dimensional image segments from the effective view images. S04. On each two-dimensional image segment, accurately identify the boundary of the particle, obtain the target particle, and extract the two-dimensional detection result of the target particle; S05. Based on the accurate identification of particle boundaries on all two-dimensional image fragments, project them back into three-dimensional space to reconstruct the three-dimensional geometric feature data of the target particle. S06. Compare the three-dimensional geometric feature data of the target particles with the two-dimensional detection results, construct feature vectors, and input them into a pre-trained machine learning classification model based on random forest to obtain a three-dimensional particle population.

[0018] Traditional point cloud segmentation requires high-density point clouds, which is costly and results in sparse point cloud boundaries that are blurry. This application adopts a strategy of "2D segmentation + 3D back projection (3D back projection refers to the back projection of particle segmentation results on a 2D image back into 3D space)". It uses sparse point clouds as spatial positioning indices and executes high-precision dual adaptive threshold segmentation and watershed algorithms in parallel on the 2D image. The fine contours of the particles segmented on the 2D image are back projected back into 3D space using their respective camera pose parameters. Clustering algorithms are then used to reconstruct the 3D geometric feature data of the target particles.

[0019] In S01: In some embodiments, a drone is used to acquire multi-angle, highly overlapping image data of the target accumulation area, and the image data of the target accumulation area is processed by the structure-reconstruction-motion algorithm to generate a three-dimensional sparse point cloud model of the target accumulation area and camera pose parameters for each image data.

[0020] For example, the drone can be a consumer-grade drone or an industrial-grade drone.

[0021] In S02: In some embodiments, an image is selected from the image data and defined as the source image. A two-dimensional rectangle is selected from the source image as the region of interest. The region of interest is back-projected using camera pose parameters and a three-dimensional sparse point cloud model to generate a corresponding three-dimensional analysis box in three-dimensional space.

[0022] In S03: In some embodiments, a common viewpoint index matching method is used to automatically calculate and filter out the effective viewpoint images that observe the three-dimensional analysis frame.

[0023] Furthermore, the method for automatically calculating and filtering the effective viewpoint images observed within the three-dimensional analysis frame using a common viewpoint index matching method includes: S031. Obtain all three-dimensional sparse points within the three-dimensional analysis frame; S032. Inquire which images observed the above three-dimensional sparse points; S033. If the number of 3D sparse points within the 3D analysis frame observed in a certain image exceeds a preset threshold. Then the image is determined to be a valid viewpoint image of the observed three-dimensional analysis frame.

[0024] For example, the preset threshold is 3 or 5.

[0025] In S04: In some embodiments, a high-precision dual adaptive threshold segmentation and watershed algorithm is executed in parallel on all generated two-dimensional image segments to accurately identify the boundaries of particles on each two-dimensional image segment, obtain the target particles, and extract the two-dimensional detection results of the target particles, which include two-dimensional centroid coordinates and two-dimensional major and minor axis lengths.

[0026] Furthermore, a high-precision dual adaptive thresholding and watershed algorithm is performed in parallel on all generated two-dimensional image segments, including the following steps: S041. Divide the two-dimensional image segment into several local micro-blocks, calculate the Otsu threshold independently for each local micro-block, and construct an initial threshold matrix. S042. Apply Gaussian filtering to the initial threshold matrix for spatial smoothing to obtain the reference threshold field; S043. Based on the benchmark threshold field, two different weighting coefficients are applied to perform weighting processing to generate a strict threshold map and a tolerance threshold map. S044. The region defined by the strict threshold map is taken as the explicit foreground, i.e. the initial seed; the region outside the tolerance threshold map is taken as the explicit background; and the region between the explicit foreground and the explicit background is defined as the unknown region. S045. Starting from the initial seed, a watershed algorithm is introduced in the unknown region. Edge optimization and boundary competition are performed based on pixel gradients to accurately identify the boundary of the particle, obtain the target particle, and automatically calculate the two-dimensional detection result of the target particle. The two-dimensional detection result of the target particle includes: two-dimensional centroid coordinates and two-dimensional major and minor axis lengths.

[0027] It is understood that, starting from the initial seed, a watershed algorithm is introduced in the unknown region to perform edge optimization and boundary competition based on pixel gradient, thereby effectively solving the particle adhesion problem, identifying the boundaries of the particles, and then performing geometric analysis on the identified particle boundaries to obtain the target particle and calculate the two-dimensional detection result of the target particle.

[0028] For example, the two different weighting coefficients include 0.9 and 0.5.

[0029] It is understandable that strict threshold maps are used to extract high-confidence particle core regions, while tolerance threshold maps are used to define the potential boundary range of particles.

[0030] It is understandable that applying Gaussian filtering to the initial threshold matrix for spatial smoothing aims to eliminate discontinuities between blocks.

[0031] It is understandable that the Otsu threshold is the Otsu threshold or the Otsu method threshold.

[0032] Traditional low-cost photogrammetry (SfM) generates sparse point clouds with extremely low density and huge gaps between points, making it impossible to delineate clear boundaries of soil and rock particles. Therefore, direct segmentation on sparse point clouds is physically infeasible. Generating high-density point clouds is not only computationally time-consuming but also prone to noise and adhesion on the surface of complex deposits. The breakthrough of this application lies in abandoning the traditional approach of "hard segmentation" directly on 3D point clouds. Instead, it utilizes sparse point clouds as "spatial positioning indexes" (i.e., only used to tell the system which images the particles are on), transferring the core segmentation task to high-resolution 2D raw images. Since the pixel density of 2D images is much higher than that of 3D point clouds and has rich texture edge information, combined with the improved high-precision dual adaptive threshold segmentation and watershed algorithm of this application, it is possible to easily identify the blurred and tiny boundaries in the point cloud.

[0033] In S05: In some embodiments, based on the accurate identification of particle boundaries on all two-dimensional image fragments, the respective camera pose parameters are used to project back into three-dimensional space. A clustering algorithm is used to aggregate multiple two-dimensional detection results from different viewpoints but pointing to the same target particle. This process is repeated to aggregate all target particles, thereby reconstructing the three-dimensional geometric feature data of all target particles. The three-dimensional geometric feature data includes the three-dimensional centroid and the three-dimensional major and minor axes.

[0034] Furthermore, the clustering algorithm includes gen2_cluster.

[0035] In S06: It is understandable that the machine learning classification model based on random forest can intelligently determine whether the target particles are real particles or misidentified particles, and automatically eliminate misidentified results to obtain a verified, high-quality three-dimensional particle population.

[0036] It is understandable that the three-dimensional geometric feature data of the target particle is compared with the two-dimensional detection results of the target particle to construct a feature vector that reflects the consistency of reconstruction.

[0037] In some embodiments, the method for obtaining a pre-trained random forest-based machine learning classification model includes: S061. Using the three-dimensional geometric feature data of the target particle as the evaluation object, the two-dimensional detection result corresponding to the target particle is retrieved back as the verification benchmark. S062. Calculate the geometric consistency statistical index between the projection of the evaluation object onto the two-dimensional image segment and the verification benchmark, and construct a feature vector; S063. Assign corresponding classification labels to the feature vectors to construct the training dataset; S064. Based on the training dataset, train a machine learning classification model based on random forest to obtain a pre-trained machine learning classification model based on random forest.

[0038] In S062: In some embodiments, the feature vector includes one or more of the following: multi-view centroid dispersion, axis length consistency, and reprojection viewpoint coverage.

[0039] Furthermore, multi-view centroid dispersion: the three-dimensional centroid of the target particle is projected onto each two-dimensional image segment to obtain multiple projected coordinates of the three-dimensional centroid. The Euclidean distance between each projected coordinate of the three-dimensional centroid and the two-dimensional centroid coordinates of the target particle on the two-dimensional image segment is calculated. The corresponding Euclidean distance is normalized using the lengths of the two-dimensional major and minor axes on each two-dimensional image segment to obtain the normalized distance of each segment. Finally, the average value of the normalized distances of all two-dimensional image segments is calculated. Axis length consistency: Statistically calculate the multiple projection lengths of the three-dimensional major and minor axes of the target particle in each two-dimensional image segment, calculate the ratio between each projection length and the two-dimensional major and minor axis lengths of the target particle in the corresponding two-dimensional image segment, and finally, calculate the average and standard deviation of the ratios in each two-dimensional image segment. Reprojection viewpoint coverage: This counts how many different viewpoints in which target particles are successfully aggregated, used to measure the confidence level of the presence of target particles.

[0040] In S063: In some embodiments, the reconstructed three-dimensional geometric feature data of each target particle is parsed and projected into a two-dimensional contour in three-dimensional space, and then precisely superimposed on the corresponding original two-dimensional image in a semi-transparent or wireframe form. By observing the boundary matching degree between the two-dimensional contour and the soil texture in the original two-dimensional image, the candidate samples are classified and labeled in two categories. The one-to-one correspondence between the two-dimensional labels and the feature vectors is recorded to generate a training dataset.

[0041] Furthermore, if the boundary height of the two-dimensional contour matches the boundary height of the soil texture in the original two-dimensional image, it is a real particle and is marked as a positive sample. If a two-dimensional contour covers vegetation texture, water surface, shadow area, or area without clear entity texture, it is considered an interference item and is marked as a negative sample.

[0042] In S064: Furthermore, by inputting the training dataset, a machine learning classification model based on random forest is trained, generating a machine learning classification model based on random forest for automatically distinguishing between real particles and misidentified particles, which is the pre-trained machine learning classification model based on random forest.

[0043] In some embodiments, a three-dimensional particle population is obtained, and the results are output. The output results include the calculation of various size parameters of the three-dimensional particle population, and finally the particle gradation curve and particle size analysis result statistics table are generated.

[0044] Furthermore, the various dimensional parameters include characteristic particle size, D25, D50, D75, non-uniformity coefficient, curvature coefficient, and particle size range.

[0045] Furthermore, the characteristic particle size is the length of the major axis of the particle.

[0046] It is understandable that the particle size distribution curve directly reflects the three-dimensional particle size distribution characteristics of the target packing, thus completing the particle size analysis.

[0047] This application sequentially performs the following steps: acquiring image data, generating a 3D sparse point cloud model, obtaining the region of interest and back-projecting it to generate a 3D analysis box, cropping 2D image fragments from the effective viewpoint image, parallelizing the acquisition of target particles, aggregating and reconstructing the 3D geometric feature data of the target particles, filtering and validating a pre-trained random forest-based machine learning classification model, and granularity analysis and result output. This application reduces the complex 3D particle segmentation problem to simpler and more accurate particle segmentation on multiple 2D images from different viewpoints, avoiding direct processing of complex 3D point clouds and instead performing identification on high-resolution 2D images. Through coordinate transformation, the particle segmentation results from multiple 2D images are back-projected back into a unified 3D space for aggregation, thereby reconstructing the true 3D geometric feature data of a single target particle. After the 3D reconstruction of the target particle, a pre-trained random forest-based machine learning classification model is introduced for validation, intelligently filtering and validating the reconstructed target particles, automatically eliminating erroneous identifications caused by factors such as shadows and vegetation, greatly improving the reliability of the analysis results.

[0048] This application achieves clearer particle boundaries by segmenting particles on multiple high-resolution 2D images, compared to segmenting directly on sparse point clouds. Utilizing the camera pose parameters from multiple 2D images, the true 3D geometric features of the particles (including 3D centroid and 3D major and minor axes) can be accurately calculated, overcoming the perspective distortion and occlusion problems of traditional 2D analysis.

[0049] This application can be completed using only a consumer-grade drone and a regular computer, eliminating the need to purchase an expensive 3D laser scanner, thus greatly lowering the application threshold.

[0050] The entire analysis process in this application runs automatically after the user selects the initial region. The two-dimensional segmentation step employs parallel computing, resulting in high processing speed. Compared to manual measurement, efficiency is improved by several orders of magnitude.

[0051] This application presents a remote sensing detection method that eliminates the need for personnel to enter hazardous sites, ensuring operational safety. Furthermore, the entire process involves no contact or disturbance to the accumulation mass, preserving its original state.

[0052] This application employs a dual adaptive threshold strategy, which effectively solves the segmentation breakage problem caused by uneven lighting and particle shadows in the field. It also innovatively introduces a pre-trained machine learning classification model based on random forest to intelligently verify and filter the detection results, which can effectively eliminate interference caused by complex on-site environments (such as changes in lighting, shadows, and vegetation), making the final analysis results more reliable and closer to the real situation than traditional algorithms.

[0053] Secondly, embodiments of this application provide a three-dimensional particle size analysis system based on UAV imagery and multi-view recognition, comprising: The data acquisition and modeling module is used to obtain image data of the target accumulation area, process the image data, and generate a three-dimensional sparse point cloud model of the target accumulation area and camera pose parameters for each image data. The 3D analysis box acquisition module is used to select an image from the image data as the source image, obtain the region of interest from the source image, and back-project the region of interest using the camera pose parameters and the 3D sparse point cloud model to generate a 3D analysis box. The two-dimensional image fragment module is used to traverse all images in the image data, automatically calculate and filter out the effective view images that observe the three-dimensional analysis frame, and then automatically crop out the corresponding two-dimensional image fragments from the effective view images. The two-dimensional detection result module is used to accurately identify the boundaries of particles in each two-dimensional image segment, obtain the target particles, and extract the two-dimensional detection results of the target particles. The 3D geometric feature data acquisition module accurately identifies the boundaries of particles on all 2D image fragments, projects them back into 3D space, and reconstructs the 3D geometric feature data of the target particles. The 3D particle swarm module is used to compare the 3D geometric feature data of the target particles with the 2D detection results, construct feature vectors, and input them into a pre-trained machine learning classification model based on random forest to obtain a 3D particle swarm.

[0054] In the data acquisition and modeling module: In some embodiments, a drone is used to acquire multi-angle, highly overlapping image data of the target accumulation area, and the image data of the target accumulation area is processed by the structure-reconstruction-motion algorithm to generate a three-dimensional sparse point cloud model of the target accumulation area and camera pose parameters for each image data.

[0055] For example, the drone can be a consumer-grade drone or an industrial-grade drone.

[0056] In the 3D analysis frame acquisition module: In some embodiments, an image is selected from the image data and defined as the source image. A two-dimensional rectangle is selected from the source image as the region of interest. The region of interest is back-projected using camera pose parameters and a three-dimensional sparse point cloud model to generate a corresponding three-dimensional analysis box in three-dimensional space.

[0057] In the two-dimensional image segment module: In some embodiments, a common viewpoint index matching method is used to automatically calculate and filter out the effective viewpoint images that observe the three-dimensional analysis frame.

[0058] Furthermore, the method for automatically calculating and filtering the effective viewpoint images observed within the three-dimensional analysis frame using a common viewpoint index matching method includes: S031. Obtain all three-dimensional sparse points within the three-dimensional analysis frame; S032. Inquire which images observed the above three-dimensional sparse points; S033. If the number of 3D sparse points within the 3D analysis frame observed in a certain image exceeds a preset threshold. Then the image is determined to be a valid viewpoint image of the observed three-dimensional analysis frame.

[0059] For example, the preset threshold is 3 or 5.

[0060] In the two-dimensional detection result module: In some embodiments, a high-precision dual adaptive threshold segmentation and watershed algorithm is executed in parallel on all generated two-dimensional image segments to accurately identify the boundaries of particles on each two-dimensional image segment, obtain the target particles, and extract the two-dimensional detection results of the target particles, which include two-dimensional centroid coordinates and two-dimensional major and minor axis lengths.

[0061] Furthermore, a high-precision dual adaptive thresholding and watershed algorithm is performed in parallel on all generated two-dimensional image segments, including the following steps: S041. Divide the two-dimensional image segment into several local micro-blocks, calculate the Otsu threshold independently for each local micro-block, and construct an initial threshold matrix. S042. Apply Gaussian filtering to the initial threshold matrix for spatial smoothing to obtain the reference threshold field; S043. Based on the benchmark threshold field, two different weighting coefficients are applied to perform weighting processing to generate a strict threshold map and a tolerance threshold map. S044. The region defined by the strict threshold map is taken as the explicit foreground, i.e. the initial seed; the region outside the tolerance threshold map is taken as the explicit background; and the region between the explicit foreground and the explicit background is defined as the unknown region. S045. Starting from the initial seed, a watershed algorithm is introduced in the unknown region. Edge optimization and boundary competition are performed based on pixel gradients to accurately identify the boundary of the particle, obtain the target particle, and automatically calculate the two-dimensional detection result of the target particle. The two-dimensional detection result of the target particle includes: two-dimensional centroid coordinates and two-dimensional major and minor axis lengths.

[0062] It is understood that, starting from the initial seed, a watershed algorithm is introduced in the unknown region to perform edge optimization and boundary competition based on pixel gradient, thereby effectively solving the particle adhesion problem, identifying the boundaries of the particles, and then performing geometric analysis on the identified particle boundaries to obtain the target particle and calculate the two-dimensional detection result of the target particle.

[0063] For example, the two different weighting coefficients include 0.9 and 0.5.

[0064] It is understandable that strict threshold maps are used to extract high-confidence particle core regions, while tolerance threshold maps are used to define the potential boundary range of particles.

[0065] It is understandable that applying Gaussian filtering to the initial threshold matrix for spatial smoothing aims to eliminate discontinuities between blocks.

[0066] Traditional low-cost photogrammetry (SfM) generates sparse point clouds with extremely low density and huge gaps between points, making it impossible to delineate clear boundaries of soil and rock particles. Therefore, direct segmentation on sparse point clouds is physically infeasible. Generating high-density point clouds is not only computationally time-consuming but also prone to noise and adhesion on the surface of complex deposits. The breakthrough of this application lies in abandoning the traditional approach of "hard segmentation" directly on 3D point clouds. Instead, it utilizes sparse point clouds as "spatial positioning indexes" (i.e., only used to tell the system which images the particles are on), transferring the core segmentation task to high-resolution 2D raw images. Since the pixel density of 2D images is much higher than that of 3D point clouds and has rich texture edge information, combined with the improved high-precision dual adaptive threshold segmentation and watershed algorithm of this application, it is possible to easily identify the blurred and tiny boundaries in the point cloud.

[0067] In the three-dimensional geometric feature data acquisition module: In some embodiments, based on the accurate identification of particle boundaries on all two-dimensional image fragments, the respective camera pose parameters are used to project back into three-dimensional space. A clustering algorithm is used to aggregate multiple two-dimensional detection results from different viewpoints but pointing to the same target particle. This process is repeated to aggregate all target particles, thereby reconstructing the three-dimensional geometric feature data of all target particles. The three-dimensional geometric feature data includes the three-dimensional centroid and the three-dimensional major and minor axes.

[0068] Furthermore, the clustering algorithm includes gen2_cluster.

[0069] In the three-dimensional particle population module: It is understandable that the machine learning classification model based on random forest can intelligently determine whether the target particles are real particles or misidentified particles, and automatically eliminate misidentified results to obtain a verified, high-quality three-dimensional particle population.

[0070] It is understandable that the three-dimensional geometric feature data of the target particle is compared with the two-dimensional detection results of the target particle to construct a feature vector that reflects the consistency of reconstruction.

[0071] In some embodiments, the method for obtaining a pre-trained random forest-based machine learning classification model includes: S061. Using the three-dimensional geometric feature data of the target particle as the evaluation object, the two-dimensional detection result corresponding to the target particle is retrieved back as the verification benchmark. S062. Calculate the geometric consistency statistical index between the projection of the evaluation object onto the two-dimensional image segment and the verification benchmark, and construct a feature vector; S063. Assign corresponding classification labels to the feature vectors to construct the training dataset; S064. Based on the training dataset, train a machine learning classification model based on random forest to obtain a pre-trained machine learning classification model based on random forest.

[0072] In S062: In some embodiments, the feature vector includes one or more of the following: multi-view centroid dispersion, axis length consistency, and reprojection viewpoint coverage.

[0073] Furthermore, multi-view centroid dispersion: the three-dimensional centroid of the target particle is projected onto each two-dimensional image segment to obtain multiple projected coordinates of the three-dimensional centroid. The Euclidean distance between each projected coordinate of the three-dimensional centroid and the two-dimensional centroid coordinates of the target particle on the two-dimensional image segment is calculated. The corresponding Euclidean distance is normalized using the lengths of the two-dimensional major and minor axes on each two-dimensional image segment to obtain the normalized distance of each segment. Finally, the average value of the normalized distances of all two-dimensional image segments is calculated. Axis length consistency: Statistically calculate the multiple projection lengths of the three-dimensional major and minor axes of the target particle in each two-dimensional image segment, calculate the ratio between each projection length and the two-dimensional major and minor axis lengths of the target particle in the corresponding two-dimensional image segment, and finally, calculate the average and standard deviation of the ratios in each two-dimensional image segment. Reprojection viewpoint coverage: This counts how many different viewpoints in which target particles are successfully aggregated, used to measure the confidence level of the presence of target particles.

[0074] In S063: In some embodiments, the reconstructed three-dimensional geometric feature data of each target particle is parsed and projected into a two-dimensional contour in three-dimensional space, and then precisely superimposed on the corresponding original two-dimensional image in a semi-transparent or wireframe form. By observing the boundary matching degree between the two-dimensional contour and the soil texture in the original two-dimensional image, the candidate samples are classified and labeled in two categories. The one-to-one correspondence between the two-dimensional labels and the feature vectors is recorded to generate a training dataset.

[0075] Furthermore, if the boundary height of the two-dimensional contour matches the boundary height of the soil texture in the original two-dimensional image, it is a real particle and is marked as a positive sample. If a two-dimensional contour covers vegetation texture, water surface, shadow area, or area without clear entity texture, it is considered an interference item and is marked as a negative sample.

[0076] In S064: Furthermore, by inputting the training dataset, a machine learning classification model based on random forest is trained, generating a machine learning classification model based on random forest for automatically distinguishing between real particles and misidentified particles, which is the pre-trained machine learning classification model based on random forest.

[0077] In some embodiments, a three-dimensional particle population is obtained, and the results are output. The output results include the calculation of various size parameters of the three-dimensional particle population, and finally the particle gradation curve and particle size analysis result statistics table are generated.

[0078] Furthermore, the various dimensional parameters include characteristic particle size, D25, D50, D75, non-uniformity coefficient, curvature coefficient, and particle size range.

[0079] Furthermore, the characteristic particle size is the length of the major axis of the particle.

[0080] It is understandable that the particle size distribution curve directly reflects the three-dimensional particle size distribution characteristics of the target packing, thus completing the particle size analysis.

[0081] Thirdly, this application provides a computer device including a storage device and a processor. The storage device stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the three-dimensional particle size analysis method based on UAV imagery and multi-view recognition as described above.

[0082] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0083] The memory includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or D-interface display memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device. Of course, the memory may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is often used to store the operating system and various application software installed on the computer device, such as the program code of the three-dimensional particle size analysis method based on UAV imagery and multi-view recognition. In addition, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0084] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of the computer device. In this embodiment, the processor is used to run program code stored in the memory or process data, for example, to run the program code for the 3D particle size analysis method based on UAV imagery and multi-view recognition.

[0085] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the three-dimensional particle size analysis method based on UAV imagery and multi-view recognition as described above.

[0086] The computer-readable storage medium stores an interface display program, which can be executed by at least one processor to perform the steps of the three-dimensional particle size analysis method based on UAV imagery and multi-view recognition as described above.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the three-dimensional particle size analysis method based on UAV imagery and multi-view recognition described in the embodiments of this application.

[0088] The invention will be further described below with reference to application examples.

[0089] Application examples This embodiment selects a typical depositional area of ​​a debris flow gully as the analysis object, and uses the system and method described in this invention to perform three-dimensional particle size analysis. The specific implementation steps are as follows: Data Acquisition and Modeling Using a drone, we conducted low-altitude close-up photography of the target accumulation area, acquiring 20 high-resolution images. Based on the SfM algorithm, we generated a three-dimensional sparse point cloud model of the target accumulation area and complete camera pose parameters.

[0090] Interactive region selection Select a clear image from the image data as the source image. For example... Figure 2As shown, users manually create green rectangles on the source image to interactively select regions of interest (ROIs). The system utilizes the spatial depth information of the 3D sparse point cloud and automatically generates corresponding 3D analysis boxes in 3D space using a back projection algorithm. Figure 2 In the image, the green box is created manually by the user to select the area of ​​interest.

[0091] Effective Viewpoint Image Selection and Two-Dimensional Feature Extraction The system automatically traverses all images in the image data, uses a common viewpoint index matching method, sets a preset threshold of 3, automatically filters out the effective viewpoint images that observe the 3D analysis box, and crops out the corresponding 2D image segments. Subsequently, the system executes the dual adaptive Otsu algorithm in parallel on all 2D image segments, setting weight coefficients of 0.9 (generating a strict threshold map) and 0.5 (generating a tolerance threshold map), respectively, and uses the watershed algorithm to accurately extract the boundaries of target particles, and automatically calculates the 2D detection results of each target particle (including 2D centroid coordinates and 2D major and minor axis lengths); 3D backprojection aggregation and geometric feature data reconstruction Based on the precisely extracted boundaries of the target particles from all two-dimensional image fragments, the system uses a clustering algorithm to project them back into three-dimensional space and aggregate them, successfully reconstructing the three-dimensional geometric feature data of the target particles (including the three-dimensional centroid and the three-dimensional major and minor axes). Figure 3 As shown, the system projects the reconstructed three-dimensional geometric feature data of the target particle onto the two-dimensional source image. Its color axis is highly consistent with the actual particle position, which verifies the accuracy of the three-dimensional reconstruction geometric information. Intelligent verification and screening A pre-trained machine learning classification model based on random forest is used to validate the features of the initial reconstructed particles, automatically removing non-soil targets such as sparse vegetation and shadows. Figure 4 As shown, the visualization results of the high-confidence particles retained after verification by the random forest-based machine learning classification model are presented in the form of semi-transparent geometric patches, which intuitively shows the distribution of effective soil and rock particles successfully identified by the system in the target area. Results Output and Analysis The system automatically calculates the three-dimensional geometric parameters of the effective particles and generates particle size distribution curves (such as...). Figure 5 The results are shown in Table 1, which presents the particle size analysis results. The results show that the uniformity coefficient Cu in this area is approximately 3.04, and the curvature coefficient Cc is approximately 0.90. According to the soil mechanics standard (Cu<5 and Cc<1), this result indicates that the particle size distribution in the target area is relatively concentrated and the sorting is good (i.e., uniform gradation), accurately reflecting the geological depositional characteristics of the debris flow deposit in the local area.

[0092] Table 1. Statistical table of particle size analysis results

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-dimensional particle size analysis method based on UAV imagery and multi-view recognition, characterized in that, include: Obtain image data of the target accumulation area, process the image data, and generate a three-dimensional sparse point cloud model of the target accumulation area and camera pose parameters for each image data. Select an image from the image data and define it as the source image. Obtain the region of interest from the source image. Use the camera pose parameters and the 3D sparse point cloud model to back-project the region of interest to generate a 3D analysis box. The system iterates through all images in the image data, automatically calculates and filters out the effective viewpoint images that observe the three-dimensional analysis frame, and then automatically crops out the corresponding two-dimensional image segments from the effective viewpoint images. On each two-dimensional image fragment, the boundaries of the particles are accurately identified, the target particles are obtained, and the two-dimensional detection results of the target particles are extracted. Based on the accurate identification of particle boundaries on all two-dimensional image fragments, the particles are projected back into three-dimensional space to reconstruct the three-dimensional geometric feature data of the target particles. The three-dimensional geometric feature data of the target particles are compared with the two-dimensional detection results to construct a feature vector, which is then input into a pre-trained machine learning classification model based on random forest to obtain a three-dimensional particle population.

2. The three-dimensional particle size analysis method based on UAV imagery and multi-view recognition according to claim 1, characterized in that, The method of automatically calculating and filtering effective viewpoint images of the observed 3D analysis frame using a common viewpoint index matching method includes: Obtain all three-dimensional sparse points within the three-dimensional analysis frame; Find out which images observed the above 3D sparse points; If the number of sparse 3D points within the 3D analysis frame observed in an image exceeds a preset threshold, then the image is determined to be a valid viewpoint image that observes the 3D analysis frame.

3. The three-dimensional particle size analysis method based on UAV imagery and multi-view recognition according to claim 1, characterized in that, A high-precision dual adaptive thresholding and watershed algorithm is performed in parallel on all generated 2D image segments. The 2D detection results include 2D centroid coordinates and 2D major and minor axis lengths; and / or A three-dimensional particle population is obtained, and the results are output, including the calculation of various size parameters of the three-dimensional particle population, and finally the particle gradation curve and particle size analysis result statistics table are generated.

4. The three-dimensional particle size analysis method based on UAV imagery and multi-view recognition according to claim 3, characterized in that, A high-precision dual adaptive thresholding and watershed algorithm is performed in parallel on all generated 2D image segments, including the following steps: The two-dimensional image segment is divided into several local micro-blocks. The Otsu threshold is calculated independently for each local micro-block to construct an initial threshold matrix. Gaussian filtering is applied to the initial threshold matrix for spatial smoothing to obtain the reference threshold field; Based on the baseline threshold field, two different weighting coefficients are applied to perform weighting processing to generate strict threshold map and tolerance threshold map; The region defined by the strict threshold map is taken as the explicit foreground, i.e. the initial seed; the region outside the tolerance threshold map is taken as the explicit background; and the region between the explicit foreground and the explicit background is defined as the unknown region. Starting with the initial seed, a watershed algorithm is introduced in the unknown region. Based on the pixel gradient, edge optimization and boundary competition are performed to accurately identify the boundary of the particle, obtain the target particle, and automatically calculate the two-dimensional detection result of the target particle.

5. The three-dimensional particle size analysis method based on UAV imagery and multi-view recognition according to claim 1, characterized in that, Using the respective camera pose parameters, the data is projected back into three-dimensional space. Then, a clustering algorithm is used to aggregate multiple two-dimensional detection results from different viewpoints that point to the same target particle. This process is repeated to aggregate all target particles, thereby reconstructing the three-dimensional geometric feature data of all target particles. The three-dimensional geometric feature data includes the three-dimensional centroid and the three-dimensional major and minor axes.

6. The three-dimensional particle size analysis method based on UAV imagery and multi-view recognition according to claim 1, characterized in that, Methods for obtaining pre-trained machine learning classification models based on random forests include: Using the three-dimensional geometric feature data of the target particle as the evaluation object, the corresponding two-dimensional detection result of the target particle is retrieved back as the verification benchmark. Calculate the geometric consistency statistical index between the projection of the evaluation object onto the two-dimensional image segment and the verification benchmark, and construct a feature vector; Assign corresponding classification labels to feature vectors to construct a training dataset; Based on the training dataset, a machine learning classification model based on random forest is trained to obtain a pre-trained machine learning classification model based on random forest.

7. The three-dimensional particle size analysis method based on UAV imagery and multi-view recognition according to claim 6, characterized in that, The feature vector includes one or more of the following: multi-view centroid dispersion, axis length consistency, and reprojection viewpoint coverage; and / or The reconstructed three-dimensional geometric feature data of each target particle is parsed and projected into a two-dimensional contour in three-dimensional space. Then, it is precisely superimposed on the corresponding original two-dimensional image in a semi-transparent or wireframe form. By observing the boundary matching degree between the two-dimensional contour and the soil texture in the original two-dimensional image, the candidate samples are classified and labeled in two categories. The one-to-one correspondence between the two-dimensional labels and the feature vectors is recorded to generate a training dataset. If the 2D contour matches the boundary of the soil texture in the original 2D image, it is a real particle and is marked as a positive sample; if the 2D contour covers the vegetation texture, water surface, shadow area or area without clear entity texture, it is a disturbance and is marked as a negative sample.

8. A three-dimensional particle size analysis system based on UAV imagery and multi-view recognition, characterized in that, include: The data acquisition and modeling module is used to obtain image data of the target accumulation area, process the image data, and generate a three-dimensional sparse point cloud model of the target accumulation area and camera pose parameters for each image data. The 3D analysis box acquisition module is used to select an image from the image data as the source image, obtain the region of interest from the source image, and back-project the region of interest using the camera pose parameters and the 3D sparse point cloud model to generate a 3D analysis box. The two-dimensional image fragment module is used to traverse all images in the image data, automatically calculate and filter out the effective view images that observe the three-dimensional analysis frame, and then automatically crop out the corresponding two-dimensional image fragments from the effective view images. The two-dimensional detection result module is used to accurately identify the boundaries of particles in each two-dimensional image segment, obtain the target particles, and extract the two-dimensional detection results of the target particles. The 3D geometric feature data acquisition module accurately identifies the boundaries of particles on all 2D image fragments, projects them back into 3D space, and reconstructs the 3D geometric feature data of the target particles. The 3D particle swarm module is used to compare the 3D geometric feature data of the target particles with the 2D detection results, construct feature vectors, and input them into a pre-trained machine learning classification model based on random forest to obtain a 3D particle swarm.

9. A computer device, characterized in that, It includes a storage device and a processor, the storage device storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The device stores a computer program that, when executed by a processor, causes the processor to perform the steps of the method as described in any one of claims 1-7.

Images