Full-automatic rapid gravel screening device and method suitable for unmanned aerial vehicle

Abstract

The invention relates to a full-automatic rapid gravel screening device and method suitable for an unmanned aerial vehicle, and belongs to the technical field of accurate size calibration and image segmentation of images shot by the unmanned aerial vehicle. The screening device comprises an unmanned aerial vehicle, a transparent rope and a calibration object, one end of the transparent rope is connected to the unmanned aerial vehicle, the other end of the transparent rope is connected with the calibration object, and the length of the transparent rope meets the requirement that the calibration object can touch the ground and is kept within the camera view when the unmanned aerial vehicle shoots. The screening method comprises the following steps: 1) acquiring image data; 2) determining a pixel size; 3) generating a gravel contour; 4) segmenting the gravel image; and (5) gravel parameters are obtained. The accuracy and efficiency of particle size measurement can be greatly improved.

Description

Technical Field

[0001] The invention relates to a fully automatic rapid gravel screening device and method suitable for unmanned aerial vehicles, belonging to the technical field of accurate size calibration and image segmentation of images taken by unmanned aerial vehicles. Background Art

[0002] The acquisition of gravel particle size distribution occupies a core position in the study of coastal and mountain rivers, and is crucial to understanding natural phenomena and river dynamics. Traditional mechanical screening and individual measurement methods are accurate but time-consuming and labor-intensive, and it is difficult to meet the needs of large-scale data. In recent years, automatic particle size determination methods based on gravel characteristics have emerged, which are mainly divided into two categories: indirect determination and direct determination. Indirect determination methods such as look-up-catalog screening methods (LUC), spectral decomposition, and deep learning-based gravel screening methods (SediNet) can provide particle size information, but there are problems such as dependence on calibration, insufficient accuracy, or the need for specific data training. Direct determination methods such as MATLAB-based automatic object detection software (BASEGRAIN), pebble automatic counting methods (PebbleCountsAuto) and the method with the publication number CN118196655A can directly measure gravel characteristics, but face challenges such as the need for manual intervention in the image processing process and slow processing speed. In addition, with the popularization of drone technology, the acquisition of large-scale gravel image data has become convenient, but it also introduces the problem of pixel size deviation caused by changes in flight altitude, which affects the accuracy of particle size measurement.

[0003] Therefore, it is urgent to develop an automatic gravel particle size determination method that does not require human intervention in the processing process, does not rely on specific data sets, has high accuracy, high stability and fast processing speed. Summary of the invention

[0004] In order to solve the above-mentioned deficiencies in the prior art, a fully automatic rapid gravel automatic screening method suitable for drones is provided, aiming to solve the above-mentioned problems in the prior art and improve the accuracy and efficiency of particle size measurement.

[0005] The invention discloses a fully automatic rapid gravel screening device suitable for drones, which is special in that it comprises: a drone 1, a transparent rope 2, and a calibration object 3, wherein one end of the transparent rope 2 is connected to the drone 1, and the other end is connected to the calibration object 3, and the length of the transparent rope 2 should be such that the calibration object 3 can touch the ground and remain within the camera field of view when the drone 1 is shooting;

[0006] The calibration object 3 is spherical, red or blue in color, and has a diameter of 3 to 5 cm;

[0007] The drone 1 is a FIMI X8Pro or FIMI X8SE drone;

[0008] The length of the transparent rope 2 is determined by formula (1):

[0009]

[0010] Where: h is the shooting height, unit is m; l ropes is the length of the suspension rope, in meters; θ is half of the lens angle of view, in rad; r is the aspect ratio of the image sensor size.

[0011] The invention discloses a fully automatic rapid gravel screening method suitable for unmanned aerial vehicles, which is special in that it comprises the following steps:

[0012] 1. Image data acquisition

[0013] After the drone 1 flies to the set waypoint position according to the predetermined route, the calibration object 3 suspended below the drone 1 contacts the ground, and then image data is acquired;

[0014] 2. Determine the pixel size

[0015] First, the area that may contain the calibration object is initially screened out by the color of the calibration object; secondly, the area of ​​the calibration object is located by the size of the candidate area; thirdly, the positioning accuracy of the calibration object is further improved according to the calibration shape; finally, the single pixel size is finally determined according to the actual size of the calibration object and the pixel format within the range of the calibration object; the specific steps are as follows:

[0016] Step 1: Convert the collected image data from RGB format to HSV format, then perform color detection on the image to generate a detection image containing the calibration object;

[0017] Step 2: Simplify the detected image information, search and identify the contours of all image patches, and determine the contour of the largest image patch, which is the contour of the calibration object;

[0018] Step 3: Using all pixel coordinate values ​​within the calibration object contour determined in step 2, calculate the center coordinates and radius of the calibration object according to formula (2);

[0019]

[0020] Where: x c ,y c is the coordinate of the center point of the calibration object, pix; x i ,y i The coordinates of the pixel points within the contour range of the calibration object, pix; m is the number of pixel points within the contour range of the calibration object; Rc is the radius of the calibration object;

[0021] Step 4: According to the center coordinates and radius of the calibration object calculated by formula (2), the area of ​​the calibration object is segmented in the image and the single pixel size is calculated according to formula (3) based on the actual size of the calibration object;

[0022]

[0023] Where: l pix is the actual size of a single pixel, in mm / pix; is the area of ​​the calibration object, in mm 2 The default value is 1256mm 2 .

[0024] Preferably, the detection image described in step 1 not only contains the calibration object itself, but also is accompanied by a large amount of noise, which requires further processing to improve the image quality. Due to the influence of light, a large number of noise patches will appear in the process of color detection of the calibration object, but the area of ​​a single noise patch is smaller than that of the calibration object. Therefore, all noise can be filtered out based on this point.

[0025] Preferably, in step 2, binarization processing is used to simplify the detection image information;

[0026] 3. Generate gravel outline

[0027] The local threshold method and Canny edge detection method are used to accurately identify the gravel contour; the gravel contour is initially outlined by the local threshold method, and the fine texture inside the gravel is removed; with the help of the sharp edge detection ability of the Canny edge detection method, the above contour is further refined and enhanced; the foreground and background patch suppression algorithm is used to remove redundant image patches to ensure the clarity and accuracy of the contour;

[0028] Preferably, the removal of fine texture inside the gravel adopts the eight-connected region suppression algorithm, which first divides all pixels in the image into different connected region patches according to eight neighborhoods; then sets a filtering threshold, the default threshold is 800; finally, all connected region patches in the image are filtered according to the set threshold to achieve the effect of suppressing noise. Preferably, in the foreground and background patch suppression algorithm, the foreground refers to the gravel, and the background refers to the boundary of the gravel. The algorithm first processes the image synthesized by the local threshold method and the Canny edge detection method according to the foreground and background by the eight-connected region suppression algorithm, with the foreground being white and the background being black;

[0029] Preferably, the local threshold method for generating gravel contours includes three steps: determining the window size, determining the Niblack local threshold, and filtering texture noise; the specific steps are as follows:

[0030] Step 1: Determine the window size: When selecting the local window size, we must ensure that the window can completely span the boundary area of ​​the gravel, and avoid the window being too large to increase unnecessary computing burden. In order to balance these two factors, a strategy is adopted: when there is a calibration object in the image, the number of pixels within the radius of the calibration object is used as the size of the local window; if there is no calibration object in the image range, the number of pixels spanned by the length of 20mm is used as the window length by default;

[0031] Step 2: Determine the Niblack local threshold: Determine the mean and standard deviation of the pixels in the local window, and further calculate the Niblack local threshold based on the mean and standard deviation in these windows:

[0032] T=M-kS(4)

[0033] Where: M is the mean of the pixels in the window; S is the standard deviation in the window; k is a constant used to control the sensitivity of the threshold to the standard deviation, and the default value is 0.05;

[0034] Step 3: Texture noise filtering: Morphological dilation is used to separate the boundary and internal textures, and the eight-connected region suppression algorithm is used to filter out the texture noise inside the gravel;

[0035] The eight-connected region suppression algorithm first divides all pixels in the image into different connected region patches according to eight neighborhoods; then sets a filtering threshold, which defaults to 800; finally, all connected region patches in the image are filtered according to the set threshold to achieve the effect of suppressing noise.

[0036] Preferably, the specific steps of generating gravel contours using the Canny edge detection method are:

[0037] The Canny edge detection method requires a high threshold and a low threshold to detect the edge more accurately. Formula (5) is used to generate a suitable threshold for the Canny edge detection method, thereby simplifying the parameter adjustment process.

[0038]

[0039] Where: v up ,v down is the upper and lower thresholds of the Canny edge detection method; k down ,k up The upper and lower magnification coefficients of the Otsu threshold, the default value is: k down =0.5, k up =1.5; K down ,K up It is the upper and lower control thresholds of the Canny edge detection method. The default value is K down =150, K up =255.

[0040] 4. Segmentation of gravel image

[0041] Gravel image segmentation includes four steps: seed point generation, maximum search area determination, pixel patch search and judgment; seed point generation determines where to start searching for gravel pixels and ensures that gaps and other areas that are prone to misjudgment are avoided; during the search process, determining the maximum search area can effectively control the under-segmentation and over-segmentation problems of gravel; pixel patch search and judgment can refine the details of the gravel boundary and improve the accuracy of segmentation;

[0042] Preferably, the seed point generation specifically includes the following steps:

[0043] The position of the seed point is limited to a certain range near the center of its influence area (within the gravel area, a random point is generated, and then extended from this point in four directions, up, down, left and right, until it touches the nearest gravel boundary. The rectangle determined by four orthogonal lines is the influence range of the random point. When the random point is located in the area with a radius of one-quarter of the length of the short side of the rectangle from the center of the influence area, the point is the seed point), ensuring that the generated polygon can more accurately reflect the shape characteristics of the gravel contour; the entire image is evenly divided into a corresponding number of areas according to the preset number of seed points. In each area, candidate points are randomly generated and screened according to the above seed point judgment criteria.

[0044] Preferably, the specific steps of determining the maximum search area are:

[0045] When determining the maximum search area for gravel, first take the influence area of ​​the seed point (extend straight lines from the seed point in the four directions of up, down, left, and right until they touch the nearest gravel boundary, and the rectangle determined by the four orthogonal lines is the influence area of ​​the seed point), and generate a square area at the centroid position of the influence area, whose side length is equal to half the length of the short side of the influence area; then, this square area is called the centroid influence area, and a series of points are generated in the area, which are defined as the control points for generating the vertices of the boundary polygon;

[0046] In order to form the boundary control polygon, rays are emitted from each control point along eight neighborhood directions until they intersect with the boundary; the intersection points are sorted according to the distance between the control points, and the nearest points (the first four by default) are selected as the vertices of the polygon. This process is repeated at each control point in the centroid influence area to obtain all relevant polygon vertices; finally, the polygon area connected by these vertices is the maximum search area for gravel. The accuracy and effectiveness of the search area are ensured by the generation of precise control points and the selection of boundary intersection points.

[0047] Preferably, the specific steps of searching and determining pixel patches are:

[0048] The search process of gravel images is performed with pixel patches as the basic unit;

[0049] A pixel patch is a set consisting of a central pixel and its eight neighboring pixels. The search process starts with a seed point. From this point, all pixel patches in the four neighborhoods of the seed are searched and added to the set to be detected. Subsequently, each pixel patch to be tested in this set is evaluated, and the pixel patch identified as gravel will become the starting point for the next round of search. As the search progresses, the pixel patches to be tested will gradually approach the boundary, and the number of pixel patches to be tested in each round will also decrease accordingly as the number of pixel patches on the gravel boundary increases. When the number of starting pixel patches decreases to zero, the search process will terminate.

[0050] The specific steps of the determination are:

[0051] The gravel pixel patch judgment function meets the following two requirements: on the one hand, the pixel patches in the vicinity of the seed point must belong to the gravel pixel patches. When the pixel patches are located within the relative influence distance β near the seed point, no matter how many pixels in the pixel patches belong to the gravel color domain, they are all identified as gravel pixel patches. On the other hand, as the distance between the pixel patches and the seed point increases, the possibility that the pixel patches belong to the gravel boundary gradually increases. When judging the pixel patch ownership, the judgment weight of the relative influence distance gradually decreases, while the judgment weight of the color domain gradually increases. Formula (6) is the gravel pixel patch judgment function:

[0052]

[0053] Where: Ψ is the gravel pixel patch judgment function; α is the relative distance factor magnification coefficient, the default value is 5; β is the relative influence distance, the default value is 0.4; w is half of the minimum of the length and width of the seed point scope rectangle; x, y are the center coordinates of the pixel patch to be tested, unit pix; x s ,y s is the coordinate of the gravel seed point, in pix; m is the number of pixels in the pixel patch to be tested that belongs to the gravel color domain space; n is the number of pixels in the pixel patch, and the default value is 9.

[0054] 5. Get gravel parameters

[0055] According to the absolute position of the center point of the pixel patch in the image, all pixel patches are integrated into a gravel image. Then, an ellipse is used to fit the outer contour of the gravel image, and the length of the minor axis of the ellipse is the b-axis length of the gravel; finally, the b-axis lengths of the gravels are arranged in ascending order, and based on the actual area of ​​each gravel and the total area of ​​all gravels, the mass fraction of the corresponding particle size on the cumulative curve is determined using formula (7):

[0056]

[0057] Where: p i Arrange the mass fraction of the i-th gravel in ascending order; A k The area corresponding to the kth gravel in ascending order is the area of ​​the gravel divided out, in mm 2 ; n is the total number of gravels divided.

[0058] The present invention provides a fully automatic rapid gravel screening device and method suitable for unmanned aerial vehicles, and the beneficial effects are as follows:

[0059] 1. High-precision particle size determination: The present invention significantly improves the accuracy of pixel size calibration by introducing calibration objects of specific colors and sizes as calibration objects and combining them with accurate pixel size calibration methods.

[0060] 2. Fully automatic processing flow: The present invention realizes a fully automatic processing flow for gravel image segmentation without manual intervention. Traditional methods such as BASEGRAIN, PebbleCountsAuto and the method with publication number: CN118196655A can directly measure gravel features, but still require manual intervention during the processing, which is inefficient. The fully automatic processing flow of the present invention greatly improves work efficiency and is suitable for large-scale data processing scenarios.

[0061] 3. Wide applicability and stability: The method of the present invention does not rely on a specific data set, has strong generalization ability, and can process complex image data of different times, seasons and tidal flat types. Experimental results show that under different lighting conditions and background environments, the method of the present invention can maintain high accuracy and stability, indicating that it has strong robustness.

[0062] 4. Fast processing speed: Compared with other automatic processing methods, the present invention has a faster processing speed while ensuring high accuracy. Although it is slightly slower than the SediNet method based on neural networks, it is better than BASEGRAIN and PebbleCountsAuto. At the same time, the accuracy of the present method is much higher than that of the SediNet method, and it supports batch processing of large-scale images, which improves the overall processing efficiency. This is especially important for application scenarios that require rapid acquisition of particle size distribution data.

[0063] 5. Simplified operation process: The present invention reduces the user's usage threshold by simplifying the operation process. The application of drones makes data acquisition more convenient, and the subsequent data processing process is also optimized through automation. Users only need to simply set the drone route to complete data collection and processing, which greatly improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] Figure 1 : Schematic diagram of drone data collection and route planning;

[0066] Figure 2 : A flowchart of the steps of the present invention;

[0067] Figure 3 : Schematic diagram of pixel patch search and determination;

[0068] Figure 4 : Gravel images taken by drones at different times, seasons and types of tidal flats;

[0069] Figure 5 :Use the method of the present invention and the manual marking method to obtain Figure 4 The gravel accumulation curve of the 35 images in the middle;

[0070] Figure 6 : The cumulative curves of gravel determined by manual labeling method, AIF, BASEGRAIN, SediNet and the method of the present invention;

[0071] Figure 7 : Normalized root mean square error diagram of PebbleCountsAuto, BASEGRAIN, SediNet and the method of the present invention determined based on the manual labeling method;

[0072] Figure 8 : Comparison of characteristic particle size values ​​obtained by digital measurement method and manually marked characteristic particle size values, a total of 35 sets of image sampling and analysis were conducted; the black dotted line is the 1:1 trend line;

[0073] Fig. 9 :The distribution of time used to analyze 35 groups of pictures using digital measurement method;

[0074] Fig.10: Manual marking method, Publication No.: CN118196655A and the gravel cumulative curve graph determined by the method of the present invention;

[0075] Fig.11 : Normalized root mean square error diagram of the method of publication number: CN118196655A determined based on the manual marking method and the method of the present invention;

[0076] Fig.12 : Publication No.: CN118196655A method and the comparison diagram of the characteristic particle size values ​​obtained by the method of the present invention and the characteristic particle size values ​​marked manually, a total of 35 groups of image sampling and analysis diagrams were performed; the black dotted line is the 1:1 trend line;

[0077] Fig.13 : A comparison chart of the time distribution used by the method of CN118196655A and the method of the present invention to analyze 35 groups of pictures. DETAILED DESCRIPTION

[0078] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It will be appreciated that the specific embodiments described herein are only used to explain the relevant inventions and are not intended to limit the invention. It should also be noted that, for ease of description, only portions related to the relevant inventions are shown in the accompanying drawings. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0079] Example 1

[0080] The invention discloses a fully automatic rapid gravel screening device suitable for an unmanned aerial vehicle, comprising: an unmanned aerial vehicle 1, a transparent rope 2, and a calibration object 3, wherein one end of the transparent rope 2 is connected to the unmanned aerial vehicle 1, and the other end is connected to the calibration object 3, and the length of the transparent rope 2 should be such that the calibration object 3 can touch the ground and remain within the camera field of view when the unmanned aerial vehicle 1 is shooting; the calibration object 3 is spherical, red or blue in color, and has a diameter of 3 to 5 cm;

[0081] The drone 1 is a FIMI X8Pro or FIMI X8SE drone;

[0082] The length of the transparent rope 2 is determined by formula (1):

[0083]

[0084] Where: h is the shooting height, unit is m; l ropes is the length of the suspension rope, in meters; θ is half of the lens angle of view, in rad; r is the aspect ratio of the image sensor size.

[0085] The invention discloses a fully automatic rapid gravel screening method suitable for unmanned aerial vehicles, which is special in that it comprises the following steps:

[0086] 1. Image data acquisition

[0087] After the drone 1 flies to the set waypoint according to the predetermined route, the calibration object 3 suspended below the drone 1 contacts the ground, and then image data is acquired. The collected data is transmitted to the near-end data storage device via the onboard Bluetooth;

[0088] 2. Determine the pixel size

[0089] First, the area that may contain the calibration object is preliminarily screened out by the color of the calibration object; secondly, the area of ​​the calibration object is located by the size of the candidate area; thirdly, the positioning accuracy of the calibration object is further improved according to the calibration shape; finally, the single pixel size is finally determined according to the actual size of the calibration object and the number of pixels within the range of the calibration object; the specific steps are as follows:

[0090] Step 1: Convert the collected image data from RGB format to HSV format, then perform color detection on the image to generate a detection image containing the calibration object;

[0091] Step 2: Simplify the detected image information, search and identify the contours of all image patches, and determine the contour of the largest image patch, which is the contour of the calibration object;

[0092] Step 3: Using all pixel coordinate values ​​within the calibration object contour determined in step 2, calculate the center coordinates and radius of the calibration object according to formula (2);

[0093]

[0094] Where: x c ,y c is the coordinate of the center point of the calibration object, in units of pix; i ,y i The coordinates of the pixel points within the contour range of the calibration object, in units of pix; m is the number of pixels within the contour range of the calibration object; R c is the radius of the calibration object, in pix;

[0095] Step 4: According to the center coordinates and radius of the calibration object calculated by formula (2), the area of ​​the calibration object is segmented in the image and the single pixel size is calculated according to formula (3) based on the actual size of the calibration object;

[0096]

[0097] Where: l pixis the actual size of a single pixel, in mm / pix; is the area of ​​the calibration object, in mm 2 The default value is 1256mm 2 .

[0098] Preferably, the detection image described in step 1 not only contains the calibration object itself, but also is accompanied by a large amount of noise, which requires further processing to improve the image quality. Due to the influence of light, a large number of noise patches will appear in the process of color detection of the calibration object, but the area of ​​a single noise patch is smaller than that of the calibration object, so all noise is filtered out based on this point.

[0099] Preferably, in step 2, binarization processing is used to simplify the detection image information;

[0100] 3. Generate gravel outline

[0101] The local threshold method and Canny edge detection method are used to accurately identify the gravel contour; the gravel contour is initially outlined by the local threshold method, and the fine texture inside the gravel is removed; with the help of the sharp edge detection ability of the Canny edge detection method, the above contour is further refined and enhanced; the foreground and background patch suppression algorithm is used to remove redundant image patches to ensure the clarity and accuracy of the contour;

[0102] The eight-connected region suppression algorithm is used to remove the fine texture inside the gravel. The algorithm first divides all pixels in the image into different connected region patches according to eight neighborhoods; then sets a filtering threshold, which defaults to 800; finally, all connected region patches in the image are filtered according to the set threshold to achieve the effect of suppressing noise.

[0103] In the foreground and background patch suppression algorithm, the foreground refers to the gravel, and the background refers to the boundary of the gravel. The algorithm first processes the image synthesized by the local threshold method and the Canny edge detection method according to the foreground and background by the eight-connected region suppression algorithm, where the foreground is white and the background is black;

[0104] The local threshold method for generating gravel contours includes three steps: determining the window size, determining the Niblack local threshold, and filtering texture noise; the specific steps are as follows:

[0105] Step 1: Determine the window size: When selecting the local window size, we must ensure that the window can completely span the boundary area of ​​the gravel, and avoid the window being too large to increase unnecessary computing burden. In order to balance these two factors, a strategy is adopted: when there is a calibration object in the image, the number of pixels within the radius of the calibration object is used as the size of the local window; if there is no calibration object in the image range, the number of pixels spanned by the length of 20mm is used as the window length by default;

[0106] Step 2: Determine the Niblack local threshold: Determine the mean and standard deviation of the pixels in the local window, and further calculate the Niblack local threshold based on the mean and standard deviation in these windows:

[0107] T=M-kS(4)

[0108] Where: M is the mean of the pixels in the window; S is the standard deviation in the window; k is a constant used to control the sensitivity of the threshold to the standard deviation, and the default value is 0.05;

[0109] Step 3: Texture noise filtering: Morphological dilation is used to separate the boundary and internal textures, and the eight-connected region suppression algorithm is used to filter out the texture noise inside the gravel;

[0110] The eight-connected region suppression algorithm first divides all pixels in the image into different connected region patches according to eight neighborhoods; then sets a filtering threshold, which defaults to 800; finally, all connected region patches in the image are filtered according to the set threshold to achieve the effect of suppressing noise.

[0111] Preferably, the specific steps of generating gravel contours using the Canny edge detection method are:

[0112] The Canny edge detection method requires a high threshold and a low threshold to detect the edge more accurately. Formula (5) is used to generate a suitable threshold for the Canny edge detection method, thereby simplifying the parameter adjustment process.

[0113]

[0114] Where: v up ,v down is the upper and lower thresholds of the Canny edge detection method; k down ,k up The upper and lower magnification coefficients of the Otsu threshold, the default value is: k down =0.5, k up =1.5; K down ,K up It is the upper and lower control thresholds of the Canny edge detection method. The default value is K down =150, K up =255.

[0115] 4. Segmentation of gravel image

[0116] Gravel image segmentation includes four steps: seed point generation, maximum search area determination, pixel patch search and judgment; seed point generation determines where to start searching for gravel pixels and ensures that gaps and other areas that are prone to misjudgment are avoided; during the search process, determining the maximum search area can effectively control the under-segmentation and over-segmentation problems of gravel; pixel patch search and judgment can refine the details of the gravel boundary and improve the accuracy of segmentation;

[0117] The seed point generation specifically includes the following steps:

[0118] The position of the seed point is limited to a certain range near the center of its influence area (within the gravel area, a random point is generated, and then extended in the four directions of up, down, left and right from this point until it touches the nearest gravel boundary. The rectangle determined by the four orthogonal lines is the influence range of the random point. When the random point is located in the area with a radius of one-quarter of the length of the short side of the rectangle from the center of the influence area, the point is the seed point), ensuring that the generated polygon can more accurately reflect the shape characteristics of the gravel contour; the entire image is evenly divided into a corresponding number of areas according to the preset number of seed points. In each area, candidate points are randomly generated and screened according to the above seed point judgment criteria.

[0119] The specific steps of determining the maximum search area are:

[0120] When determining the maximum search area for gravel, first use the influence area of ​​the seed point ( Figure 3 Based on the rectangle in (a) (a random point is generated within the gravel area, and then extended in the four directions of up, down, left and right from this point until it touches the nearest gravel boundary, then the rectangle determined by the four orthogonal lines is the influence area of ​​the random point), a square area is generated at the centroid position of the influence area, and its side length is equal to half the length of the short side of the influence area; then, this square area is called the centroid influence area, and a series of points are generated in this area (such as Figure 3 (b)) (a square area is generated at the centroid position of the affected area, and its side length is equal to half of the short side length of the affected area), and these points are defined as control points for generating the vertices of the boundary polygon;

[0121] In order to form the boundary control polygon, rays are emitted from each control point along eight neighborhood directions until they intersect with the boundary; the intersection points are sorted according to the distance between the control points, and the nearest points (the first four by default) are selected (arranged in ascending order according to the distance between the intersection points and the control points, and the first four are taken) as the vertices of the polygon. This process is repeated at each control point in the control area to obtain all relevant polygon vertices; finally, the polygon area connected by these vertices is the maximum search area for gravel. The accuracy and effectiveness of the search area are ensured by the generation of precise control points and the selection of boundary intersection points.

[0122] Preferably, the specific steps of searching and determining pixel patches are:

[0123] The search process of gravel images is performed with pixel patches as the basic unit.

[0124] A pixel patch is a set consisting of a central pixel and its eight neighboring pixels. The search process starts with a seed point. From this point, all pixel patches in the four neighborhoods of the seed are searched and added to the set to be detected. Subsequently, each pixel patch to be tested in this set is evaluated, and the pixel patch identified as gravel will become the starting point for the next round of search. As the search progresses, the pixel patches to be tested will gradually approach the boundary, and the number of pixel patches to be tested in each round will also decrease accordingly as the number of pixel patches on the gravel boundary increases. When the number of starting pixel patches decreases to zero, the search process will terminate.

[0125] The specific steps of the determination are:

[0126] The gravel pixel patch judgment function meets the following two requirements: on the one hand, the pixel patches in the vicinity of the seed point must belong to the gravel pixel patches. When the pixel patches are located within the relative influence distance β near the seed point, no matter how many pixels in the pixel patches belong to the gravel color domain, they are all identified as gravel pixel patches. On the other hand, as the distance between the pixel patches and the seed point increases, the possibility that the pixel patches belong to the gravel boundary gradually increases. When judging the pixel patch ownership, the judgment weight of the relative influence distance gradually decreases, while the judgment weight of the color domain gradually increases. Formula (6) is the gravel pixel patch judgment function:

[0127]

[0128] Where: Ψ is the gravel pixel patch judgment function; α is the relative distance factor magnification coefficient, the default value is 5; β is the relative influence distance, the default value is 0.4; w is half of the minimum of the length and width of the seed point scope rectangle; x, y are the center coordinates of the pixel patch to be tested, unit pix; x s ,ys is the coordinate of the gravel seed point, in pix; m is the number of pixels in the pixel patch to be tested that belongs to the gravel color domain space; n is the number of pixels in the pixel patch, and the default value is 9.

[0129] 5. Get gravel parameters

[0130] According to the absolute position of the center point of the pixel patch in the image, all pixel patches are integrated into a gravel image; then, an ellipse is used to fit the outer contour of the gravel image, and the length of the minor axis of the ellipse is the b-axis length of the gravel; finally, the b-axis lengths of the gravels are arranged in ascending order, and according to the actual area of ​​each gravel and the total area of ​​all gravels, the mass fraction of the corresponding particle size on the cumulative curve is determined using formula (7):

[0131]

[0132] Where: p i The mass fraction of the i-th gravel is arranged in ascending order; A k Arrange the divided gravels in ascending order, and the area corresponding to the kth gravel in mm 2 ; n is the total number of gravels divided.

[0133] In order to verify the accuracy and stability of the method of the present invention, 35 photos taken in different regions, times and lighting conditions were selected (such as Figure 4 These photos cover a wide range of complex shooting environments, aiming to systematically evaluate the performance of the method of the present invention. Specifically, the selected photos include scenes covered by ice, snow, vegetation, or green moss, shadows, insufficient lighting, and complex situations with significant differences in color and particle size. These complex shooting environments fully represent the various challenges that may be encountered in practical applications. Figure 4 Among them, (1) to (23) show photos of coastal mudflats, and (24) to (35) show scenes of river mudflats. Through these photos, the accuracy and stability of the method of the present invention in various practical scenarios are objectively and comprehensively evaluated.

[0134] Table 1 Normalized root mean square error of the cumulative curve obtained by the method of the present invention relative to the cumulative curve obtained by the manual marking method

[0135]

[0136] Miazza et al. (2024) and others confirmed in their research that the particle size obtained by manual marking with ImageJ software (hereinafter referred to as the manual marking method) is highly consistent with the on-site manual measurement results. Therefore, the present invention adopts the manual marking method as a benchmark for measuring the accuracy of other methods. In addition, in order to quantitatively evaluate the cumulative curve error based on digital particle size analysis, the present invention adopts the normalized root mean square error (NRMSE) (see formula (8)). This indicator compares the image gravel cumulative curve obtained by the manual marking method and its corresponding average particle size with the method to be tested, thereby estimating the accuracy of the method to be tested. The formula for the normalized root mean square error is as follows:

[0137]

[0138] Where: D i is the particle size content of the ith particle size point in the cumulative curve determined by the manual marking method, R i is the particle size content of the ith particle size point in the cumulative curve determined by the comparison method, and n is the number of divisions of the cumulative curve.

[0139] according to Figure 5 From the analysis results shown, it can be clearly observed that when the cumulative frequency analysis of 35 images is performed, the result trend of the method of the present invention is highly consistent with that of the manual identification method. Combined with the normalized root mean square error data listed in Table 1, it is not difficult to find that the normalized root mean square error of most images is maintained below 0.3. This data provides conclusive evidence for the accuracy of the method of the present invention. Therefore, it can be concluded that the results obtained by the method of the present invention are reliable.

[0140] In order to evaluate the accuracy and stability of the method of the present invention, three methods, PebbleCountsAuto, BASEGRAIN and SediNet, were selected as references. In the performance comparison, the manual labeling method was established as the evaluation benchmark. The comprehensive experimental results show that the data trends of the method of the present invention and the manual labeling method show a high consistency (see Figure 6 ).

[0141] When comparing the performance of different methods, it was found that the cumulative curve deviation of the SediNet method was the largest, and its results tended to overestimate the particle size. The results of the BASEGRAIN method also had a large deviation, which was mainly manifested in over-segmentation of the image. Relatively speaking, the main problem of the PebbleCountsAuto method was under-segmentation, especially when the particle sizes were similar and the texture features were similar.

[0142] In addition, through Figure 7The normalized root mean square error data shown further quantifies the performance differences of each method. Specifically, the normalized root mean square error of the PebbleCountsAuto method ranges from 0.24 to 0.98, the error range of the BASEGRAIN method is between 0.35 and 2.33, and the error of the SediNet method is as high as 1.16 to 9.13. In contrast, the method of the present invention shows a significant advantage, with a normalized root mean square error of only 0.07 to 0.58, which shows that the method of the present invention has the best performance in terms of image segmentation accuracy.

[0143] In summary, by comparing the accuracy and stability of different automatic segmentation methods, the superiority of the method of the present invention in particle size image analysis is verified. Its segmentation results are highly consistent with the manual labeling method, and the normalized root mean square error is significantly lower than that of other reference methods, so it has significant advantages in the task of particle size image segmentation.

[0144] Through Figure 8 In-depth analysis shows that different methods have significant differences in the performance of characteristic particle size classification. 50 The PebbleCountsAuto method obviously suffers from under-segmentation problems when the characteristic particle size is larger than d. This phenomenon is mainly due to the presence of multiple joints, long strips or branched textures inside larger gravels, which leads to the erroneous formation of multiple cluster centers inside a single gravel. 50 The performance of this method is improved when , which is mainly due to the fact that it splits the larger particles into medium-sized particles.

[0145] Similar to the PebbleCountsAuto method, the BASEGRAIN method also has segmentation problems, and its over-segmentation problem is more serious. This is mainly because the watershed algorithm used in this method is extremely sensitive to weak boundaries. Due to the complexity and changeability of gravel texture, a large amount of noise is introduced, which is mistakenly identified as gravel boundaries by the watershed algorithm, resulting in significant over-segmentation when segmenting larger gravels.

[0146] In contrast, the SediNet method has a 50 The performance was relatively good in the previous partition. However, in d 50 After that, the method tends to overestimate the size of the gravel. This overestimation can be attributed to the characteristics of the neural network algorithm on which the SediNet method is based. The results are affected by the number of hidden layers in the neural network, the number of perceptrons, and the complexity of the learning task, which leads to the underfitting problem of the method when processing new gravel images due to "insufficient learning".

[0147] Finally, the overall performance of the method of the present invention is consistent with the manual identification method, which shows that the method of the present invention has certain stability and reliability in dealing with gravel particle size classification tasks.

[0148] When the time consumption of the four gravel segmentation methods is counted, although the manual statistics may lead to certain deviations due to differences in operators, overall these data can still effectively reflect the time consumption characteristics of various methods.

[0149] according to Fig. 9 From the data shown, it can be clearly observed that the SediNet method takes the shortest time among the four methods, and the expected time is only 10 seconds. The second is the method of the present invention, which is expected to take 28 seconds. The average time of the BASEGRAIN method is relatively long, which is 73 seconds. The longest time is the PebbleCountsAuto method, which is expected to take 111 seconds. This is mainly because the PebbleCountsAuto method includes multiple manual intervention links during the execution process, which significantly increases the overall processing time. It is also worth noting that the BASEGRAIN method does not have batch processing capabilities, which means that more repetitive manual operations are required when processing large-scale images, increasing the workload. In contrast, although the method of the present invention is slightly inferior to SediNet in terms of time consumption, it is better than BASEGRAIN, and can perform batch processing of large-scale images, thereby improving processing efficiency. Compared with the SediNet method based on neural network, the method of the present invention is slightly slower than the SediNet method in terms of speed, but the accuracy of the method of the present invention is much higher than that of the SediNet method.

[0150] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications and substitutions are all included in the scope defined by the claims of this application.

[0151] Example 2

[0152] In order to accurately evaluate the improvement effect of the method of the present invention on the existing problems of the method of Publication (Announcement) No.: CN118196655A in the field of gravel segmentation (such as the need for manual intervention, insufficient segmentation speed and accuracy), we selected a wide range of data sets, including 35 photos taken from different environmental conditions, and systematically tested the two methods. In this performance comparison, we used the manual labeling method as the benchmark for judging accuracy. The experimental results show that the method of the present invention is highly consistent with the data trend of the manual labeling (see Fig.10 ), showing excellent segmentation consistency.

[0153] Specifically, when comparing the performance of the two, we found that the method of publication number CN118196655A performed well in the low percentile cumulative curve part, but showed obvious deficiencies in the high percentile area, which highlights the significant advantages of the present invention in reducing the phenomenon of gravel over-segmentation.

[0154] Further through Fig.11 The advantages of the method of the present invention are particularly obvious in the quantitative analysis of the normalized root mean square error data in the publication (publication) No. CN118196655A. Compared with the error rate of up to 166% in the method of the publication (publication) No. CN118196655A, the method of the present invention only achieves an error rate as low as 0.58%, which fully proves its excellent performance in image segmentation accuracy.

[0155] In-depth analysis Fig.12 The stability and reliability of the method of the present invention in the characteristic particle size classification are obvious. Especially when processing gravel with a characteristic particle size greater than d50, the method of the present invention effectively avoids the over-segmentation problem caused by the complex texture inside the gravel, and its performance is almost the same as the manual marking method, demonstrating its ability to handle complex scenes.

[0156] Finally, in terms of processing speed (see Fig.13 ), the method of the present invention also performs well. Its average processing time is only 28 seconds, compared with 1260 seconds of the manual labeling method and 1018 seconds of the method with publication number CN118196655A, which achieves significant efficiency improvement and meets the demand for fast processing in practical applications. In summary, the method of the present invention has achieved comprehensive optimization of traditional methods in segmentation accuracy, stability and processing speed.

Claims

1. A fully automatic rapid gravel screening device suitable for drones, comprising: Drone, transparent rope, calibration object. One end of the transparent rope is connected to the drone, and the other end is connected to the calibration object. The length of the transparent rope should be enough for the calibration object to touch the ground and remain within the camera's field of view when the drone is shooting. The calibration object is spherical, red or blue in color, and has a diameter of 3 to 5 cm; The length of the transparent rope is determined by formula (1): Where: h is the shooting height, unit is m; l ropes is the length of the suspension rope, in meters; θ is half of the lens angle of view, in rad; r is the aspect ratio of the image sensor size.

2. A fully automatic rapid gravel screening method suitable for drones, characterized in that The following steps are involved: 1) Image data acquisition After the drone flies to the set waypoint according to the predetermined route, it lowers the drone's flight altitude so that the calibration object hanging under the drone touches the ground. Then, the image data is acquired and transmitted to the near-end storage device via the onboard Bluetooth for subsequent processing. 2) Determine pixel size First, the area that may contain the calibration object is preliminarily screened out by the color of the calibration object; secondly, the area of ​​the calibration object is located by the size of the candidate area; thirdly, the positioning accuracy of the calibration object is further improved according to the calibration shape; finally, the single pixel size is finally determined according to the actual size of the calibration object and the number of pixels within the range of the calibration object; 3) Generate gravel profile The local threshold method and Canny edge detection method are used to accurately identify the gravel contour; the gravel contour is initially outlined by the local threshold method, and the fine texture inside the gravel is removed; with the help of the sharp edge detection ability of the Canny edge detection method, the above contour is further refined and enhanced; the foreground and background patch suppression algorithm is used to remove redundant image patches to ensure the clarity and accuracy of the contour; 4) Segmentation of gravel image Gravel image segmentation includes four steps: seed point generation, maximum search area determination, pixel patch search and judgment; the generation of seed points determines where to start searching for gravel pixels and ensures that gaps and other areas that are prone to misjudgment are avoided; During the search process, determining the maximum search area can effectively control the under-segmentation and over-segmentation problems of gravel; searching and determining pixel patches can refine the details of gravel boundaries and improve the accuracy of segmentation; 5) Get gravel parameters According to the absolute position of the center point of the pixel patch in the image, all pixel patches are integrated into a gravel image. Then, an ellipse is used to fit the outer contour of the gravel image. The length of the minor axis of the ellipse is the b-axis length of the gravel. Finally, the b-axis lengths of the gravels are arranged in ascending order, and according to the actual area of ​​each gravel and the total area of ​​all gravels, the mass fraction of the corresponding particle size on the cumulative curve is determined using formula (7): Where: p i The mass fraction of the i-th gravel is arranged in ascending order; A k Arrange the divided gravels in ascending order, and the area corresponding to the kth gravel in mm 2 ; n is the total number of gravels divided.

3. A fully automatic rapid gravel screening method suitable for drones according to claim 2, characterized in that Step 2) The specific steps to determine the pixel size are as follows: Step 1: Convert the collected image data from RGB format to HSV format, then perform color detection on the image to generate a detection image containing the calibration object; Step 2: Simplify the detected image information, search and identify the contours of all image patches, and determine the largest image patch contour, which is the calibration object contour; Step 3: Using all pixel coordinate values ​​within the calibration object contour determined in step 2, calculate the center coordinates and radius of the calibration object according to formula (2); Where: x c ,y c is the coordinate of the center point of the calibration object, in units of pix; i ,y i The coordinates of the pixel points within the contour range of the calibration object, in units of pix; m is the number of pixels within the contour range of the calibration object; R c The radius of the calibration object in the image, in pix; Step 4: According to the center coordinates and radius of the calibration object calculated by formula (2), the area of ​​the calibration object is segmented in the image and the single pixel size is calculated according to formula (3) based on the actual size of the calibration object; Where: l pix is the actual size of a single pixel, in mm / pix; is the area of ​​the calibration object, in mm 2 , the default value is 1256mm 2 .

4. A fully automatic rapid gravel screening method suitable for drones according to claim 3, characterized in that: The detection image described in step 1) not only contains the calibration object itself, but also is accompanied by a large amount of noise, which requires further processing to improve the image quality. Due to the influence of light, a large number of noise patches will appear in the process of color detection of the calibration object, but the area of ​​a single noise patch is smaller than that of the calibration object, so all noise is filtered out based on this point; in step 2), binarization processing is used to simplify the detection image information.

5. A fully automatic rapid gravel screening method suitable for drones according to claim 2, characterized in that The local threshold method in step 3) for generating gravel contours includes three steps: determining the window size, determining the Niblack local threshold, and filtering texture noise; the specific steps are as follows: Step 1: Determine the window size: When selecting the local window size, we must ensure that the window can completely span the boundary area of ​​the gravel, and avoid the window being too large to increase unnecessary computing burden. In order to balance these two factors, a strategy is adopted: when there is a calibration object in the image, the number of pixels within the radius of the calibration object is used as the size of the local window; if there is no calibration object in the image range, the number of pixels spanned by the length of 20mm is used as the window length by default; Step 2: Determine the Niblack local threshold: Determine the mean and standard deviation of the pixels in the local window, and further calculate the Niblack local threshold based on the mean and standard deviation in these windows: T=M-kS (4) Where: M is the mean of the pixels in the window; S is the standard deviation in the window; k is a constant used to control the sensitivity of the threshold to the standard deviation, and the default value is 0.05; Step 3: Texture noise filtering: Morphological dilation is used to separate the boundary and internal textures, and the eight-connected region suppression algorithm is used to filter out the texture noise inside the gravel.

6. A fully automatic rapid gravel screening method suitable for drones according to claim 5, characterized in that The specific steps of generating gravel contours using the Canny edge detection method described in step 3) are: The Canny edge detection method requires a high threshold and a low threshold to detect the edge more accurately. Formula (5) is used to generate a suitable threshold for the Canny edge detection method, thereby simplifying the parameter adjustment process. Where: v up ,v down is the upper and lower thresholds of the Canny edge detection method; k down ,k up The upper and lower magnification coefficients of the Otsu threshold, the default value is: k down =0.5, k up =1.5; K down ,K up It is the upper and lower control thresholds of the Canny edge detection method. The default value is K down =150, K up =255.

7. A fully automatic rapid gravel screening method suitable for drones according to claim 6, characterized in that Step 3) In generating the gravel outline, the eight-connected region suppression algorithm is used to remove the fine texture inside the gravel. The algorithm first divides all pixels in the image into different connected region patches according to eight neighborhoods; then sets a filtering threshold, which defaults to 800; finally, all connected region patches in the image are filtered according to the set threshold to achieve the effect of suppressing noise; In the foreground and background patch suppression algorithm, the foreground refers to gravel and the background refers to the boundary of the gravel. The algorithm first suppresses the fragmented patches in the foreground and background using the eight-connected region suppression algorithm on the image synthesized by the local threshold method and the Canny edge detection method. The foreground is white and the background is black.

8. A fully automatic rapid gravel screening method suitable for drones according to claim 2, characterized in that Step 4) The generation of seed points in the segmented gravel image specifically includes the following steps: The position of the seed point is limited to a certain range near the center of its influence area to ensure that the generated polygon can more accurately reflect the shape characteristics of the gravel contour; the entire image is evenly divided into a corresponding number of regions according to the preset number of seed points, and in each region, candidate points are randomly generated and screened according to the seed point judgment criteria; The specific steps of determining the maximum search area are: When determining the maximum search area for gravel, firstly, the influence area of ​​the seed point is used as the centroid of the influence area to generate a square area whose side length is equal to half the length of the short side of the influence area; then, this square area is called the centroid influence area, and a series of points are generated in the area, which are defined as the control points for generating the vertices of the boundary polygon; In order to form the boundary control polygon, rays are emitted from each control point in eight neighborhood directions until they intersect with the boundary. The intersection points are sorted according to the distance between them and the control points, and the points with the closest distance are selected. By default, the first four are used as the vertices of the polygon. This process is repeated at each control point in the control area to obtain all relevant polygon vertices. Finally, the polygonal area connected by these vertices is the maximum search area for gravel. The accuracy and effectiveness of the search area are ensured by the generation of precise control points and the selection of boundary intersection points.

9. A fully automatic rapid gravel screening method suitable for drones according to claim 2, characterized in that Step 4) The specific steps of searching and determining the pixel patches in the segmented gravel image are as follows: The search process of gravel images is performed with pixel patches as the basic unit; A pixel patch is a set consisting of a central pixel and its eight neighboring pixels. The search process starts with a seed point. From this point, all pixel patches within the four neighborhoods of the seed are searched and added to the set to be detected. Subsequently, each pixel patch to be tested in this set is evaluated, and the pixel patches identified as gravel will become the starting point for the next round of search. As the search progresses, the pixel patches to be tested will gradually approach the boundary. The number of pixel patches to be tested in each round will decrease accordingly as the number of pixel patches on the gravel boundary increases. When the number of pixel patches at the starting point decreases to zero, the search process will terminate. The specific steps of the determination are: The gravel pixel patch judgment function meets the following two requirements: on the one hand, the pixel patches in the vicinity of the seed point must belong to the gravel pixel patches. When the pixel patches are located within the relative influence distance β near the seed point, no matter how many pixels in the pixel patches belong to the gravel color domain, they are all identified as gravel pixel patches. On the other hand, as the distance between the pixel patches and the seed point increases, the possibility that the pixel patches belong to the gravel boundary gradually increases. When judging the pixel patch ownership, the judgment weight of the relative influence distance gradually decreases, while the judgment weight of the color domain gradually increases. Formula (6) is the gravel pixel patch judgment function: Where: Ψ is the gravel pixel patch judgment function; α is the relative distance factor magnification coefficient, the default value is 5; β is the relative influence distance, the default value is 0.4; w is half of the minimum of the length and width of the seed point scope rectangle; x, y are the center coordinates of the pixel patch to be tested, unit pix; x s ,y s is the coordinate of the gravel seed point, in pix; m is the number of pixels in the pixel patch to be tested that belongs to the gravel color domain space; n is the number of pixels in the pixel patch, and the default value is 9.

Images