Fine particle sediment identification method fusing traditional threshold value and neural network
By combining traditional threshold and neural network methods, the problems of low image recognition accuracy and complex calculation in the prior art are solved, and fast and accurate particle characteristic parameters are achieved, and particle analysis is automated and accurate.
Patent Information
- Application Number
- CN202411155704.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-12-03
AI Technical Summary
The accuracy of existing image recognition algorithms is difficult to meet the needs in the processing of silt particles. Traditional methods such as threshold segmentation and maximum inter-class variance segmentation have problems of increasing noise and computational complexity, making it difficult to achieve efficient and automated particle characteristic parameter measurements.
Combining traditional thresholds and neural networks, grayscale criterion is used to perform image preprocessing, image segmentation is performed through contour feature extraction, Mask R-CNN and ResNet models, and morphological processing and Canny edge detection to achieve accurate identification of particle images and feature parameters. Measurement.
It realizes rapid and orderly image processing under different particle size distributions, accurately identify the sediment particle images, obtain particle morphology-related parameters, and realizes automation and precision of particle analysis.
Smart Images

Figure CN119068253A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of particle image recognition, and in particular relates to a fine-grained sediment recognition method integrating a traditional threshold value with a neural network. Background Art
[0002] The basic properties of sediment particles, such as particle size and gradation, are related to the scouring, transport and deposition of sediment. They are the basic data for the study of sediment movement, soil and water conservation, and environmental sediment. They have important theoretical significance and practical value in revealing the complex laws of sediment movement.
[0003] With the development of computer technology, image recognition technology has been widely used in various fields, providing a new means for particle grading detection.
[0004] However, according to existing experience and experimental test results, various basic image segmentation methods, such as threshold segmentation, region growing, edge detection, watershed, etc., if directly applied to particle images, the results obtained are not satisfactory. For sediment particle images, traditional image recognition algorithms use mathematical and topological methods to process gray values. The theory is mature, but the accuracy is difficult to meet the requirements.
[0005] The histogram-based segmentation algorithm often smoothes some local extreme points while smoothing the histogram, adding additional noise and affecting the image segmentation quality; the maximum inter-class variance segmentation algorithm may not calculate the optimal threshold value when there are multiple maxima of the inter-class variance; the segmentation algorithm based on minimum error and homogenization error has obvious advantages in output effect, but at the cost of computational complexity, which makes the code run time too long.
[0006] Therefore, it is necessary to provide a new type of sediment particle characteristic parameter measurement and analysis technology to solve the problems of the above-mentioned image recognition algorithm and meet the requirements of efficient and automated analysis of sediment particle characteristic parameters. Summary of the invention
[0007] The purpose of this application is to combine traditional image recognition with neural network data analysis to establish an image-based particle grading detection model, which can more effectively identify sediment particle images to achieve more accurate measurement of particle characteristic parameters.
[0008] To achieve the above object, the present invention provides the following technical solutions:
[0009] A fine-grained sediment identification method integrating traditional threshold and neural network, comprising:
[0010] Step S1, collecting original images of sediment particles;
[0011] Step S2: Based on the acquired original image, perform grayscale criterion-based sediment particle image preprocessing:
[0012] Step S3, image feature analysis and feature extraction: based on the classification of typical soil and stone samples, feature extraction is performed according to the characteristics of the target area, and the contour features and angularity feature indicators of the particles are identified by image recognition and quantitative indicators are extracted;
[0013] Step S4: For more complex conditions and high image content similarity, a sediment particle image segmentation and recognition method based on Mask R-CNN is used;
[0014] Step S5: image edge detection and contour extraction;
[0015] Step S6, analysis of particle volume mass model;
[0016] Step S7: particle parameter detection.
[0017] Furthermore, step S2 is an image recognition method based on the grayscale criterion, which analyzes the intensity of each pixel in the particle grayscale image through preprocessing, feature analysis, target extraction and other operations, summarizes the image features from the aspects of area, color, gradient, texture, etc., and realizes the analysis and recognition of the particle image; specifically includes:
[0018] Step S201: using a grayscale histogram to perform statistics on the grayscale and its distribution in the image, and obtaining an initial grayscale image through grayscale preprocessing, grayscale equalization and noise smoothing;
[0019] Step S202, using a threshold-based sediment particle image segmentation and recognition method to threshold the initial grayscale image, and then convert the thresholded image into a binary image;
[0020] The maximum inter-class variance method with adaptive thresholding is used for image segmentation, and the regional grayscale variance between the foreground area and the background area is used as the evaluation criterion:
[0021] Taking a certain threshold as the standard, when the gray value is higher than the threshold, the gray value of the point is made to be a larger value, and when the gray value is lower than the threshold, it is made to be a smaller value, and each pixel is classified;
[0022] The maximum inter-class variance method with adaptive thresholding is adopted, and the regional grayscale variance between the foreground area and the background area in the image is used as the evaluation criterion; it is judged whether the sediment particles exist in a complex state of stacking each other. If there is adhesion, the corrosion and expansion operations in morphological processing are required. If applied in actual engineering environments, the combined operation of corrosion followed by expansion is used to process the image to achieve the cutting of single particles.
[0023] Step S203, using the Canny edge detection algorithm, that is, processing the binary image in sequence according to the steps of Gaussian filtering, gradient solution, non-maximum suppression, double threshold detection, and isolated point suppression.
[0024] Furthermore, in step S4, the foreground of the image refers to the area where the sand, gravel and sediment particles are located, and the background area refers to other areas in the image except the foreground area; if there is adhesion between the sediment particles, the corrosion and dilation operations in the morphological processing are used to process the image to achieve the cutting of single particles; for uneven sediment images with densely distributed particles, the marker-based watershed algorithm is used for image segmentation.
[0025] Furthermore, step S4 specifically includes:
[0026] S401. Based on the ResNet model, feature candidate regions are formed through the region proposal network RPN to form regions of interest of different sizes, that is, a region frame is set in the form of a box selection on the sediment particle image to classify and judge the regional features contained in the frame;
[0027] S402. To solve the problem of pixel alignment in candidate regions, all candidate regions are traversed in the feature image. The borders of the candidate regions are not quantized, the floating-point edges are kept, and the candidate regions are equally divided into n×n regions. The coordinates of the center points of each small region are calculated by bilinear interpolation, and the pixel values corresponding to the coordinates are solved. Finally, the regions are pooled to achieve alignment of the RoI regions and obtain the candidate feature regions. Then, Softmax classification and Bbox boundary regression are performed on the RoI regions, and mask prediction is formed using a fully convolutional neural network (FCN).
[0028] Furthermore, the dataset required for neural network training was annotated using the open source image annotation software Labelme, and the annotated contours were consistent with the actual edges of the particles; and rotation was used to enhance the image data and expand the sample database to ensure contour integrity while increasing the amount of feature extraction training.
[0029] The beneficial effects of the present invention are:
[0030] The particle image recognition technology of the present invention can quickly and orderly perform image processing, target segmentation, edge detection and other operations under different particle size distribution conditions, can more effectively identify sediment particle images, and obtain a series of parameters related to particle morphology, thereby realizing automation and precision of particle analysis. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 This is a flowchart of the application method;
[0032] Figure 2This is a schematic diagram of the basic steps of canny algorithm edge detection;
[0033] Figure 3 This is a schematic diagram of the image of sediment particles after grayscale processing;
[0034] Figure 4 for Figure 3 Schematic diagram of an image after bilateral filtering;
[0035] Figure 5 for Figure 4 Schematic diagram of the image after threshold segmentation;
[0036] Figure 6 for Figure 5 Schematic diagram of the image after edge detection. DETAILED DESCRIPTION
[0037] The following is a clear and complete description of the technical solutions in the embodiments of the present invention through specific implementation methods combined with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0038] like Figure 1 As shown, the method includes:
[0039] Step S1, collecting the original image of the sediment particles. The original image can be an image of the sediment particles collected by an image acquisition device (which can be but is not limited to a camera). Different shooting cameras are used to obtain images, such as high-speed cameras, industrial cameras, micro cameras, etc., with appropriate focal lengths and optical systems to obtain real-life images of the particles, that is, original images.
[0040] Step S2: Based on the acquired original image, perform grayscale criterion-based sediment particle image preprocessing:
[0041] Step S201: Use grayscale histogram to count the grayscale and its distribution in the image. The initial grayscale image is obtained by grayscale preprocessing, grayscale equalization and noise smoothing. Figure 3 shown.
[0042] Step S202: The initial grayscale image is then subjected to threshold processing. The image after threshold processing is as follows: Figure 5 The image processed by thresholding is converted into a binary image. If it is applied in an actual engineering environment, a combination of corrosion and dilation is used to deal with the stacking and adhesion of particles in the image.
[0043] The maximum inter-class variance method with adaptive thresholding is used for image segmentation, and the regional grayscale variance between the foreground area and the background area is used as the evaluation criterion. The foreground in the image refers to the area where the sand, gravel and sand particles are located, and the background area refers to other areas in the image except the foreground area, generally shadows, particle edges, etc.
[0044] In the sediment particle image, let the total number of pixels be L, the maximum grayscale be M, the number of pixels with grayscale k be l, and t be the boundary, and divide the image into the foreground area with pixel values greater than t and the background area with pixel values less than t. The probability of the background area in the image is P1, and the probability of the foreground area in the image is P2, expressed as:
[0045]
[0046] P(t) is the probability that a region with a pixel value greater than t appears in the image;
[0047] in:
[0048]
[0049] P k is the grayscale mean of the image;
[0050] According to the between-class variance formula: σ 2 =P1(u1-u T ) 2 +P2(u2-u T ) 2 , σ 2 is the inter-class variance of regions 1 and 2, and the threshold t is iterated in an iterative manner so that σ 2 The maximum value is obtained, indicating that the difference between the foreground and background areas is the largest, and the corresponding t * The value is the optimal threshold:
[0051]
[0052] The above u1 and u2 are the grayscale averages of the foreground and background areas respectively, u T is the overall grayscale average of the image:
[0053]
[0054] Determine whether the sediment particles exist in a complex state of stacking on each other. If there is adhesion, the erosion and dilation operations in morphological processing need to be used to process the image to achieve the cutting of individual particles.
[0055] For uneven sediment images with densely distributed particles, the watershed algorithm based on markers is used for image segmentation. In the process of image segmentation based on the watershed algorithm based on markers, the background area will no longer be "watered" and the boundary information of the area will no longer be considered. In order to reduce the loss of target particle information, the expansion operation is used to expand the boundary of the target area to prevent the loss of edge information.
[0056] For the foreground part, i.e., the sediment particles, there are a lot of adhesion and stacking phenomena between the particles, so the distance transformation method is used to determine the skeleton part of each particle, and on this basis, the approximate area of the sediment particles is delineated.
[0057] Step S203, using the Canny edge detection algorithm, that is, processing the binary image in sequence according to the steps of Gaussian filtering, gradient solution, non-maximum suppression, double threshold detection, and isolated point suppression.
[0058] When using gradient templates to calculate edge points, noise information can easily affect the accuracy of particle edge detection. Therefore, Gaussian filtering is first used to remove some noise information, as shown in the following formula:
[0059]
[0060] Where Eε is the image function after Gaussian filtering, Gc is the Gaussian filtering function;
[0061] By taking partial derivatives of the gradient function, we can get the gradient size in each direction, and thus determine the grayscale transformation in the image. For the image f(x,y), its gradient is
[0062] Then the Canny operator is used to calculate the gradient vector and image gradient amplitude in the x and y directions:
[0063]
[0064] Among them, E x (x, y) and E y (x, y) represents the gradient magnitude components of the image in the x and y directions;
[0065] At this time, the amplitude of the image gradient is expressed as:
[0066]
[0067] Where M(x, y) represents the image gradient magnitude, and θ(x, y) represents the gradient direction;
[0068] Non-maximum suppression is used to determine potential edge points by comparing the gradient amplitude of each pixel with its two adjacent points in the gradient direction. If the gradient amplitude of a point is greater than the gradient amplitude of its two adjacent points, then the point may be an edge point, otherwise its grayscale value is set to 0; after traversing each element point of the image, an image is formed.
[0069] After the above processing, the image still has many edge points. We can set the size thresholds based on the image characteristics. If the gray value of a point is greater than the set large threshold, it is regarded as a strong edge point; if it is less than the small threshold, it is regarded as a non-edge point and its gray value is set to 0; if it is between the two, it is regarded as a weak edge point; perform neighborhood detection on the weak edge points. If a point is within the neighborhood of a strong edge point, it is identified as an edge point, otherwise its gray value is set to 0.
[0070] Neighborhood detection is performed on "weak edge" points between size thresholds, such as Figure 4 This is a schematic diagram of the image of sediment particles after grayscale processing.
[0071] Step S3, image feature analysis and feature extraction: based on the classification of typical soil and stone samples, feature extraction is performed according to the characteristics of the target area, and the contour features and angularity feature indicators of the particles are identified by image recognition and quantitative indicators are extracted;
[0072] Sand and gravel are a kind of granular material, which has a wide range of causes and complex and changeable states, and has strong random characteristics. According to the shape and color characteristics of sand and gravel, typical samples can be divided into three categories: quartz sand, limestone sand, and granite sand.
[0073] Quartz sand includes green, yellow, gray and other colors, and its color composition is relatively complex. Because it contains more quartz crystals, its shape is angular and has obvious texture. Limestone sand is mainly composed of calcite, mixed with other mineral components. The limestone material is grayish white, with a relatively simple color, relatively broken particles and clear edges. Granite sand has more varied colors and compositions, with small particles and rounded edges, mostly oval.
[0074] Step S4: For more complex conditions and high image content similarity, a sediment particle image segmentation and recognition method based on Mask R-CNN is used, which specifically includes:
[0075] S401. Based on the ResNet model, feature candidate regions are formed through the region proposal network RPN to form regions of interest (RoI) of different sizes, that is, a region box is set in the form of a box selection on the sediment particle image to classify and judge the regional features contained in the box.
[0076] The above-mentioned ResNet model is a neural network that optimizes the learning ability of the model through residual learning. It achieves residual learning of the mapping relationship by adding direct mapping relationships, namely "short-circuit connections", between different layers. This greatly improves the depth of the neural network model while ensuring that the model does not degrade due to excessive number of layers.
[0077] S402. To solve the problem of pixel alignment in candidate regions, the present application traverses all candidate regions in the feature image, does not quantize the borders of the candidate regions, keeps the floating point edges, divides the candidate regions into n×n regions, calculates the coordinates of the center points of each small region by bilinear interpolation, and solves the pixel value corresponding to the coordinates, and finally performs a pooling operation on the region to achieve alignment of the RoI region and obtain the candidate feature region. Then, the RoI region is subjected to Softmax classification, Bbox boundary regression, and a fully convolutional neural network (FCN) is used to form a mask prediction.
[0078] The dataset required for the neural network training mentioned above was annotated using the open source image annotation software Labelme, and the annotated contours were consistent with the real edges of the particles. The image data was enhanced by rotation to expand the sample database, thereby increasing the amount of feature extraction training while ensuring the integrity of the contours.
[0079] Step S5: Image edge detection and contour extraction
[0080] By taking partial derivatives of the gradient function, we can get the gradient size in each direction, and thus determine the grayscale transformation in the image. For the image f(x,y), its gradient is The Sobel operator in eight directions is used to detect the edge of each pixel, and the edge information noise is suppressed at the same time.
[0081] Non-maximum suppression is used to determine potential edge points by comparing the gradient amplitude of each pixel with its two adjacent points in the gradient direction. If the gradient amplitude of a point is greater than the gradient amplitude of its two adjacent points, then the point may be an edge point. Otherwise, its grayscale value is set to 0, and the contour information of each particle can be obtained.
[0082] Step S6, analysis of particle volume mass model;
[0083] Digital images can only show the two-dimensional morphological information of soil and stone particles, but cannot provide the three-dimensional thickness information of particles. Therefore, the average thickness is used in the calculation process. To replace the thickness information of soil and stone particles within each particle size range.
[0084] Average thickness Average flatness W is the longer axis of the particle; the volume V of this particle is: A represents the maximum projected area of soil and stone particles when they are stably placed.
[0085] Step S7, particle parameter detection;
[0086] Through system calibration, the coordinate transformation relationship, data transformation relationship and basic parameters of each hardware device in the system are determined, so that the final required quantity can be accurately calculated and output.
[0087] The purpose of system calibration is to establish a mapping relationship between the pixels in the image and the actual world coordinates, so that the measurement results in the image can be converted into physical quantities. After system calibration, the actual length corresponding to the unit pixel is obtained, and then it is multiplied by the size of the pixel occupied by each particle.
[0088] First, the unit calibration of the image measurement is performed. For a square target with a side length of a×a, the conversion formula between the pixel unit and the actual measurement range is:
[0089]
[0090] MmperPixel is the actual unit length in millimeters corresponding to 1 pixel;
[0091] Calculate the length of 1 pixel and label the different particle target areas in the binary image based on the recursive search method. Starting from a certain pixel in the image, check whether the pixel belongs to a particle (for example, the color or grayscale threshold meets certain conditions). If it is a particle pixel, assign a label to the pixel and recursively perform the same operation on the pixels adjacent to the pixel. Once the area scan of a particle is completed, the algorithm will jump out of the area and continue to scan the image to find the next closed area. When a new closed area is encountered, a new label is assigned to the pixels in it, and the area is recursively scanned. This process is repeated until the entire image is scanned. Finally, the number of pixels with each different label is counted, and the largest label represents the number of particles in the image.
[0092] The main physical quantities measured in sediment particle analysis include particle size and particle shape.
[0093] Particle size parameters include area, perimeter, equivalent circle diameter, gradation curve and characteristic particle size, etc. The area is the sum of the number of pixels occupied by a single target particle, the perimeter is the length of the pixels around its boundary, the equivalent circle diameter is the diameter of a circle equal to the projected area of the particle, and the characteristic particle size is d75, which means that the proportion of sediment particles smaller than this particle in the whole is 75%.
[0094] The gradation curve is made with the particle size as the abscissa and the percentage of particles smaller than the particle size D as the ordinate.
[0095] Particle shape parameters include roundness and shape factor.
[0096] The roundness is expressed as the ratio of the circumference of a circle with the same area as the particle's projection to the circumference of the projection contour, i.e. in, Indicates the circularity of the target particle, A is the particle projection area, P is the perimeter of the particle projection outline, The larger the value, the closer the particle is to a round shape. B is the short diameter and L is the long diameter of the sediment particles.
Claims
1. A fine-grained sediment identification method integrating traditional threshold and neural network, characterized in that: include: Step S1, collecting original images of sediment particles; Step S2: Based on the acquired original image, perform grayscale criterion-based sediment particle image preprocessing: Step S3, image feature analysis and feature extraction: based on the classification of typical soil and stone samples, feature extraction is performed according to the characteristics of the target area, and the contour features and angularity feature indicators of the particles are identified by image recognition and quantitative indicators are extracted; Step S4: For more complex conditions and high image content similarity, a sediment particle image segmentation and recognition method based on Mask R-CNN is used; Step S5: image edge detection and contour extraction; Step S6, analysis of particle volume mass model; Step S7: particle parameter detection.
2. The fine-grained sediment identification method integrating traditional threshold and neural network as claimed in claim 1 is characterized in that: Step S2 specifically includes: Step S201, using a grayscale histogram to perform statistics on the grayscale and its distribution in the image, and obtaining an initial grayscale image through grayscale preprocessing, grayscale equalization and noise smoothing, and the initial grayscale image is then subjected to threshold processing; Step S202: threshold the initial grayscale image using a threshold-based sediment particle image segmentation and recognition method, and then convert the thresholded image into a binary image. If applied in an actual engineering environment, a combined operation of first corrosion and then expansion is used to handle the stacking and adhesion of particles in the image; Step S203, using the Canny edge detection algorithm, that is, processing the binary image in sequence according to the steps of Gaussian filtering, gradient solution, non-maximum suppression, double threshold detection, and isolated point suppression.
3. The fine-grained sediment identification method integrating traditional threshold and neural network as claimed in claim 1 is characterized in that: In step S3, the foreground of the image refers to the area where the sand, gravel and sediment particles are located, and the background area refers to the other areas in the image except the foreground area; if there is adhesion between the sediment particles, the corrosion and dilation operations in the morphological processing are used to process the image to achieve the cutting of single particles; for the uneven sediment image with densely distributed particles, the marker-based watershed algorithm is used for image segmentation.
4. The fine-grained sediment identification method integrating traditional threshold and neural network as claimed in claim 1 is characterized in that: Step S4 specifically includes: S401. Based on the ResNet model, feature candidate regions are formed through the region proposal network RPN to form regions of interest of different sizes, that is, a region frame is set in the form of a box selection on the sediment particle image to classify and judge the regional features contained in the frame; S402. To solve the problem of pixel alignment in candidate regions, all candidate regions are traversed in the feature image. The borders of the candidate regions are not quantized, the floating-point edges are kept, and the candidate regions are equally divided into n×n regions. The coordinates of the center points of each small region are calculated by bilinear interpolation, and the pixel values corresponding to the coordinates are solved. Finally, the regions are pooled to achieve alignment of the RoI regions and obtain the candidate feature regions. Then, Softmax classification and Bbox boundary regression are performed on the RoI regions, and mask prediction is formed using a fully convolutional neural network (FCN).
5. The fine-grained sediment identification method integrating traditional threshold and neural network as claimed in claim 4 is characterized in that: The dataset required for neural network training was annotated using the open source image annotation software Labelme, and the annotated contours were consistent with the actual edges of the particles. The image data was enhanced by rotation to expand the sample database in order to ensure contour integrity while increasing the amount of feature extraction training.
Citation Information
Patent Citations
Sediment particle analysis device and analysis method based on digital imaging technology
CN102288522A
Sediment recognition method and sediment recognition system based on image fusion
CN102955935A
Shatian pomelo image recognition method for picking robot
CN114187254A
Mask R-CNN mineral particle identification and particle size detection method based on improved mask
CN114897816A
Crop seed automatic counting method based on machine vision
CN115187788A
Cited By
Method and device for detecting particle size of aerobic granular sludge based on image recognition
CN120259407A
Plastic particle detection method and system based on image recognition
CN120375006A
Plastic particle detection method and system based on image recognition
CN120375006B
Intelligent analysis method for outline of benzimidazolone pigment particles based on graph search algorithm
CN122089775A
Intelligent analysis method for benzimidazolone pigment particle profile based on graph search algorithm
CN122089775B
