Method and device for identifying and grading tunnel waste particles

Abstract

The present invention provides a method and device for identifying and grading tunnel waste particles, comprising: S3, obtaining a segmentation mask dataset of waste particle contours; S4, using an improved TOPSIS method based on Mahalanobis distance to obtain a comprehensive shape factor dataset of the waste particles based on the segmentation mask dataset; S5, obtaining the morphological grade of each waste particle based on the segmentation mask dataset and the comprehensive shape factor dataset, thereby forming a waste particle image dataset with morphological grade labels and marker labels; S6, inputting the waste particle image dataset into a neural network for training to obtain a second neural network model; S7, inputting any newly collected waste particle image into the second neural network model to output the waste particle segmentation mask data and morphological grade. The improved TOPSIS method based on Mahalanobis distance of the present invention can obtain a comprehensive shape factor that takes into account the correlation between various shape indicators and more accurately reflects the morphological information relationship embodied by each indicator.

Description

Technical Field

[0001] The present invention relates to the field of engineering waste detection and analysis, and in particular to a method and device for identifying and grading tunnel waste particles. Background Art

[0002] Tunnel waste is generated during the construction of tunnels and underground projects, and its disposal has become a critical component of engineering construction. For rock masses of varying lithologies, blasting or mechanical crushing produces slag of varying particle shapes and sizes. The mechanical properties of these slag are significantly affected by the particle morphology and size.

[0003] Currently, there are two main approaches to the disposal of tunnel waste. For mountain tunnel waste, waste is often deposited at selected locations to form waste dumps. The engineering stability of the waste dump is closely related to the shape, size distribution, and other characteristics of the waste particles. For tunnels and underground projects in urban environments, the transportation and storage of waste require high transportation and land costs. In recent years, engineering and technical personnel have paid more attention to the reuse of tunnel waste. Common reuse methods include roadbed filling and building material aggregate reuse. The shape, size distribution, and other characteristics of the waste particles significantly affect the engineering properties of the recycled resources (such as the strength and compaction characteristics of the roadbed and the bulk density of the aggregate). In summary, rapid and accurate assessment of the particle information of tunnel waste can help achieve refined management and classification of tunnel waste. On the one hand, it can assist in evaluating the engineering stability of the waste dump, and on the other hand, it can improve the quality control level of tunnel waste recycling products.

[0004] In terms of collecting particle information, for tunnel waste, traditional particle size collection is mainly based on on-site or indoor testing, while the capture of particle morphology contours relies on equipment such as handheld 3D scanners. Traditional methods have high labor and time costs, a small number of samples per test, and a high degree of randomness in data collection, making it difficult to quickly and accurately reflect the gradation and particle morphology characteristics of large-scale waste. In terms of processing and analyzing particle morphology information, individual rock and soil particles often have multi-scale morphological characteristics, including macroscopic shape, angular characteristics, and microscopic texture. There are many morphological characterization parameters, and some parameters are correlated with each other, which leads to redundancy problems in the batch analysis of large-scale particle information data. Reasonable particle morphology classification helps to eliminate the redundancy of the system, but the existing particle morphology classification method can only realize the classification and judgment of the macroscopic shape scale information of particles, ignoring the microscopic scale morphological characteristics such as angularity and texture.

[0005] Image processing methods based on convolutional neural networks have excellent performance in terms of accuracy and batch processing, and have been innovatively applied in particle identification in the fields of rockfill materials and construction waste: Authorization No. CN111242909B discloses a method for rapid identification of particle size distribution of construction waste based on convolutional neural networks, but this method does not realize the identification and classification of particle morphological information; Application No. CN202110836767.8 discloses a rockfill image data extraction method and device, which can realize the extraction of rockfill particle area and roundness, but does not involve the identification of more scale particle information such as edges, corners, and textures. Summary of the Invention

[0006] In response to the above technical problems in the related art, the present invention proposes a method for identifying and grading tunnel waste particles, which includes the following steps:

[0007] S3, obtaining a segmentation mask dataset M1 of the waste particle contours;

[0008] S4. Using an improved TOPSIS method based on Mahalanobis distance, the shape index and comprehensive shape factor of each waste particle are calculated according to the segmentation mask dataset M1 to form a comprehensive shape factor dataset S of the waste particles. The shape index includes the aspect ratio AR, roundness Cir, concavity Sol, angularity Ang, and roughness Tex. The improved TOPSIS method based on Mahalanobis distance replaces the distance scoring formula in TOPSIS with the Mahalanobis distance scoring formula.

[0009] S5. Construct a morphological classification standard for waste particles based on the segmentation mask dataset M1 and the comprehensive shape factor dataset S, calculate the morphological grade L of each waste particle in the segmentation mask dataset M1, and form a waste particle image dataset D3 with morphological grade labels and marker labels;

[0010] S6, inputting the waste particle image dataset D3 into the neural network for training to obtain a second neural network model N2 containing the waste particle outlines and their morphological grade feature weights;

[0011] S7. Input any newly collected waste particle image into the second neural network model N2 to perform contour recognition and morphology detection of the waste particles, and output the waste particle segmentation mask data and morphology level, and calculate the waste particle equivalent particle size D based on the segmentation mask data.

[0012] Specifically, the method further includes: S1, obtaining a waste slag particle image dataset D1 including waste slag particle contours and size marker contour labels;

[0013] S2, inputting the waste particle image dataset D1 into the neural network for training to obtain a first neural network model N1 containing waste particle contour feature weights;

[0014] Specifically, step S3 is to input the unlabeled waste particle image dataset D2 into the first neural network model N1, identify and label the waste particle contours and markers, and output a segmentation mask dataset M1 forming the waste particle contours.

[0015] Specifically, during the image acquisition process in step S1, a size marker needs to be placed near the center of the image.

[0016] Specifically, in step S2, the image dataset D1 is randomly divided into a training set, a validation set, and a test set in proportion.

[0017] Specifically, the unlabeled waste particle image dataset D2 in step S3 is constructed by randomly selecting a second preset number of waste particle images from the images collected in step S1; the dataset M1 includes a set of mask images of each waste particle, a set of corner point coordinate data of the minimum bounding box of each waste particle outline, and a corresponding original image set.

[0018] Specifically, the shape indicators of the waste particles calculated in step S4 include: the major-minor axis ratio AR is the ratio of the maximum Feret radius of the waste particles to the minimum Feret radius of the waste particles; the roundness Cir is the area ratio of the waste particle projection to the equal-perimeter circle; the convexity Sol is the area ratio of the waste particle projection to the minimum circumscribed polygon; the edge angle Ang is the ratio of the average maximum inscribed circle radius of each edge in the waste particle to the maximum inscribed circle radius; the roughness Tex is the ratio of the waste particle projection to the perimeter of the minimum circumscribed polygon.

[0019] Specifically, the method for calculating the comprehensive shape factor in step S4 includes the following steps:

[0020] S41. Perform positive processing on the shape index according to the following formula:

[0021]

[0022] Among them, x' ij is a positive indicator; x ij It is the column vector element composed of the indicators to be positively converted;

[0023] S42. Calculate the positive ideal solution Y of the five shape indices + , the calculation formula is:

[0024]

[0025] Among them, Y + is the positive ideal solution of the index, x ij It is the column vector element composed of the indicators to be positively converted;

[0026] Calculate the negative ideal solution Y of five shape indices - , the calculation formula is:

[0027]

[0028] Among them, Y - is the negative ideal solution of the index, x ij It is the column vector element composed of the indicators to be positively converted;

[0029] S43, the normalized aspect ratio AR, edge angle Ang, roughness Tex, roundness Cir, and concavity Sol are used to form an evaluation index matrix X; each index X is calculated. i To the positive ideal solution Y + The Mahalanobis distance d(X i ,Y + ), the calculation formula is:

[0030]

[0031] Calculate each indicator X i To the negative ideal solution Y - The Mahalanobis distance d(X i ,Y - ), the calculation formula is:

[0032]

[0033] S44. Calculate the comprehensive shape factor f of each waste particle i , the calculation formula is:

[0034]

[0035] Specifically, the morphological classification standard of the waste particles in step S5 is based on the comprehensive shape factor f of each waste particle. i The size of the morphological level L is divided into 5 levels: L1, L2, L3, L4, and L5; the waste particle image dataset D3 is automatically annotated according to the comprehensive shape factor dataset S and the segmentation mask dataset M1, and includes the original image set in the dataset M1 and the corresponding waste particle morphological level L label and marker label data set.

[0036] Specifically, it is characterized in that the calculation formula of the equivalent particle size D of the waste particles is:

[0037]

[0038] Among them, m R is the pixel area of ​​the waste particle mask; m s is the pixel area of ​​the marker mask; S sis the actual area of ​​the marker.

[0039] In a second aspect, another embodiment of the present invention discloses a tunnel waste particle identification and classification device, comprising the following units:

[0040] A segmentation mask data set generating unit, used for obtaining a segmentation mask data set M1 of the waste slag particle contour;

[0041] A comprehensive shape factor dataset generation unit uses an improved TOPSIS method based on Mahalanobis distance to calculate the shape index and comprehensive shape factor of each waste particle according to the segmentation mask dataset M1, thereby forming a comprehensive shape factor dataset S of the waste particles. The shape index includes the aspect ratio AR, roundness Cir, concavity Sol, angularity Ang, and roughness Tex. The improved TOPSIS method based on Mahalanobis distance replaces the distance scoring formula in TOPSIS with the Mahalanobis distance scoring formula.

[0042] The second data set generating unit is used to construct a morphological classification standard for waste particles based on the segmentation mask data set M1 and the comprehensive shape factor data set S, calculate the morphological grade L of each waste particle in the segmentation mask data set M1, and form a waste particle image data set D3 with morphological grade labels and marker labels;

[0043] The second modeling unit is used to input the waste particle image dataset D3 into the neural network for training to obtain a second neural network model N2 including the waste particle outline and its morphological grade feature weights;

[0044] The grading unit is used to input any newly collected waste particle image into the second neural network model N2, perform contour recognition and morphology detection of the waste particles, and output the waste particle segmentation mask data and morphology grade, and calculate the equivalent particle size D of the waste particles based on the segmentation mask data.

[0045] The tunnel waste particle identification and grading method of the present invention adopts an improved TOPSIS method based on Mahalanobis distance, calculates the shape index and comprehensive shape factor of each waste particle according to the segmentation mask data set M1, and forms a comprehensive shape factor data set S of the waste particles; constructs a morphological grading standard for the waste particles according to the segmentation mask data set M1 and the comprehensive shape factor data set S, calculates the morphological grade L of each waste particle in the segmentation mask data set M1, and forms a waste particle image data set D3 with a morphological grade label and a marker label; inputs the waste particle image data set D3 into a neural network for training, and obtains a neural network model N2 containing the waste particle contours and their morphological grade feature weights; inputs any newly collected waste particle images into the neural network model N2, performs waste particle contour identification and morphological detection, and outputs the waste particle segmentation mask data and morphological grade, and calculates the waste particle equivalent particle size D from the segmentation mask data. Furthermore, the present application also uses a first neural network model N1 to identify the collected tunnel waste particle graphics, thereby obtaining a segmentation mask dataset M1 of the waste particle contour. The present invention collects images of tunnel waste particles, selects a first preset number of waste particle images, and uses an auxiliary labeling tool to label the waste particle contours and size markers to form a waste particle image dataset D1 with waste particle contours and marker labels; the waste particle image dataset D1 is input into the neural network for training to obtain a neural network model N1 containing waste particle contour feature weights; the unlabeled waste particle image dataset D2 is input into the neural network model N1 to identify and label the waste particle contours and markers, and output the segmentation mask dataset M1 forming the waste particle contour. The auxiliary labeling tool developed based on the general segmentation neural network can greatly improve the labeling efficiency of the waste particle contour.

[0046] The improved TOPSIS classification method based on the Mahalanobis distance of the statistical principle in the present invention is simple to calculate. The obtained comprehensive shape factor can take into account the correlation between various shape indicators and eliminate the interference of different degrees of correlation between various indicators, which can more accurately reflect the morphological information relationship embodied by various indicators.

[0047] Furthermore, the tunnel waste particle recognition neural network finally trained in the present invention can not only automatically and accurately mark the outlines of the tunnel waste particles, but also directly predict and output the morphological classification results of the tunnel waste particles, and calculate the equivalent particle size distribution through simple post-processing, which can greatly simplify the manual workflow of quantitative extraction and classification of size characteristics and shape characteristics indicators, save time and manpower, and improve work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 This is a flow chart of a method for identifying and grading tunnel waste particles according to an embodiment of the present invention;

[0050] Figure 2 is an example image of tunnel waste particles collected according to an embodiment of the present invention;

[0051] Figure 3 1 is an example diagram of segmentation mask data predicted and output by the neural network model N2 according to an embodiment of the present invention;

[0052] Figure 4 It is a statistical diagram of waste slag particle classification output by the neural network model N2 provided in an embodiment of the present invention;

[0053] Figure 5 The neural network model N2 provided in an embodiment of the present invention predicts and outputs a waste slag particle size distribution diagram;

[0054] Figure 6 2 is a schematic diagram of a tunnel waste particle identification and classification device provided according to an embodiment of the present invention;

[0055] Figure 7 Schematic diagram of a tunnel waste particle identification and classification device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention are within the scope of protection of the present invention.

[0057] Example 1

[0058] refer to Figure 1 According to this embodiment, a method for identifying and grading tunnel waste particles is disclosed, which includes the following steps:

[0059] S3, obtaining a segmentation mask dataset M1 of the waste particle contours;

[0060] In step S3 of this embodiment, the segmentation mask dataset M1 of the waste particle contours is obtained, which can be identified from the waste particle image in the form of a convolutional neural network.

[0061] Specifically, the present application can use an auxiliary annotation tool developed by a general segmentation neural network to improve the efficiency of annotation of the waste slag particle contours; this embodiment also includes:

[0062] S1. Capturing images of tunnel waste particles, selecting a first preset number of waste particle images, and annotating the waste particle contours and size markers using auxiliary annotation tools to form a waste particle image dataset D1 with waste particle contours and marker labels;

[0063] Specifically, during the image acquisition process in step S1, a size marker needs to be placed near the center of the image.

[0064] Specifically, the size marker is a thin plate or a sphere with a certain diameter, and its outline is marked on the image to represent the area to which the image belongs.

[0065] refer to Figure 2 The size marker is a rectangular thin plate with a white background and a black marker; the size of the size marker is 15 cm × 10 cm; the size of the collected image is 4000 pixels × 6000 pixels.

[0066] Specifically, the auxiliary annotation tool used in step S1 is an auxiliary annotation software developed based on the general segmentation neural network SegmentAnything Model, which can automatically and quickly generate high-quality annotation contours through input points or box prompts.

[0067] Preferably, the first preset number in step S1 is 400, and the selection method is random selection. The auxiliary annotation tool used is based on the general segmentation neural network Segment Anything Model and adopts training weights with a large network backbone size. It improves the traditional manual multi-point envelope fitting annotation method, allowing users to automatically and quickly generate high-quality segmentation masks for specified objects in the image by entering points or box prompts with the mouse, and converting the mask contour fitting into polygon point annotation. The annotation method used is to annotate the outline of each clear waste particle on the surface of each image with polygon point annotation and assign sequential labels such as "stone_1", "stone_2", "stone_3", etc., and also annotate size markers with polygon point annotation and assign sequential labels such as "signboard_1", "signboard_2", "signboard_3", etc. The dataset D1 specifically includes a set of selected waste particle images and a corresponding set of waste particle contour and size marker contour label data.

[0068] S2, inputting the waste particle image dataset D1 into the neural network for training to obtain a neural network model N1 containing waste particle contour feature weights;

[0069] Specifically, in step S2, the image dataset D1 is randomly divided into a training set, a validation set, and a test set in proportion.

[0070] Preferably, in step S2, the image dataset D1 is randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1; the images and annotation data of the training set, validation set, and test set are divided into equal-sized blocks with an image size of 1000 pixels × 1000 pixels; and the neural network adopts a mask-rcnn model based on the pre-trained weights of the Microsoft Common Objects in Context dataset.

[0071] Specifically, the contours and labels input into the neural network are mapped one to one. The input contour point set is processed to form a pixel segmentation mask for the corresponding enclosed area, which is then passed to the neural network for operations such as convolution.

[0072] Furthermore, step S3 is to input the unlabeled waste particle image dataset D2 into the neural network model N1 to identify and label the waste particle contours and markers, and output a segmentation mask dataset M1 forming the waste particle contours.

[0073] Specifically, the unlabeled waste particle image dataset D2 in step S3 is constructed by randomly selecting a second preset number of waste particle images from the images collected in step S1; the dataset M1 includes a set of mask images of each waste particle, a set of corner point coordinate data of the minimum bounding box of each waste particle outline, and a corresponding original image set.

[0074] Preferably, the second preset number in step S3 is 1000 images, and 1000 images are randomly selected from the images collected in step S1 to form the image dataset D2;

[0075] Specifically, the segmentation mask is a three-dimensional matrix of W×H×1, which can be saved in grayscale image format, where W and H are the image width and height respectively, and the third dimension is the pixel value. The pixel value corresponding to the particle area is 1, and the non-particle area is 0.

[0076] Specifically, while executing the neural network prediction to output the segmentation mask, a simple processing is performed on the segmentation mask, so that it can produce some by-products, such as the minimum bounding box of the mask, etc.

[0077] S4. Using an improved TOPSIS method based on Mahalanobis distance, the shape index and comprehensive shape factor of each waste particle are calculated according to the segmentation mask dataset M1 to form a comprehensive shape factor dataset S of the waste particles. The shape index includes the aspect ratio AR, roundness Cir, concavity Sol, angular angle Ang, and roughness Tex. The improved TOPSIS method based on Mahalanobis distance replaces the distance scoring formula in TOPSIS with the Mahalanobis distance scoring formula.

[0078] Specifically, the shape indicators of the waste particles calculated in step S4 include: the major-minor axis ratio AR is the ratio of the maximum Feret radius of the waste particles to the minimum Feret radius of the waste particles; the roundness Cir is the area ratio of the waste particle projection to the equal-perimeter circle; the convexity Sol is the area ratio of the waste particle projection to the minimum circumscribed polygon; the edge angle Ang is the ratio of the average maximum inscribed circle radius of each edge in the waste particle to the maximum inscribed circle radius; the roughness Tex is the ratio of the waste particle projection to the perimeter of the minimum circumscribed polygon.

[0079] Specifically, the calculation formulas for the five shape indicators are as follows:

[0080] A. The calculation formula for the major-minor axis ratio AR is:

[0081]

[0082] Among them, F max is the maximum Feret radius of the waste particles, F min is the minimum Feret radius of the waste particles;

[0083] B. The calculation formula of roundness Cir is:

[0084]

[0085] Where S is the area of ​​the spoil particle mask, and P is the perimeter of the spoil particle mask;

[0086] C. The calculation formula of concave-convexity Sol is:

[0087]

[0088] Among them, S is the mask area of ​​waste particles, S con is the area of ​​the minimum circumscribed polygon of the waste particle mask;

[0089] D. The calculation formula of edge angle Ang is:

[0090]

[0091] Among them, r i is the maximum inscribed circle radius of each corner in the waste particle mask; n is the number of corners in the waste particle mask; R is the maximum inscribed circle radius of the waste particle mask;

[0092] E. The calculation formula of roughness Tex is:

[0093]

[0094] Where P is the perimeter of the waste particle mask; P con is the perimeter of the minimum circumscribed polygon of the spoil particle mask.

[0095] Specifically, the method for calculating the comprehensive shape factor in step S4 includes the following steps:

[0096] S41. Perform positive processing on the shape index according to the following formula:

[0097]

[0098] Among them, x' ij is a positive indicator; x ij It is the column vector element composed of the indicators to be positively converted;

[0099] S42. Calculate the positive ideal solution Y of the five shape indices + , the calculation formula is:

[0100]

[0101] Among them, Y + is the positive ideal solution of the index, x ij It is the column vector element composed of the indicators to be positively converted;

[0102] Calculate the negative ideal solution Y of five shape indices - , the calculation formula is:

[0103]

[0104] Among them, Y - is the negative ideal solution of the index, x ij It is the column vector element composed of the indicators to be positively converted;

[0105] S43, the normalized aspect ratio AR, edge angle Ang, roughness Tex, roundness Cir, and concavity Sol are used to form an evaluation index matrix X; each index X is calculated. i To the positive ideal solution Y + The Mahalanobis distance d(X i ,Y + ), the calculation formula is:

[0106]

[0107] Calculate each indicator X i To the negative ideal solution Y - The Mahalanobis distance d(X i ,Y - ), the calculation formula is:

[0108]

[0109] S44. Calculate the comprehensive shape factor f of each waste particle i , the calculation formula is:

[0110]

[0111] S5. Construct a morphological classification standard for waste particles based on the segmentation mask dataset M1 and the comprehensive shape factor dataset S, calculate the morphological grade L of each waste particle in the segmentation mask dataset M1, and form a waste particle image dataset D3 with morphological grade labels and marker labels;

[0112] Specifically, the morphological classification standard of the waste particles in step S5 is based on the comprehensive shape factor f of each waste particle. i The size of the morphological level L is divided into 5 levels: L1, L2, L3, L4, and L5; the waste particle image dataset D3 is automatically annotated according to the comprehensive shape factor dataset S and the segmentation mask dataset M1, and includes the original image set in the dataset M1 and the corresponding waste particle morphological level L label and marker label data set.

[0113] Preferably, the specific method of the morphological classification standard of the waste particles in step S5 is: according to the comprehensive shape factor f of each waste particle i The size of 0≤f i <0.2, 0.2≤f i <0.4, 0.4≤f i <0.6, 0.6≤f i <0.8, 0.8≤f i <1 is divided into 5 levels of morphological level L, namely L1, L2, L3, L4, and L5; the dataset D3 is automatically annotated based on the comprehensive shape factor dataset S and the segmentation mask dataset M1, and includes the original image set in the dataset M1 and the corresponding waste slag particle morphological level L labels and marker label data sets.

[0114] S6, inputting the waste particle image dataset D3 into the neural network for training, and obtaining a neural network model N2 including the waste particle outline and its morphological grade feature weights;

[0115] Preferably, in step S6, the image dataset D3 is randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1; the neural network adopts a mask-rcnn model with pre-trained weights based on the Microsoft Common Objects in Context dataset.

[0116] Specifically, the label of the input dataset D3 is morphological level L, so the segmentation neural network will output contours classified by morphological level, which is equivalent to the neural network N1 outputting a single-class (stone) contour mask, while N2 can output a contour mask classified according to morphological level including the TOPSIS grading method.

[0117] Neural network model N2 and neural network model N1 are independent neural networks, but use the same neural network architecture and pre-trained data weights. The neural networks used by both are instance segmentation neural networks.

[0118] S7. Input any newly collected waste particle image into the neural network model N2 to perform contour recognition and morphology detection of the waste particles, and output the waste particle segmentation mask data and morphology level, and calculate the waste particle equivalent particle size D based on the segmentation mask data.

[0119] refer to Figure 3-Figure 5 The segmentation mask data in step S7 includes a set of mask images of a single waste particle and a marker in the predicted image, a set of corner point coordinate data of a minimum bounding box of a single waste particle and a marker, and the corresponding original predicted image.

[0120] Specifically, the calculation formula for the equivalent particle size D of the waste particles is:

[0121]

[0122] Among them, m R is the pixel area of ​​the waste particle mask; m s is the pixel area of ​​the marker mask; S s is the actual area of ​​the marker.

[0123] The tunnel waste particle identification and grading method of this embodiment adopts an improved TOPSIS method based on Mahalanobis distance to calculate the shape index and comprehensive shape factor of each waste particle based on the segmentation mask dataset M1 to form a comprehensive shape factor dataset S of the waste particles; based on the segmentation mask dataset M1 and the comprehensive shape factor dataset S, a morphological grading standard for the waste particles is constructed, and the morphological grade L of each waste particle in the segmentation mask dataset M1 is calculated to form a waste particle image dataset D3 with morphological grade labels and marker labels; the waste particle image dataset D3 is input into a neural network for training to obtain a neural network model N2 containing the waste particle outlines and their morphological grade feature weights; any newly collected waste particle images are input into the neural network model N2 to perform waste particle outline recognition and morphological detection, and the waste particle segmentation mask data and morphological grade are output, and the waste particle equivalent particle size D is calculated from the segmentation mask data. Furthermore, the present application also uses a first neural network model N1 to identify the collected tunnel waste particle graphics, thereby obtaining a segmentation mask dataset M1 of the waste particle contour. The present invention collects images of tunnel waste particles, selects a first preset number of waste particle images, and uses an auxiliary labeling tool to label the waste particle contours and size markers to form a waste particle image dataset D1 with waste particle contours and marker labels; the waste particle image dataset D1 is input into the neural network for training to obtain a neural network model N1 containing waste particle contour feature weights; the unlabeled waste particle image dataset D2 is input into the neural network model N1 to identify and label the waste particle contours and markers, and output the segmentation mask dataset M1 forming the waste particle contour. The auxiliary labeling tool developed based on the general segmentation neural network can greatly improve the labeling efficiency of the waste particle contour.

[0124] The improved TOPSIS grading method based on the statistical principle of Mahalanobis distance in this embodiment is simple to calculate. The obtained comprehensive shape factor can take into account the correlation between various shape indicators and eliminate the interference of different degrees of correlation between various indicators, which can more accurately reflect the morphological information relationship embodied by various indicators.

[0125] The tunnel waste particle recognition neural network finally obtained by training in this embodiment can not only automatically and accurately mark the outlines of tunnel waste particles, but also directly predict and output the morphological classification results of tunnel waste particles, and calculate the equivalent particle size distribution through simple post-processing, which can greatly simplify the manual workflow of quantitative extraction and classification of size and shape feature indicators, save time and manpower, and improve work efficiency.

[0126] Example 2

[0127] refer to Figure 6 According to this embodiment, a tunnel waste particle identification and classification device is disclosed, which includes the following units:

[0128] A first data set generating unit is configured to collect images of tunnel spoil particles, select a first preset number of spoil particle images, and annotate the spoil particle contours and size markers using an auxiliary annotation tool to form a spoil particle image dataset D1 with spoil particle contours and marker labels;

[0129] Specifically, during the image acquisition process used in the first data set generating unit, a size marker needs to be placed near the center of the image.

[0130] Specifically, the size marker is a thin plate or a sphere with a certain diameter, and its outline is marked on the image to represent the area to which the image belongs.

[0131] refer to Figure 2 The size marker is a rectangular thin plate with a white background and a black marker; the size of the size marker is 15 cm × 10 cm; the size of the collected image is 4000 pixels × 6000 pixels.

[0132] Specifically, the auxiliary annotation tool used in the first data set generation unit is an auxiliary annotation software developed based on the general segmentation neural network Segment Anything Model, which can automatically and quickly generate high-quality annotation contours through input points or box prompts.

[0133] Preferably, the first preset number in the first dataset generation unit is 400, and the selection method is random selection; the auxiliary annotation tool used is based on the general segmentation neural network Segment Anything Model and adopts training weights with a large network backbone size, which improves the traditional manual multi-point envelope fitting annotation method, allowing users to automatically and quickly generate high-quality segmentation masks for specified objects in the image by inputting points or box prompts with the mouse, and converting the mask contour fitting into polygon point annotation form. The annotation method used is to annotate the outline of each clear waste particle on the surface of each image with polygon point annotation form, and assign it a sequence label "stone_1", "stone_2", "stone_3", etc., and also annotate the size markers with polygon point annotation form, and assign it a sequence label "signboard_1", "signboard_2", "signboard_3", etc.; the dataset D1 specifically includes a set of selected and collected waste particle images and a corresponding set of waste particle contour and size marker contour label data.

[0134] The first modeling unit is used to input the waste particle image dataset D1 into the neural network for training, and obtain a neural network model N1 including the waste particle contour feature weights;

[0135] Specifically, the image dataset D1 in the first modeling unit is randomly divided into a training set, a validation set, and a test set in proportion.

[0136] Preferably, in the first modeling unit, the image dataset D1 is randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1; the images and annotation data of the training set, validation set, and test set are divided into equal-sized blocks with an image size of 1000 pixels × 1000 pixels; and the neural network adopts a mask-rcnn model with pre-trained weights based on the Microsoft Common Objects inContext dataset.

[0137] Specifically, the contours and labels input into the neural network are mapped one to one. The input contour point set is processed to form a pixel segmentation mask for the corresponding enclosed area, which is then passed to the neural network for operations such as convolution.

[0138] A segmentation mask dataset generation unit is used to input the unlabeled waste particle image dataset D2 into the neural network model N1, identify and label the waste particle contours and markers, and output a segmentation mask dataset M1 forming the waste particle contours, and calculate five shape indices of the waste particles, the five shape indices including the major-minor axis ratio AR, roundness Cir, concavity Sol, angularity Ang, and roughness Tex;

[0139] Specifically, the unlabeled waste particle image dataset D2 in the segmentation mask dataset generation unit is constructed by randomly selecting a second preset number of waste particle images from the images collected in the first dataset generation unit; the dataset M1 includes a set of mask images of each waste particle, a set of corner point coordinate data of the minimum bounding box of each waste particle outline, and a corresponding original image set.

[0140] Preferably, the second preset number in the segmentation mask dataset generation unit is 1000 images, and 1000 images are randomly selected from the images collected by the first dataset generation unit to form the image dataset D2;

[0141] Specifically, the segmentation mask is a three-dimensional matrix of W×H×1, which can be saved in grayscale image format, where W and H are the image width and height respectively, and the third dimension is the pixel value. The pixel value corresponding to the particle area is 1, and the non-particle area is 0.

[0142] Specifically, while executing the neural network prediction to output the segmentation mask, a simple processing is performed on the segmentation mask, so that it can produce some by-products, such as the minimum bounding box of the mask, etc.

[0143] Specifically, the five shape indicators of the waste particles calculated in the segmentation mask data set generation unit include: the major-minor axis ratio AR is the ratio of the maximum Feret radius of the waste particles to the minimum Feret radius of the waste particles; the roundness Cir is the area ratio of the waste particle projection to the equal-perimeter circle; the convexity Sol is the area ratio of the waste particle projection to the minimum circumscribed polygon; the edge angle Ang is the ratio of the average maximum inscribed circle radius of each edge in the waste particle to the maximum inscribed circle radius; the roughness Tex is the ratio of the waste particle projection to the perimeter of the minimum circumscribed polygon.

[0144] Specifically, the calculation formulas for the five shape indicators are as follows:

[0145] A. The calculation formula for the major-minor axis ratio AR is:

[0146]

[0147] Among them, F max is the maximum Feret radius of the waste particles, F min is the minimum Feret radius of the waste particles;

[0148] B. The calculation formula of roundness Cir is:

[0149]

[0150] Where S is the area of ​​the spoil particle mask, and P is the perimeter of the spoil particle mask;

[0151] C. The calculation formula of concave-convexity Sol is:

[0152]

[0153] Among them, S is the mask area of ​​waste particles, S con is the area of ​​the minimum circumscribed polygon of the waste particle mask;

[0154] D. The calculation formula of edge angle Ang is:

[0155]

[0156] Among them, r i is the maximum inscribed circle radius of each corner in the waste particle mask; n is the number of corners in the waste particle mask; R is the maximum inscribed circle radius of the waste particle mask;

[0157] E. The calculation formula of roughness Tex is:

[0158]

[0159] Where P is the perimeter of the waste particle mask; P con is the perimeter of the minimum circumscribed polygon of the spoil particle mask.

[0160] The comprehensive shape factor dataset generation unit uses the improved TOPSIS method based on Mahalanobis distance to calculate the comprehensive shape factor of each waste particle according to the five shape indicators, and forms the comprehensive shape factor dataset S of the waste particles;

[0161] Specifically, the method for calculating the comprehensive shape factor in the segmentation mask dataset generation unit includes the following steps:

[0162] S41. Perform positive processing on the shape index according to the following formula:

[0163]

[0164] Among them, x' ij is a positive indicator; x ij It is the column vector element composed of the indicators to be positively converted;

[0165] S42. Calculate the positive ideal solution Y of the five shape indices + , the calculation formula is:

[0166]

[0167] Among them, Y + is the positive ideal solution of the index, x ij It is the column vector element composed of the indicators to be positively converted;

[0168] Calculate the negative ideal solution Y of five shape indices- , the calculation formula is:

[0169]

[0170] Among them, Y - is the negative ideal solution of the index, x ij It is the column vector element composed of the indicators to be positively converted;

[0171] S43, the normalized aspect ratio AR, edge angle Ang, roughness Tex, roundness Cir, and concavity Sol are used to form an evaluation index matrix X; each index X is calculated. i To the positive ideal solution Y + The Mahalanobis distance d(X i ,Y + ), the calculation formula is:

[0172]

[0173] Calculate each indicator X i To the negative ideal solution Y - The Mahalanobis distance d(X i ,Y - ), the calculation formula is:

[0174]

[0175] S44. Calculate the comprehensive shape factor f of each waste particle i , the calculation formula is:

[0176]

[0177] Specifically, the improved TOPSIS method based on Mahalanobis distance is to replace the distance scoring formula in TOPSIS with the Mahalanobis distance scoring formula;

[0178] The second data set generating unit is used to construct a morphological classification standard for waste particles based on the segmentation mask data set M1 and the comprehensive shape factor data set S, calculate the morphological grade L of each waste particle in the segmentation mask data set M1, and form a waste particle image data set D3 with morphological grade labels and marker labels;

[0179] Specifically, the morphological classification standard of the waste particles in the second data set generation unit is based on the comprehensive shape factor f of each waste particle. i The size of the morphological level L is divided into 5 levels: L1, L2, L3, L4, and L5; the waste particle image dataset D3 is automatically annotated according to the comprehensive shape factor dataset S and the segmentation mask dataset M1, and includes the original image set in the dataset M1 and the corresponding waste particle morphological level L label and marker label data set.

[0180] Preferably, the specific method of the morphological classification standard of the waste particles in the second data set generation unit is: according to the comprehensive shape factor f of each waste particle i The size of 0≤f i <0.2, 0.2≤f i <0.4, 0.4≤f i <0.6, 0.6≤f i <0.8, 0.8≤f i <1 is divided into 5 levels of morphological level L, namely L1, L2, L3, L4, and L5; the dataset D3 is automatically annotated based on the comprehensive shape factor dataset S and the segmentation mask dataset M1, and includes the original image set in the dataset M1 and the corresponding waste slag particle morphological level L labels and marker label data sets.

[0181] The second modeling unit is used to input the waste particle image dataset D3 into the neural network for training to obtain a second neural network model N2 including the waste particle outline and its morphological grade feature weights;

[0182] Preferably, in the second modeling unit, the image dataset D3 is randomly divided into a training set, a validation set, and a test set in a ratio of 8:1:1; the neural network adopts a mask-rcnn model with pre-trained weights based on the Microsoft Common Objects in Context dataset.

[0183] Specifically, the label of the input dataset D3 is morphological level L, so the segmentation neural network will output contours classified by morphological level, which is equivalent to the neural network N1 outputting a single-class (stone) contour mask, while N2 can output a contour mask classified according to morphological level including the TOPSIS grading method.

[0184] Specifically, the second neural network model N2 and the neural network model N1 are independent neural networks, but use the same neural network architecture and pre-trained data weights, and the neural networks used by both are instance segmentation neural networks;

[0185] The grading unit is used to input any newly collected waste particle image into the second neural network model N2, perform contour recognition and morphology detection of the waste particles, and output the waste particle segmentation mask data and morphology grade, and calculate the equivalent particle size D of the waste particles based on the segmentation mask data.

[0186] refer to Figure 3-Figure 5 The segmentation mask data in the hierarchical unit includes a set of mask images of a single piece of waste particles and a marker in the predicted image, a set of corner point coordinate data of a minimum bounding box of a single piece of waste particles and a marker, and the corresponding original predicted image.

[0187] Specifically, the calculation formula for the equivalent particle size D of the waste particles is:

[0188]

[0189] Among them, m R is the pixel area of ​​the waste particle mask; m s is the pixel area of ​​the marker mask; S s is the actual area of ​​the marker.

[0190] This example leverages the powerful learning capabilities of deep convolutional neural networks and the convenience of image processing technology. It only requires a general segmentation neural network-based auxiliary annotation tool to quickly annotate images containing tunnel spoil particles. After training on the annotated image data, the outlines of tunnel spoil particles can be detected and identified. A modified TOPSIS method based on Mahalanobis distance is then used to statistically classify the multi-scale features of the spoil particle morphology, ultimately achieving identification and particle morphology determination of tunnel spoil particles. This method is simple, efficient, and easy to use.

[0191] The auxiliary labeling tool developed based on the general segmentation neural network in this embodiment can greatly improve the efficiency of labeling the outline of the waste slag particles;

[0192] The improved TOPSIS classification method based on the statistical principle of Mahalanobis distance in this embodiment is simple to calculate. The resulting comprehensive shape factor can take into account the correlation between various shape indicators and eliminate the interference of different degrees of correlation between various indicators, thereby more accurately reflecting the morphological information relationship embodied by various indicators.

[0193] The tunnel waste particle recognition neural network finally obtained by training in this embodiment can not only automatically and accurately mark the outlines of tunnel waste particles, but also directly predict and output the morphological classification results of tunnel waste particles, and calculate the equivalent particle size distribution through simple post-processing, which can greatly simplify the manual workflow of quantitative extraction and classification of size and shape feature indicators, save time and manpower, and improve work efficiency.

[0194] Example 3

[0195] refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of a tunnel waste particle identification and grading device according to this embodiment. The tunnel waste particle identification and grading device 20 according to this embodiment includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps of the aforementioned method embodiments. Alternatively, when the processor 21 executes the computer program, it implements the functions of the modules / units in the aforementioned device embodiments.

[0196] Exemplarily, the computer program can be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to implement the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the tunnel waste particle identification and classification device 20. For example, the computer program can be divided into the modules described in Example 2. The specific functions of each module can be referred to the operating process of the device described in the above embodiment, and will not be repeated here.

[0197] The tunnel waste particle identification and classification device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will appreciate that the schematic diagram is merely an example of a tunnel waste particle identification and classification device 20 and does not limit the tunnel waste particle identification and classification device 20. The tunnel waste particle identification and classification device 20 may include more or fewer components than shown in the figure, or may combine certain components, or different components. For example, the tunnel waste particle identification and classification device 20 may also include input and output devices, network access devices, buses, and the like.

[0198] The processor 21 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The processor 21 is the control center of the tunnel waste particle identification and classification device 20, and utilizes various interfaces and lines to connect various parts of the entire tunnel waste particle identification and classification device 20.

[0199] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements the various functions of the tunnel waste particle identification and classification device 20 by running or executing the computer programs and / or modules stored in the memory 22 and accessing the data stored in the memory 22. The memory 22 can primarily include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.); the data storage area can store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory 22 can include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0200] Wherein, if the module / unit integrated in the tunnel waste particle identification and classification device 20 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor 21, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0201] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0202] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying and classifying tunnel waste particles, characterized in that: The steps include: S3, obtaining the segmentation mask dataset M1 of the waste slag particle contour; S4. Using an improved TOPSIS method based on Mahalanobis distance, the shape index and comprehensive shape factor of each waste particle are calculated according to the segmentation mask dataset M1 to form a comprehensive shape factor dataset S of the waste particles. The shape index includes the aspect ratio AR, roundness Cir, concavity Sol, angularity Ang, and roughness Tex. The improved TOPSIS method based on Mahalanobis distance replaces the distance scoring formula in TOPSIS with the Mahalanobis distance scoring formula. S5. Construct a morphological classification standard for waste particles based on the segmentation mask dataset M1 and the comprehensive shape factor dataset S, calculate the morphological grade L of each waste particle in the segmentation mask dataset M1, and form a waste particle image dataset D3 with morphological grade labels and marker labels; S6, inputting the waste particle image dataset D3 into the neural network for training to obtain a second neural network model N2 containing the waste particle outlines and their morphological grade feature weights; S7. Input any newly collected waste particle image into the second neural network model N2 to perform contour recognition and morphology detection of the waste particles, and output the waste particle segmentation mask data and morphology level, and calculate the waste particle equivalent particle size D based on the segmentation mask data.

2. The method according to claim 1, wherein: The method further comprises: S1, acquiring a waste slag particle image dataset D1 including waste slag particle contours and size marker contour labels; S2, inputting the waste particle image dataset D1 into the neural network for training to obtain a first neural network model N1 containing waste particle contour feature weights; Specifically, step S3 includes inputting the unlabeled waste particle image dataset D2 into the first neural network model N1 to identify and label the waste particle contours and markers, and outputting a segmentation mask dataset M1 forming the waste particle contours.

3. The method according to claim 2, characterized in that During the image acquisition process in step S1 , a size marker needs to be placed near the center of the image.

4. The method according to claim 2, characterized in that In step S2, the image dataset D1 is randomly divided into a training set, a validation set, and a test set in proportion.

5. The method according to claim 2, characterized in that The unlabeled waste particle image dataset D2 in step S3 is constructed by randomly selecting a second preset number of waste particle images from the images collected in step S1; the dataset M1 includes a set of mask images of each waste particle, a set of corner point coordinate data of the minimum bounding box of each waste particle outline, and a corresponding original image set.

6. The method according to claim 1, characterized in that The shape indicators of the waste particles calculated in step S4 include: the major-minor axis ratio AR is the ratio of the maximum Feret radius of the waste particle to the minimum Feret radius of the waste particle; the roundness Cir is the area ratio of the waste particle projection to the equal-perimeter circle; the convexity Sol is the area ratio of the waste particle projection to the minimum circumscribed polygon; the edge angle Ang is the ratio of the average maximum inscribed circle radius of each edge in the waste particle to the maximum inscribed circle radius; the roughness Tex is the ratio of the waste particle projection to the perimeter of the minimum circumscribed polygon.

7. The method according to claim 1, characterized in that: The method for calculating the comprehensive shape factor in step S4 comprises the following steps: S41. Perform positive processing on the shape index according to the following formula: Where x′ ij is a positive indicator; x ij It is the column vector element composed of the indicators to be positively converted; S42. Calculate the positive ideal solution Y of five shape indices + , the calculation formula is: Among them, Y + is the positive ideal solution of the index, x ij It is the column vector element composed of the indicators to be positively converted; Calculate the negative ideal solution Y of five shape indices - , the calculation formula is: Among them, Y - is the negative ideal solution of the index, x ij It is the column vector element composed of the indicators to be positively converted; S43, the normalized aspect ratio AR, edge angle Ang, roughness Tex, roundness Cir, and concavity Sol are used to form an evaluation index matrix X; each index X is calculated. i To the positive ideal solution Y + The Mahalanobis distance d(X i ,Y + ), the calculation formula is: Calculate each indicator X i To the negative ideal solution Y - The Mahalanobis distance d(X i ,Y - ), the calculation formula is: S44. Calculate the comprehensive shape factor f of each waste particle i , the calculation formula is:

8. The method according to claim 7, characterized in that The specific method of the morphological classification standard of the waste particles in step S5 is based on the comprehensive shape factor f of each waste particle. i The size of the morphological level L is divided into 5 levels: L1, L2, L3, L4, and L5; the waste particle image dataset D3 is automatically annotated according to the comprehensive shape factor dataset S and the segmentation mask dataset M1, and includes the original image set in the dataset M1 and the corresponding waste particle morphological level L label and marker label data set.

9. The method according to claim 1, characterized in that The calculation formula for the equivalent particle size D of the waste particles is: Among them, m R is the pixel area of ​​the waste particle mask; m s is the pixel area of ​​the marker mask; S s is the actual area of ​​the marker.

10. A tunnel waste particle identification and classification device, characterized by: Includes the following units: A segmentation mask data set generating unit, used for obtaining a segmentation mask data set M1 of the waste slag particle contour; A comprehensive shape factor dataset generation unit uses an improved TOPSIS method based on Mahalanobis distance to calculate the shape index and comprehensive shape factor of each waste particle according to the segmentation mask dataset M1, thereby forming a comprehensive shape factor dataset S of the waste particles. The shape index includes the aspect ratio AR, roundness Cir, concavity Sol, angular angle Ang, and roughness Tex. The improved TOPSIS method based on Mahalanobis distance replaces the distance scoring formula in TOPSIS with the Mahalanobis distance scoring formula. The second data set generating unit is used to construct a morphological classification standard for waste particles based on the segmentation mask data set M1 and the comprehensive shape factor data set S, calculate the morphological grade L of each waste particle in the segmentation mask data set M1, and form a waste particle image data set D3 with morphological grade labels and marker labels; The second modeling unit is used to input the waste particle image dataset D3 into the neural network for training to obtain a second neural network model N2 including the waste particle outline and its morphological grade feature weights; The grading unit is used to input any newly collected waste particle image into the second neural network model N2, perform contour recognition and morphology detection of the waste particles, and output the waste particle segmentation mask data and morphology grade, and calculate the equivalent particle size D of the waste particles based on the segmentation mask data.

Images