Coal slime flotation froth three-dimensional model classification method based on Pointnet network

Abstract

The invention relates to a coal slime flotation froth three-dimensional model classification method based on a Pointnet network, and aims to solve the technical problem that the current two-dimensional froth image cannot process the states of three-dimensional froth space fluctuation and the like and the internal relationship of the space, and the technical scheme is as follows: expanding a data set; designing a foam characteristic network architecture; extracting a high-dimensional feature lambda 1 by using a point cloud feature extractor f theta; projecting the high-dimensional feature lambda 1 into a low-dimensional space by using a feature projection head # imgabs0 # to obtain a low-dimensional feature lambda 2 of the single-groove foam point cloud; splicing and fusing the single-groove foam point cloud features by using the feature fusion function yf to obtain an overall feature gamma; classifying the overall feature gamma through a multi-layer perceptron (MLP) and a Softmax function, and outputting a classification result; evaluating and analyzing. According to the method, the foam three-dimensional point cloud models of a plurality of flotation cells are used as input, the limitation of a single-cell data source is made up, the generalization ability and stability of the model are improved by using the flotation machine overall feature training network, and the performance of the model is optimized.

Description

Technical Field

[0001] The invention belongs to the technical field of coal slime flotation, and in particular relates to a coal slime flotation foam three-dimensional model classification method based on a Pointnet network. Background Art

[0002] Froth flotation is one of the most widely used fine coal separation methods in coal slime flotation. At present, most coal preparation plants rely on the experience of flotation masters to judge the state of flotation foam and the current production conditions, and control them. However, the use of manual recognition has disadvantages such as long time consumption and instability, which reduces production efficiency and leads to reagent waste. In order to improve the efficiency of coal slime flotation, the intelligent development of coal slime flotation is an important research direction for the development of coal slime flotation. With the rapid development of computer vision, it has been widely used in many fields. In the context of coal slime flotation, using computer vision instead of human vision to judge flotation foam can solve problems such as production instability and index fluctuations caused by intuitive operation during flotation.

[0003] At present, most studies use computer vision to study and analyze flotation foam images. Oktaba, H. et al. processed flotation foam images by using techniques based on image color analysis and fast Fourier transform, and extracted global features from the visual features of the surface foam. The experiment showed that the image processing system can distinguish different copper contents in rougher foam. He Mingfang proposed a nonparametric kernel estimation method based on a fixed kernel basis, constructed a combination of a normal kernel of a sparse multi-kernel least squares support vector machine classifier based on a polynomial kernel and a polynomial support vector machine classifier, and achieved performance classification. Gui Weihua et al. calculated the color co-occurrence matrix after spatial color conversion of the flotation foam image, extracted feature statistics after normalization, and designed texture complexity based on the feature statistics to describe the foam texture condition. Melissa Kistner et al. used primitives of high-order statistical laws and complex multi-scale wavelet representations to model foam texture. The above studies all processed flotation foam images through machine learning, and the advancement of computer hardware has led to the rapid development of deep learning. Deep learning algorithms can simulate the processing method of the human brain and extract features from images. Convolutional neural networks dominate the field of image processing. In flotation foam image processing, neural convolutional networks can directly extract image features, which has a higher accuracy than traditional machine learning. Wen, Zhiping et al. used convolutional neural networks to build a flotation soft sensor for predicting the ash content of coal flotation concentrate. Fu, Y., and C. Aldrich et al. used a pre-trained neural network architecture to estimate the foam grade from industrial image data. Wang, Xiaoli segmented the foam image into individual foam images, assigned labels using K-means clustering, extracted pixel set features of each bubble image using a convolutional neural network, and used a weighted mean shift algorithm to fuse the morphological feature vector and pixel set features of the bubble image, calculated the frequency of various types of bubbles in the foam image, and formed a bubble frequency set of the foam image.

[0004] However, flotation foam is a three-dimensional object with rich spatial features inside. The height and fluctuation of flotation foam are different under different working conditions. The image acquisition process is a process of converting three-dimensional space into a two-dimensional plane image, and a lot of three-dimensional space information is lost during the conversion process. Therefore, we cannot only consider the flotation foam processing problem from the perspective of a two-dimensional plane image, but should regard it as a three-dimensional space for analysis and processing. In the actual production process, the flotation operator needs to observe the foam state of all flotation cells, but the current research only processes the foam data of the flotation cell. Summary of the invention

[0005] The purpose of the present invention is to solve the above problems and provide a method for classifying a three-dimensional model of coal slime flotation foam based on a Pointnet network.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A classification method for a three-dimensional model of coal slime flotation foam based on a Pointnet network. The acquisition and processing system used in the three-dimensional model of flotation foam includes a dosing device, a preprocessor, a flotation machine, a stereo camera, a camera bracket, a data storage and a computer processing system. The stereo camera uses a structured light binocular stereo camera. The classification method includes the following steps:

[0008] Step 1: Using the point cloud model as the representation of foam 3D data, first pre-process the original data collected by the stereo camera, remove the abnormal data in the data collection, crop the point cloud in the same area to normalize the point cloud area, and then perform data enhancement operations, including rotation, movement and scaling, to obtain an extended database. Use 80% of the extended database as the training set, and the remaining 20% ​​of the data is used for testing;

[0009] Step 2: Design a foam feature network architecture based on the Pointnet network improvement and integration of several flotation cell foam features;

[0010] Step 3: Use the point cloud feature extractor f θ Extract high-dimensional features λ1 from the foam point cloud;

[0011] Step 4: Using the Feature Projection Head Project the high-dimensional feature λ1 into the low-dimensional space to obtain the low-dimensional feature λ2 of the single-slot foam point cloud;

[0012] Step 5: Use feature fusion function y f The single-slot foam point cloud features are spliced ​​and fused to obtain the overall feature γ;

[0013] Step 6: Classify the overall feature γ through the multi-layer perceptron MLP and Softmax function, and output the classification result;

[0014] Step 7: Use the confusion matrix to evaluate the performance of the classification model.

[0015] Furthermore, in step 1, the point cloud model is represented as a set of 3D points {x k |k=1,...,n}, where each point x k are its coordinates (x,y,z).

[0016] Furthermore, the preprocessing and data enhancement of each data in step 1 are random, so as to maximize the difference of the foam model and obtain a richer extended data set.

[0017] Furthermore, in step 3, a point cloud feature extractor f is usedθ The steps to extract high-dimensional features λ1 from the foam point cloud are as follows:

[0018] The point cloud feature extractor f θ Select the symmetry function to process the input point cloud. The definition of the symmetry function is as follows:

[0019]

[0020] in for (x1,...,x n ), it can be seen from the above formula that the output value of the symmetric function does not change with the order of the input; the point cloud feature extractor f θ The symmetric function chosen is the maximum function:

[0021] f(x1,x2,...,x n )=max{x1,x2,...,x n} (2)

[0022] Using only the maximum function to process foam point cloud data will result in incomplete network learning and loss of important information. Therefore, the point cloud feature extractor f based on the Pointnet network is θ Constructing a high-dimensional space mapping network, mapping each 3D point to a high-dimensional space and then performing a symmetric function operation can reduce the loss of features, make the network richer, and enhance stability. The obtained foam point cloud high-dimensional feature λ1 is expressed as:

[0023] λ1=g(h(x1),...,h(x n )) (3)

[0024] Among them, h(·) is the dimension-raising function, g(·) is the maximum function, x1,...,x n is the 3D point of the input foam point cloud model;

[0025] According to the transformation invariance of point cloud, the point cloud feature extractor f θ It is expected that the learned high-dimensional features will not be affected by the point cloud transformation. Therefore, the network introduces a rotation matrix so that all input foam point clouds are aligned to the same standard space. The point cloud feature extractor f θ The affine transformation matrix is ​​predicted by the T network, and the transformation is used to process the input foam point cloud model. Similarly, the T network applies another alignment network to the point features after dimensionality increase, and predicts a feature transformation matrix to align the dimensionality increase features of different input point clouds. Due to the high dimensionality of the feature space, the optimization difficulty increases. In order to improve the calculation speed of gradient descent, the network uses a regularization term to train the loss function. The gradient descent is performed along the direction of the regularization term, and the constraint feature is an orthogonal matrix composed of alignment matrices:

[0026]

[0027] Among them, A is the feature alignment matrix predicted by the T network. By using the regularization term, the optimized network is more stable;

[0028] From the above, we can see that the point cloud feature extractor f θ It has two major modules: an alignment network is used to calibrate the direction of the point cloud, a multi-layer perceptron with shared weights maps the input point cloud to the same feature space with the same dimension, and a maximum pooling layer is used as a symmetric function to aggregate the information of all input points and extract the high-dimensional feature representation λ1 of the foam point cloud:

[0029]

[0030] Where, i = 1, 2, 3, 4, 5, i is the number of flotation cells, is the input single-slot foam point cloud, k=1,...,n is the number of input points, h(·) is the dimension-raising function, is the high-dimensional point after dimensionality upgrade, are affine transformation matrices with two different dimensions, which are obtained through T network training. f(·) is the maximum function, which is used to select the maximum value of each dimension.

[0031] Furthermore, in step 4, a feature projection head is used Project the high-dimensional features into the low-dimensional space to obtain the low-dimensional features λ2 of the single-slot foam point cloud:

[0032]

[0033] Where, i = 1, 2, 3, 4, 5, i is the number of flotation cells, We chose to use a two-layer perceptron MLP, which is a typical feedforward network based on back-propagation learning. It consists of an input layer, a hidden layer, and an output layer. Each layer is fully connected to the next layer. Except for the input node, each node is a neuron with a nonlinear activation function. The hidden layer realizes nonlinear mapping of the input space. The BP back-propagation algorithm is used to train the MLP.

[0034] Furthermore, in step 5, the single slot feature λ2 is fused using a cascade operation, y f Select the CONCAT() concatenation function to get the overall characteristic γ of the flotation machine foam as follows:

[0035]

[0036] Among them, CONCAT() is a concatenation function. Through the above calculation, the overall foam characteristic γ of the flotation machine is obtained. γ includes the foam characteristics of each flotation cell. Compared with the foam data of a single flotation cell, the overall foam characteristic of the flotation machine contains richer information. This global feature can more accurately describe the overall foam state of the current flotation machine, so that the training model can learn more comprehensive knowledge, which is conducive to improving the accuracy of the model, increasing the robustness of the model, and enabling it to perform excellent performance under complex working conditions.

[0037] Furthermore, in step 6, the multi-layer perceptron MLP and Softmax function are used to classify the overall feature γ of the foam. Softmax converts the classification value into a probability distribution, and selects the highest probability label as the output result to realize the classification of the foam point cloud model, and finally realize the recognition of different states of the foam:

[0038]

[0039] In the formula, z i It is the input of Softmax layer, with 3 types of flotation foam, P i is the output result.

[0040] Furthermore, in step 7, a comparative experiment is used to evaluate the performance of the network proposed by the classification model. Network 1 is the result of a single-tank flotation foam identified using only the Pointnet network, Network 2 integrates the foam features of two flotation tanks for foam identification, Network 3 is the test result of using the foam data of three flotation tanks as input for fusion and classification, and Network 4 is the output result of this classification model;

[0041] The confusion matrix is ​​plotted using the test set data to analyze the flotation foam recognition results of the four networks on the test set. The precision, recall, and F1 value indicators of the model in each category are calculated through the confusion matrix, thereby comprehensively evaluating the performance of the model in the flotation foam recognition task.

[0042] In the multi-classification task, for the i-th class sample data, the number of samples correctly predicted by the model for the i-th class is defined as True Positive (TP), the number of samples incorrectly predicted by the model for the i-th class is defined as False Positive (FP), the number of samples correctly predicted by the model for non-i-th class is defined as True Negative (TN), and the number of samples incorrectly predicted by the model for non-i-th class is defined as False Negative (FN);

[0043] Precision refers to the proportion of samples with actual labels of the i-th category among the samples predicted by the model to be of the i-th category. The calculation formula is:

[0044]

[0045] Recall rate refers to the proportion of samples correctly predicted by the model as class i among all samples whose actual label is class i. A high recall rate means that the model can effectively capture samples whose actual label is class i. The calculation formula is:

[0046]

[0047] Generally, there is a trade-off between precision and recall. Improving precision will reduce recall, and vice versa. F1 value is the harmonic mean of precision and recall, which takes into account the precision and recall of the model. Under unbalanced category distribution, F1 value can more accurately reflect the performance of the model than accuracy. The calculation formula is:

[0048]

[0049] Calculate the precision, recall and F1 value of each category under different networks respectively, and calculate the average precision, recall and F1 value of each network;

[0050] The receiver operating characteristic (ROC) curve is a tool for evaluating the performance of a classification model. The horizontal axis represents the false positive rate (FPR) and the vertical axis represents the true positive rate (TPR). The ROC curve can show the impact of different thresholds on the performance of the classifier and help select the optimal threshold. It is defined as:

[0051]

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. The present invention solves the problem that two-dimensional foam images cannot process the three-dimensional foam space ups and downs and the internal relationship of the space. At the same time, the foam three-dimensional point cloud model of multiple flotation tanks is used as input to make up for the limitations of a single tank data source. The overall feature training network of the flotation machine is used to improve the generalization ability and stability of the model, and optimize the performance of the model.

[0054] 2. The multi-tank data fusion network based on Pointnet proposed in the present invention is superior to the single-tank data network, the double-tank data fusion network and the triple-tank data fusion network. The network can efficiently identify the flotation foam type, has high accuracy, high stability, low misjudgment rate and good generalization ability, and provides new ideas for the intelligent flotation production;

[0055] 3. The present invention uses deep learning and three-dimensional modeling methods to classify flotation foam. The model has a good classification effect, which helps to accurately identify the type of flotation foam, provide guidance for the judgment of the foam state in the flotation process, and avoid the subjectivity and lag of adjusting the dosage by relying solely on the observation of the flotation driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a structural schematic diagram of the data acquisition and processing system of the present invention;

[0057] Figure 2 The photos of the flotation foam model under different dosing ratios of the present invention are shown;

[0058] Figure 3 It is a schematic diagram of the classification method flow of the present invention;

[0059] Figure 4 This is a schematic diagram of the multi-slot data fusion network structure based on Pointnet improvement of the present invention;

[0060] Figure 5 The flotation foam feature extractor based on the Pointnet network of the present invention is θ Network structure diagram;

[0061] Figure 6 The characteristic projection head of the present invention Schematic diagram of the network structure;

[0062] Figure 7 Schematic diagram of the Softmax function structure of the present invention;

[0063] Figure 8 A schematic diagram of a network structure of a comparative experiment of the present invention;

[0064] Fig. 9 A schematic diagram of the structure of the second network for comparative experiments of the present invention;

[0065] Fig.10 A schematic diagram of the third structure of the network for comparative experiments of the present invention;

[0066] Fig.11 A comparison chart of the accuracy and loss values ​​of different networks in the embodiment of the present invention;

[0067] Fig.12 The confusion matrix result diagram of different networks obtained in the embodiments of the present invention;

[0068] Where (a) is the confusion matrix result diagram of network one;

[0069] (b) is the confusion matrix result diagram of network 2;

[0070] (c) is the confusion matrix result diagram of network three;

[0071] (d) is a confusion matrix result diagram of the network of the present invention;

[0072] Fig.13 The precision, recall, F1 value and their average values ​​obtained by different networks in the embodiments of the present invention are shown in the figure;

[0073] Where (a) is the result graph of network 1;

[0074] (b) is the result graph of network 2;

[0075] (c) is the result graph of network 3;

[0076] (d) is the result graph of the network of the present invention;

[0077] Fig.14 The ROC curves and AUC result diagrams obtained by different networks in the embodiments of the present invention;

[0078] Where (a) is the result graph of network 1;

[0079] (b) is the result graph of network 2;

[0080] (c) is the result graph of network 3;

[0081] (d) is the result graph of the network of the present invention; DETAILED DESCRIPTION

[0082] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0083] This embodiment takes a coal preparation plant in Taiyuan, Shanxi as the research object. Since the foam area hanging next to the scraper is too small and the flow speed is too fast, it is not easy to shoot, while the foam flow rate next to the aerator is moderate and the area is large, so the foam area next to the aerator is selected as the shooting area. Although the flotation process is dynamically changing, the flow rate of the flotation foam is relatively stable, which means that the conversion of the working conditions is not frequent. Although the flotation process changes in real time, the conversion of the working conditions is not frequent. The flow rate of the flotation foam is usually maintained between 0.8cm / s and 2.4cm / s. Therefore, it is selected to sample the flotation foam once every 0.5s, thereby establishing a three-dimensional model database of the foam. The original database is enhanced to obtain an extended database, 80% of the extended database is used as a training set, and the remaining 20% ​​of the data is used for testing. The present invention uses loss value and overall accuracy as evaluation indicators, the batch size is 16, and the dropout layer is used to reduce the dependency between neurons, reduce the risk of overfitting, and improve the generalization ability of the model. The network is iterated 100 times to obtain a flotation foam recognition model.

[0084] like Figure 1As shown in the figure, the acquisition and processing system of the flotation foam three-dimensional model is mainly composed of a stereo camera, a camera bracket, data storage and a computer processing system. The stereo camera selected is the Tuyang infrared binocular FM855-E1 industrial camera, which is a structured light binocular stereo camera. By actively emitting structured light, the camera can accurately capture the spatial information of the flotation foam, and with the help of the built-in processor, it can realize depth perception and instant reconstruction of the three-dimensional model. At the same time, the camera uses a network port interface to quickly transmit data and realize low-latency three-dimensional reconstruction of the flotation foam. Considering the huge amount of data of the three-dimensional model, the data storage and computer processing system is built with NVIDIA GeForce RTX 3090 (GPU processor) and Core i912900K (CPU processor). The Anaconda platform is used to build a development environment based on Python3.8 on the Window11 operating system to achieve efficient data processing. This configuration ensures that the entire flotation foam three-dimensional modeling system can run efficiently and stably to meet the requirements of industrial-grade applications.

[0085] In the flotation process, the dosage of reagents is closely related to the morphology of flotation foam. The right dosage of reagents can improve the properties of running foam, improve production efficiency and clean coal grade, while improper dosage of reagents will lead to poor foam stability and easy collapse. At present, the flotation reagents used in most coal preparation plants are mainly frothers and collectors. The main function of frothers is to help form a stable foam layer, which is essential for the flotation process. It reduces the surface tension of water, making it easier to form bubbles, and maintains the stability of bubbles to prevent them from breaking too quickly. Good foam should have sufficient carrying capacity to carry mineral particles and flow to the surface of the flotation tank and then be scraped away. Collectors target the surface properties of specific minerals and enhance the adhesion of mineral particles to bubbles. Collector molecules usually have polar groups at one end, which can selectively adsorb on the surface of the target mineral; while the other end is a hydrophilic and hydrophobic group, which helps mineral particles repel water molecules in the water medium and combine with air foam. Therefore, the selection of collectors is selective, which can enable useful minerals to have flotation ability without affecting other unnecessary minerals.

[0086] The present invention collected the flotation foam three-dimensional model when the addition ratio of frother and collector was 3:2, 2:1, and 1:1. The foam model of different dosages is as follows Figure 2 shown. Figure 2 The Class 1 shown is a Class 1 foam model. The ratio of frother to collector is 3:2. The foam is in the best state, with moderate size and uniform distribution, strong stability, and can effectively capture useful coal particles, thereby improving the recovery rate of useful minerals. Figure 2The Class 2 shown is a Class 2 foam model. The ratio of frother to collector is 2:1. The foam is small and dense, the flotation foam has poor fluidity, and the foam contains too many non-target minerals, which increases the impurity content in the concentrate and reduces the quality of the mineral. Figure 2 The Class 3 shown is the third type of foam model. The ratio of frother to collector is 1:1. The adsorption capacity of coal particles is weakened, and the foam contains too much water, resulting in a larger foam shape and insufficient stability, which is easy to collapse. The flotation foam flow rate increases and the clean coal recovery rate decreases.

[0087] like Figure 3-10 As shown, a classification method for a three-dimensional model of coal slime flotation foam based on a Pointnet network is provided, and the classification method comprises the following steps:

[0088] Step 1: The point cloud model is used as the representation of the foam three-dimensional data. Since the flotation foam is in a flowing state during the production process, the collected foam model has problems such as blurred boundary information and noise points. The original data is first preprocessed to remove abnormal data in data collection, and the point cloud is cropped in the same area to normalize the point cloud area. Due to the limited amount of collected data, data enhancement of the foam three-dimensional model can improve the robustness of the model and avoid overfitting. Therefore, a series of data enhancement operations are performed on the preprocessed data, including rotation, movement and scaling, to obtain an extended database. 80% of the extended database is used as the training set, and the remaining 20% ​​of the data is used for testing; in actual operation, the processing method of each data is random, and the rotation angle, movement distance, and scaling ratio are randomly selected. The purpose is to maximize the difference of the foam model, and the obtained extended data set is richer, so that the model can obtain better generalization ability and higher accuracy during the training process. The quantity distribution of the original data set and the extended data set is shown in Table 1.

[0089] Table 1 Number of foam models in the original and extended datasets

[0090]

[0091] The point cloud model is represented as a collection of 3D points {x k |k=1,...,n}, where each point x k is its coordinates (x, y, z), plus other feature channels, the foam point cloud model has the following characteristics:

[0092] Disorder: Point cloud is different from the pixel array in the image. It is a set of points without a specific order. If we process a set of N 3D points, there are N! kinds of input order of 3D points.

[0093] Interactions between points: The point cloud model comes from a space with a distance metric. Each point is not isolated, and adjacent points form a meaningful subset. The point cloud feature extractor needs to have the ability to capture the local neighborhood structure.

[0094] Transformation invariance: When a point cloud is rotated or translated, its actual meaning remains unchanged, so the output features of the point cloud feature extractor should also be the same.

[0095] Step 2: Design a foam feature network architecture based on the improved Pointnet network and integrating the foam features of 5 flotation cells;

[0096] The present invention designs a network architecture that integrates the foam characteristics of five flotation tanks. The network uses the Pointnet network to extract the flotation foam characteristics of the five tanks, and then fuses the obtained single tank characteristics and sends them to the classifier for classification. Since the network is trained with the foam data of all flotation tanks, the trained model can better judge the state of flotation foam, thereby adjusting the dosage.

[0097] The network consists of 5 branches, which extract the point cloud features of 5 slots respectively. Different branches perform the same operation on the input point cloud, but the weights are not shared.

[0098] Step 3: Use the point cloud feature extractor f θ Extract high-dimensional features λ1 from the foam point cloud;

[0099] Point cloud feature extractorf θ Using the Pointnet network, the input point cloud is rotated, dimensioned, and pooled to obtain a point cloud high-dimensional feature with a dimension of 1024.

[0100] In view of the disorder of point clouds and the interaction characteristics between points, the point cloud feature extractor f θ Select the symmetry function to process the input point cloud. The definition of the symmetry function is as follows:

[0101]

[0102] in for (x1,...,x n ), it can be seen from the above formula that the output value of the symmetric function does not change with the order of the input; the point cloud feature extractor f θ The symmetric function chosen is the maximum function:

[0103] f(x1,x2,...,x n )=max{x1,x2,...,x n} (2)

[0104] Using only the maximum function to process foam point cloud data will result in incomplete network learning and loss of important information. Therefore, the point cloud feature extractor f based on the Pointnet network is θ Constructing a high-dimensional space mapping network, mapping each 3D point to a high-dimensional space and then performing a symmetric function operation can reduce the loss of features, make the network richer, and enhance stability. The obtained foam point cloud high-dimensional feature λ1 is expressed as:

[0105] λ1=g(h(x1),...,h(x n )) (3)

[0106] Among them, h(·) is the dimension-raising function, g(·) is the maximum function, x1,...,x n is the 3D point of the input foam point cloud model;

[0107] According to the transformation invariance of point cloud, the point cloud feature extractor f θ It is expected that the learned high-dimensional features will not be affected by the point cloud transformation. Therefore, the network introduces a rotation matrix so that all input foam point clouds are aligned to the same standard space. The point cloud feature extractor f θ The affine transformation matrix is ​​predicted by the T network, and the transformation is used to process the input foam point cloud model. Similarly, the T network applies another alignment network to the point features after dimensionality increase, and predicts a feature transformation matrix to align the dimensionality increase features of different input point clouds. Due to the high dimensionality of the feature space, the optimization difficulty increases. In order to improve the calculation speed of gradient descent, the network uses a regularization term to train the loss function. The gradient descent is performed along the direction of the regularization term, and the constraint feature is an orthogonal matrix composed of alignment matrices:

[0108]

[0109] Among them, A is the feature alignment matrix predicted by the T network. By using the regularization term, the optimized network is more stable;

[0110] From the above, we can see that the point cloud feature extractor f θ It has two major modules: an alignment network is used to calibrate the direction of the point cloud, a multi-layer perceptron with shared weights maps the input point cloud to the same feature space with the same dimension, and a maximum pooling layer is used as a symmetric function to aggregate the information of all input points and extract the high-dimensional feature representation λ1 of the foam point cloud:

[0111]

[0112] Where, i = 1, 2, 3, 4, 5, i is the number of flotation cells, is the input single-slot foam point cloud, k=1,...,n is the number of input points, h(·) is the dimension-raising function, It is a high-dimensional point after dimensionality upgrade, with a feature dimension of 1024. are affine transformation matrices with two different dimensions, which are obtained through T network training. f(·) is the maximum function, which is used to select the maximum value of each dimension.

[0113] Step 4: Using the Feature Projection Head The high-dimensional feature λ1 is projected into the low-dimensional space to obtain the low-dimensional feature λ2 of the single-slot foam point cloud; due to the high feature dimension, the computational complexity is large when performing feature fusion, so the feature projection head is used Project high-dimensional features into low-dimensional space, feature projection head It consists of a two-layer multilayer perceptron, which is in the invariant space Generate a 256-dimensional feature projection vector and obtain the low-dimensional point cloud feature λ2:

[0114]

[0115] Where, i = 1, 2, 3, 4, 5, i is the number of flotation cells, We chose to use a two-layer perceptron MLP, which is a typical feedforward network based on back-propagation learning. It consists of an input layer, a hidden layer, and an output layer. Each layer is fully connected to the next layer. Except for the input node, each node is a neuron with a nonlinear activation function. The hidden layer realizes nonlinear mapping of the input space. The BP back-propagation algorithm is used to train the MLP.

[0116] Step 5: Use feature fusion function y f The single-slot foam point cloud features are spliced ​​and fused to obtain the overall feature γ;

[0117] Use cascade operation to fuse single slot feature λ2, y f Select the CONCAT() concatenation function to get the overall characteristic γ of the flotation machine foam as follows:

[0118]

[0119] Among them, CONCAT() is the concatenation function, and the feature dimension of γ is 256*5=1280. Through the above calculation, the overall foam feature γ of the flotation machine is obtained. γ includes the foam characteristics of each flotation cell. Compared with the foam data of a single flotation cell, the overall foam feature of the flotation machine contains richer information. This global feature can more accurately describe the overall foam state of the current flotation machine, so that the training model can learn more comprehensive knowledge, which is conducive to improving the accuracy of the model, increasing the robustness of the model, and enabling it to perform excellent performance under complex working conditions.

[0120] Step 6: Use the multi-layer perceptron MLP and Softmax function to classify the overall feature γ of the foam and output the classification result; Softmax converts the classification value into a probability distribution, selects the highest probability label as the output result, realizes the classification of the foam point cloud model, and finally realizes the recognition of different states of the foam:

[0121]

[0122] In the formula, z i It is the input of Softmax layer, with 3 types of flotation foam, P i is the output result.

[0123] Step 7: A comparative experiment is used to evaluate the performance of the network proposed by the classification model. Network 1 is the result of a single-tank flotation foam identified using only the Pointnet network. Network 2 integrates the foam features of two flotation tanks for foam identification. Network 3 is the experimental result of using the foam data of three flotation tanks as input for fusion and classification. Network 4 is the output result of this classification model.

[0124] like Fig.11 Table 2 lists the classification accuracy of coal slime flotation foam of Network 1, Network 2, Network 3 and the improved network based on Pointnet proposed in this embodiment. Among them, the accuracy of the improved network based on Pointnet is 94.89%, which is the highest among the four networks. Compared with the other three networks, the accuracy is increased by 27.30%, 48.14% and 10.54% respectively; the loss value of the improved network based on Pointnet is 0.2169, which is reduced by 0.2544, 0.4103 and 0.024 respectively compared with the other three networks. It can be seen that the network of this embodiment has excellent generalization ability and stable recognition ability, and the gap between the predicted data of the network model and the actual data is the smallest.

[0125] Table 2 Accuracy and loss values ​​of different networks

[0126]

[0127] The confusion matrix is ​​a commonly used tool in machine learning to evaluate the performance of classification models. It shows the relationship between the prediction results and the actual labels of the classification model in the form of a matrix, which can intuitively understand the pros and cons of the model. Each row of the confusion matrix represents the actual category, and each column represents the prediction result. The elements on the diagonal of the matrix represent the number of correctly predicted samples, and the off-diagonal elements represent the number of incorrectly predicted samples.

[0128] This example uses the test set data to draw a confusion matrix and analyzes the flotation foam recognition results of the four networks on the test set. The obtained confusion matrix is ​​as follows: Fig.12shown.

[0129] In network 1, 730 foam point cloud models were successfully identified, and 363 foam point cloud models were identified as wrong types. Among them, the recognition accuracy of the second type of foam model was the lowest, only 18.77%, and 80.94% of the second type of foam models were identified as the third type of foam models. This is because the second and third types of foam models are similar in morphology and difficult to distinguish. In network 2, 396 foam point cloud models were successfully identified, and 450 foam point cloud models were identified as wrong types. The recognition accuracy of the first type of foam model was only 26.21%, of which 70.34% were incorrectly identified as the second type of foam models. In network 3, 715 foam point cloud models were successfully identified, and 131 foam point cloud models were identified as wrong types. Among them, the recognition accuracy of the first and second types of foam models was high. Only 57.62% of the third type of foam models were correctly identified, and the rest of the foam models were all incorrectly identified as the second type. The improved classification network based on Pointnet proposed in this embodiment has 801 foam point cloud models successfully identified, and 42 foam point cloud models are identified as wrong types. The recognition accuracy of various foam models is quite high, among which the recognition accuracy of the third type of foam model has been greatly improved, reaching 88.06%. Based on the above analysis, it can be concluded that the flotation foam recognition network based on Pointnet proposed in this embodiment shows extremely high accuracy in the recognition of foam point cloud models, far exceeding other networks. This proves the powerful performance of the model in recognition ability and excellent network performance. The model can be applied to the accurate recognition of foam categories in the flotation process.

[0130] The model's precision, recall, and F1 value indicators for each category are calculated through the confusion matrix, so as to comprehensively evaluate the performance of the model in the flotation foam identification task;

[0131] In the multi-classification task, for the i-th class sample data, the number of samples correctly predicted by the model for the i-th class is defined as True Positive (TP), the number of samples incorrectly predicted by the model for the i-th class is defined as False Positive (FP), the number of samples correctly predicted by the model for non-i-th class is defined as True Negative (TN), and the number of samples incorrectly predicted by the model for non-i-th class is defined as False Negative (FN);

[0132] Precision refers to the proportion of samples with actual labels of the i-th category among the samples predicted by the model to be of the i-th category. The calculation formula is:

[0133]

[0134] Recall rate refers to the proportion of samples correctly predicted by the model as class i among all samples whose actual label is class i. A high recall rate means that the model can effectively capture samples whose actual label is class i. The calculation formula is:

[0135]

[0136] Generally, there is a trade-off between precision and recall. Improving precision will reduce recall, and vice versa. F1 value is the harmonic mean of precision and recall, which takes into account the precision and recall of the model. Under unbalanced category distribution, F1 value can more accurately reflect the performance of the model than accuracy. The calculation formula is:

[0137]

[0138] Calculate the precision, recall and F1 value of each category under different networks respectively, and calculate the average precision, recall and F1 value of each network. The calculation results are as follows Fig.13 As shown;

[0139] The receiver operating characteristic (ROC) curve is a tool for evaluating the performance of a classification model. The horizontal axis represents the false positive rate (FPR) and the vertical axis represents the true positive rate (TPR). The ROC curve can show the impact of different thresholds on the performance of the classifier and help select the optimal threshold. It is defined as:

[0140]

[0141] The ROC curve and AUC of each network are as follows Fig.14 As shown in the figure, the closer the ROC curve is to the upper left corner, the higher the accuracy of the model and the lower the false positive rate. The point on the ROC curve closest to the upper left corner is the point where the classifier achieves the best performance, with the least number of false positives and false negatives, and the threshold at this point is the optimal threshold. The area under the curve (AUC) can quantify the ROC curve and is used to measure the performance of the classifier. The value range of AUC is between 0.5 and 1. When AUC is equal to 1, it means that the performance of the classifier is perfect; when AUC is equal to 0.5, it means that the performance of the classifier is equivalent to random guessing.

[0142] Depend on Fig.14 It can be seen that the ROC curves of each category of the five-slot data fusion network improved based on Pointnet are closest to the upper left corner, and the AUC of each category is the highest. Compared with network one, network two and network three, this network has the strongest recognition ability for flotation foam and has better generalization ability and stability.

Claims

1. A three-dimensional model classification method for coal slime flotation foam based on Pointnet network, characterized by: The collection and processing system used in the flotation foam three-dimensional model includes a drug delivery device, a preprocessor, a flotation machine, a stereo camera, a camera bracket, a data storage and a computer processing system. The stereo camera uses a structured light binocular stereo camera. The classification method includes the following steps: Step 1: Using the point cloud model as the representation of foam 3D data, first pre-process the original data collected by the stereo camera, remove the abnormal data in the data collection, crop the point cloud in the same area to normalize the point cloud area, and then perform data enhancement operations, including rotation, movement and scaling, to obtain an extended database. Use 80% of the extended database as the training set, and the remaining 20% ​​of the data is used for testing; Step 2: Design a foam feature network architecture based on the Pointnet network improvement and integration of several flotation cell foam features; Step 3: Use the point cloud feature extractor f θ Extract high-dimensional features λ1 from the foam point cloud; Step 4: Using the Feature Projection Head Project the high-dimensional feature λ1 into the low-dimensional space to obtain the low-dimensional feature λ2 of the single-slot foam point cloud; Step 5: Use feature fusion function y f The single-slot foam point cloud features are spliced ​​and fused to obtain the overall feature γ; Step 6: Classify the overall feature Y through the multi-layer perceptron MLP and Softmax function, and output the classification result; Step 7: Use the confusion matrix to evaluate the performance of the classification model.

2. A method for classifying a three-dimensional model of coal slime flotation foam based on Pointnet network according to claim 1, characterized in that: In step 1, the point cloud model is represented as a set of 3D points {x k |k=1,...,n}, where each point x k are its coordinates (x, y, z).

3. The method for classifying a three-dimensional model of coal slime flotation foam based on Pointnet network according to claim 1, characterized in that: In step 1, the preprocessing and data enhancement of each data are random, so as to maximize the difference of the foam model and obtain a richer extended data set.

4. The method for classifying a three-dimensional model of coal slime flotation foam based on Pointnet network according to claim 1, characterized in that: In step 3, the point cloud feature extractor f is used θ The steps to extract high-dimensional features λ1 from the foam point cloud are as follows: The point cloud feature extractor f θ Select the symmetry function to process the input point cloud. The definition of the symmetry function is as follows: in for (x1, ..., x n ), it can be seen from the above formula that the output value of the symmetric function does not change with the order of the input; Point cloud feature extractorf θ The symmetric function chosen is the maximum function: f(x1,x2,…,x n )=max{x1,x2,...,x n } (2) Using only the maximum function to process foam point cloud data will result in incomplete network learning and loss of important information. Therefore, the point cloud feature extractor f based on the Pointnet network is θ Constructing a high-dimensional space mapping network, mapping each 3D point to a high-dimensional space and then performing a symmetric function operation can reduce the loss of features, make the network richer, and enhance stability. The obtained foam point cloud high-dimensional feature λ1 is expressed as: λ1=g(h(x1),…,h(x n )) (3) Among them, h(·) is the dimension-raising function, g(·) is the maximum value function, x1,...,x n is the 3D point of the input foam point cloud model; According to the transformation invariance of point cloud, the point cloud feature extractor f θ It is expected that the learned high-dimensional features will not be affected by the point cloud transformation. Therefore, the network introduces a rotation matrix so that all input foam point clouds are aligned to the same standard space. The point cloud feature extractor f θ The affine transformation matrix is ​​predicted by the T network, and the transformation is used to process the input foam point cloud model. Similarly, the T network applies another alignment network to the point features after dimensionality increase, and predicts a feature transformation matrix to align the dimensionality increase features of different input point clouds. Due to the high dimensionality of the feature space, the optimization difficulty increases. In order to improve the calculation speed of gradient descent, the network uses a regularization term to train the loss function. The gradient descent is performed along the direction of the regularization term, and the constraint feature is an orthogonal matrix composed of alignment matrices: Among them, A is the feature alignment matrix predicted by the T network. By using the regularization term, the optimized network is more stable; From the above, we can see that the point cloud feature extractor f θ It has two major modules: an alignment network is used to calibrate the direction of the point cloud, a multi-layer perceptron with shared weights maps the input point cloud to the same feature space with the same dimension, and a maximum pooling layer is used as a symmetric function to aggregate the information of all input points and extract the high-dimensional feature representation λ1 of the foam point cloud: Where, i = 1, 2, 3, 4, 5, i is the number of flotation cells, is the input single-slot foam point cloud, k=1,..., n is the number of input points, h(·) is the dimension-raising function, is the high-dimensional point after dimensionality upgrade, are affine transformation matrices with two different dimensions, which are obtained through T network training. f(·) is the maximum function, which is used to select the maximum value of each dimension.

5. The method for classifying a three-dimensional model of coal slime flotation foam based on Pointnet network according to claim 1, characterized in that: In step 4, use the feature projection head Project the high-dimensional features into the low-dimensional space to obtain the low-dimensional features λ2 of the single-slot foam point cloud: Where, i = 1, 2, 3, 4, 5, i is the number of flotation cells, We chose to use a two-layer perceptron MLP, which is a typical feedforward network based on back-propagation learning. It consists of an input layer, a hidden layer, and an output layer. Each layer is fully connected to the next layer. Except for the input node, each node is a neuron with a nonlinear activation function. The hidden layer realizes nonlinear mapping of the input space. The BP back-propagation algorithm is used to train the MLP.

6. The method for classifying a three-dimensional model of coal slime flotation foam based on Pointnet network according to claim 1, characterized in that: In step 5, the single slot feature λ2 is fused using a cascade operation, y f Select the CONCAT() concatenation function to obtain the overall characteristic γ of the flotation machine foam as follows: Among them, CONCAT() is a concatenation function. Through the above calculation, the overall foam characteristic γ of the flotation machine is obtained. γ includes the foam characteristics of each flotation cell. Compared with the foam data of a single flotation cell, the overall foam characteristic of the flotation machine contains richer information. This global feature can more accurately describe the overall foam state of the current flotation machine, so that the training model can learn more comprehensive knowledge, which is conducive to improving the accuracy of the model, increasing the robustness of the model, and enabling it to perform excellent performance under complex working conditions.

7. The method for classifying a three-dimensional model of coal slime flotation foam based on Pointnet network according to claim 1, characterized in that: In step 6, the multi-layer perceptron MLP and Softmax function are used to classify the overall feature y of the foam. Softmax converts the classification value into a probability distribution, and selects the highest probability label as the output result to achieve the classification of the foam point cloud model, and finally realize the recognition of different states of the foam: In the formula, z i It is the input of Softmax layer, with 3 types of flotation foam, P i is the output result.

8. The method for classifying a three-dimensional model of coal slime flotation foam based on Pointnet network according to claim 1, characterized in that: In step 7, a comparative experiment is used to evaluate the performance of the network proposed by the classification model. Network 1 is the result of a single-tank flotation foam identified using only the Pointnet network, Network 2 is the result of froth identification by fusing the froth features of two flotation tanks, Network 3 is the test result of fusing and classifying using the froth data of three flotation tanks as input, and Network 4 is the output result of this classification model. The confusion matrix is ​​plotted using the test set data to analyze the flotation foam recognition results of the four networks on the test set. The precision, recall, and F1 value indicators of the model in each category are calculated through the confusion matrix, thereby comprehensively evaluating the performance of the model in the flotation foam recognition task. In the multi-classification task, for the i-th category sample data, the number of samples correctly predicted by the model for the i-th category is defined as True Positive (TP), the number of samples incorrectly predicted by the model for the i-th category is defined as False Positive (FP), the number of samples correctly predicted by the model for non-i-th category is defined as True Negative (TN), and the number of samples incorrectly predicted by the model for non-i-th category is defined as False Negative (FN); Precision refers to the proportion of samples with actual labels of the i-th category among the samples predicted by the model to be of the i-th category. The calculation formula is: Recall rate (Reca1l) indicates the proportion of samples correctly predicted by the model as class i among all samples whose actual label is class i. A high recall rate means that the model can effectively capture samples whose actual label is class i. The calculation formula is: Generally, there is a trade-off between precision and recall. Improving precision will reduce recall, and vice versa. F1 value is the harmonic mean of precision and recall, which takes into account the precision and recall of the model. Under unbalanced category distribution, F1 value can more accurately reflect the performance of the model than accuracy. The calculation formula is: Calculate the precision, recall and F1 value of each category under different networks respectively, and calculate the average precision, recall and F1 value of each network; The receiver operating characteristic (ROC) curve is a tool for evaluating the performance of a classification model. The horizontal axis represents the false positive rate (FPR) and the vertical axis represents the true positive rate (TPR). The ROC curve can show the impact of different thresholds on the performance of the classifier and help select the optimal threshold. It is defined as:

Images