A flotation process condition recognition method based on hybrid learning of manual features and deep learning features

Abstract

The present invention relates to the technical field of flotation process condition identification, and discloses a flotation process condition identification method based on hybrid learning of manual features and deep learning features, which is composed of a manual feature extraction model, a deep convolution representation learning model and a one-dimensional convolution fusion model. The method inputs the hybrid feature vector extracted by the manual feature extraction model and the deep convolution representation learning model into the one-dimensional convolution fusion model for processing and classification, and further introduces a consistency loss auxiliary training model on the basis of classification cross entropy loss to enhance the accuracy and distinguishability of model feature extraction. The method takes into account the representation ability of manual features for domain knowledge and the ability of deep learning to comprehensively and efficiently capture information. It can not only reduce the influence of noise effects and training data limitations on deep learning features to a certain extent, but also effectively extract rich information in foam images, improve the classification performance of the network, and better adapt to the changes in different working conditions of the flotation process.

Description

Technical Field

[0001] The present invention relates to the technical field of flotation process working condition identification, and specifically to a flotation process working condition identification method based on hybrid learning of manual features and deep learning features. Background Art

[0002] Mineral resources are the material basis for the survival of human society and an important guarantee for national security and economic development. As an important mineral processing method, froth flotation has been widely used in the flotation of non-ferrous metal ores. Froth flotation is based on surface chemistry and uses the interaction between bubbles and mineral particles to separate useful minerals from gangue. After the raw ore is crushed, ball-milled and graded, water is added to mix it fully to form a slurry. Subsequently, specific types of flotation agents are added to the slurry. These flotation agents can adjust the surface properties of the mineral, improve the hydrophobicity of useful mineral particles, and promote the separation of gangue from ore. Then, the blended slurry is sent to the flotation tank. At the same time, the agitator drives the impeller to rotate and inhale air into the flotation tank to generate a large number of bubbles. These bubbles diffuse in the slurry and collide with mineral particles. The hydrophobic mineral particles will be adsorbed on the surface of the bubbles and float to the surface of the slurry with them to form a foam layer. Finally, the scraper scrapes the foam layer from the slurry and collects it centrally. The collected foam product is rich in a large number of hydrophobic useful minerals, while the hydrophilic gangue is deposited in the slurry and then discharged from the tank to form tailings. It can be seen that froth flotation is an effective means to separate hydrophobic minerals wetted by water by utilizing the gas-liquid interface of bubbles.

[0003] Research has shown that foam image features can not only reflect information about flotation conditions in real time, but are also key indicators of production indicators. The foam layer image formed on the slurry surface acquired by an industrial camera can effectively and quickly eliminate faulty conditions or adjust operating parameters based on current conditions. However, although experienced concentrator workers can infer the state of the flotation system from the visual features of the foam surface, they find it difficult to diagnose conditions related to the fine structure of the foam and are unable to consistently and reliably conduct a refined evaluation of flotation operating performance. Summary of the invention

[0004] 1. Technical issues to be solved

[0005] In view of the shortcomings of the prior art, the present invention provides a flotation process condition identification method based on hybrid learning of manual features and deep learning features, which has the advantages of accurate identification and solves the above-mentioned technical problems.

[0006] (II) Technical solution

[0007] To achieve the above object, the present invention provides the following technical solution: a method for identifying flotation process conditions based on hybrid learning of manual features and deep learning features, comprising the following steps:

[0008] S1. Data acquisition and preprocessing: obtain the video shot by the industrial camera above the flotation tank and normalize the video image;

[0009] S2. Feature extraction and fusion: Manual feature extraction and fusion of flotation foam image samples are performed on each sample. h and deep learning features F d The manual feature F h and deep learning features F d Fusion generates a mixed feature vector F, the manual feature F h Including foam color, bubble size, bubble carrying rate, bubble speed and bubble breakage rate, the deep learning feature F d Extraction through deep convolutional representation learning models;

[0010] S3. The fused mixed feature vector is used as the input of the 1-DCFN model for processing and classification.

[0011] S4. Train the 1-DCFN model: First, define the 1-DCFN model output value P i The actual corresponding label y of the sample i The cross entropy loss between F , Secondly, a two-layer fully connected classification module is connected in series based on the feature output of the deep convolutional representation learning model, and the classification output of the deep convolutional representation learning model is defined The actual corresponding label y of the sample i The cross entropy loss between D , used to guide the training of the representation model, and finally the predicted output of the 1-DCFN model is used as the soft target of the deep convolutional representation learning model, and the 1-DCFN model output value P is defined i Classification output of the learning model with deep convolutional representation The consistency loss function l between Consistency , and then the Adam algorithm is used to F , l D and l Consistency Make optimizations.

[0012] As a preferred technical solution of the present invention, the acquisition and preprocessing of data in step S1 specifically includes:

[0013] S1.1, each 1-minute video is cropped into multiple video clips, each video clip contains 10 frames of 600×800 images, and then the 600×800 images are downsampled to get 300×300 images;

[0014] S1.2, normalize the image in step S1.1, and finally obtain the processed flotation foam image, which is used to accelerate the convergence speed of the model and improve the stability of training. The specific calculation formula is as follows:

[0015]

[0016] Among them, output is the image pixel value output, input is the image pixel value input, max(input) and min(input) represent the minimum pixel value and maximum pixel value of the image respectively.

[0017] As a preferred technical solution of the present invention, the manual feature F in step S2 h and deep learning features F d The specific steps of extraction are as follows:

[0018] S2.1. Foam color feature extraction: The mean and standard deviation of the red component, green component, and blue component of the foam image RGB space are used as the color features of the foam image;

[0019] S2.2, bubble size feature extraction: using the highlight point extraction method based on illumination modeling, taking the highlight point as the foreground, its surrounding area as the background, and the highlight area at the top of the bubble as the marking point, the watershed algorithm is used to calculate the area generation of the bubble;

[0020] S2.3, bubble carrying rate feature extraction: the percentage of the bright spot area of ​​the bubble in the foam image to the total area of ​​the foam image is calculated by a formula;

[0021] S2.4, bubble velocity feature extraction: select N at the top of the first frame of foam image row The foam image block composed of rows is reference B reference ; Then extract a series of N consecutive row row-sized blocks, and finally by finding With B reference The maximum key point matching block in the image is used to calculate the foam velocity feature, and the velocity feature of multiple foam images is extracted to obtain the average foam velocity feature. Represents the image block starting from row i;

[0022] S2.5, bubble breakage rate feature extraction: After obtaining the foam image velocity information through step S2.4, the latter frame image in the image pair is converted to the same position as the previous frame image to obtain the difference image between the two. The bubble breakage rate is determined by calculating the ratio of the number of pixels in the difference image that exceeds a given threshold to the total number of pixels in the foam image;

[0023] S2.6. Deep convolutional representation learning model for deep learning features Fd Extraction: The deep learning feature extraction model uses the inception-V3 network to extract the deep learning feature F d Perform extraction;

[0024] S2.7, for manual features F h and deep learning features F d Fusion is performed to obtain the mixed feature vector F = [F h , F d ].

[0025] As a preferred technical solution of the present invention, the specific calculation process of step S2.2 is as follows:

[0026] S2.2.1. Highlight point extraction: The entire foam image is filtered by a small-kernel two-dimensional median filter, and the filtering result is I b , and then introduce a large kernel two-dimensional median filter to filter the entire foam image, and the filtering result is I d , combining the two filtering results, we get the bubble top I s The expression is as follows:

[0027] I s =I d -I b

[0028] Among them, I d Represents the filtering result of the large-kernel two-dimensional median filter, I b represents the filtering result of the small kernel two-dimensional median filter, I s Indicates the top mark point of the bubble;

[0029] S2.2.2. Distribution estimation: Select the log-normal distribution to model the bubble size distribution on the surface of the froth flotation tank, and then use the maximum likelihood method to estimate the log-normal distribution equation to obtain the optimal fit of the bubble size distribution, and use the standard deviation σ of the log-normal distribution equation to obtain the optimal fit of the bubble size distribution. 2 , the average value μ is used as a descriptor of the foam size distribution in the foam image sample.

[0030] As a preferred technical solution of the present invention, the bubble load rate feature extraction in step S2.3 is specifically calculated as follows:

[0031]

[0032] Where n represents the number of bubbles, L rate Indicates the bubble load rate, M i represents the area of ​​the i-th bubble bright spot in the image, and N represents the total area of ​​the foam image.

[0033] As a preferred technical solution of the present invention, the 1-DCFN model output value P in step S4 i The actual corresponding label y of the sample i The cross entropy loss between F , the specific expression is as follows:

[0034]

[0035] Among them, Q represents the number of training samples, C represents the number of categories, and l F Represents the 1-DCFN model output value P i The actual corresponding label y of the sample i The cross entropy loss between i (m) represents the probability of category m in the actual label of the sample, log represents the logarithmic function, P i (m) represents the probability that the output category of the 1-DCFN model is m, and They respectively represent the summation of Q and C internal data.

[0036] As a preferred technical solution of the present invention, the deep convolution representation learning model classification output in step S4 The actual corresponding label y of the sample i The cross entropy loss between D , the specific expression is as follows:

[0037]

[0038] Among them, Q represents the number of training samples, C represents the number of categories, and y i (m) represents the probability of category m in the actual label of the sample, log represents the logarithmic function, and Respectively represent the summation of the internal Q and C data, represents the probability of category m output by the deep convolutional representation learning model.

[0039] As a preferred technical solution of the present invention, the 1-DCFN model output value P in step S4 i Classification output of the learning model with deep convolutional representation The consistency loss function l between Consistency , the specific expression is as follows:

[0040]

[0041] Among them, l Consistency Represents the 1-DCFN model output value P i Classification output of the learning model with deep convolutional representation The consistency loss function between them, Q represents the number of training samples, C represents the number of categories, log represents the logarithmic function, and Respectively represent the summation of the internal Q and C data, represents the probability of category m output by the deep convolutional representation learning model, P i (m) represents the probability that the output category of the 1-DCFN model is m.

[0042] Compared with the prior art, the present invention provides a flotation process condition recognition method based on hybrid learning of manual features and deep learning features, which has the following beneficial effects:

[0043] 1. The present invention identifies working conditions by integrating manual features with deep learning features, taking into account the ability of manual features to characterize domain knowledge and the ability of deep learning to capture information comprehensively and efficiently. After making full use of the complementarity of these two features, the model can not only alleviate the impact of noise effects and training data limitations on deep learning features to a certain extent, improve the robustness and stability of working condition identification, but also effectively extract rich information in foam images to improve the classification performance of the network and better adapt to changes in different working conditions in the flotation process.

[0044] 2. The present invention can improve the ability to mine the correlation between features while reducing model parameters through the feature fusion and classification network based on 1-DCFN. Through the stacking or parallel combination of multiple one-dimensional convolutional layers, 1-DCFN can learn feature dependencies of different scales and abstraction levels, and fuse them together, so as to more comprehensively express the local patterns and correlations between manual features and deep learning features, and improve the performance and generalization of working condition classification. In addition, the introduction of batch normalization layers in the network helps to accelerate model convergence, enhance the robustness of the model to input data, and alleviate the problem of gradient disappearance. The application of BN layers in 1-DCFN can better standardize the input of the middle layer of the network, promote the model to learn more stable and reliable feature representations, further improve the performance and generalization of the model, enable it to more effectively process mixed feature vectors, and provide more powerful performance support for image classification tasks.

[0045] 3. The present invention uses the classification output of the fusion of manual features and deep learning features as the soft target of convolutional neural network representation learning, constructs consistency loss to guide the deep convolutional representation learning model to learn the rich semantic information implied in the manual features, and improves the accuracy and distinguishability of deep learning feature extraction. Specifically, on the one hand, the introduction of consistency loss helps to encourage the deep convolutional representation learning model to learn the rich domain knowledge contained in the manual features, so as to more comprehensively and effectively capture the essential characteristics of the foam image related to the mineral content, and effectively avoid the problem of gradient vanishing caused by too many convolution layers in the deep learning model; on the other hand, the introduction of consistency loss enables the deep convolutional representation learning model to better learn and utilize the semantic information implied in the manual features, and can more accurately extract image features, improve the distinguishability between features, and then improve the performance of feature extraction of the deep convolutional representation learning model, and finally improve the accuracy of the classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the process of the present invention;

[0047] Figure 2 This is a schematic diagram of the 1-DCFN structure of the present invention;

[0048] Figure 3 This is a schematic diagram of the inception-V3 network architecture of the present invention;

[0049] Figure 4 This is a schematic diagram of the 1-DCFN network architecture of the present invention;

[0050] Figure 5 This is a schematic diagram of the principle of the froth flotation process of the present invention;

[0051] Figure 6 A schematic diagram of a highlight point extraction method based on illumination modeling of the present invention;

[0052] Figure 7 A schematic diagram of speed feature extraction based on SURF key point matching and macroblock tracking in the present invention; DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] See also Figure 1-7 , a flotation process condition recognition method based on hybrid learning of manual features and deep learning features, comprising the following steps:

[0055] S1. Data acquisition and preprocessing: Install a light source and an industrial camera above the flotation tank, and take certain measures to protect the camera to ensure the stability of the on-site lighting and to protect the camera lens from external influences as much as possible, so as to obtain foam images with less noise interference. The foam video is transmitted to the industrial computer via optical fiber, and the frame rate is set to 30fps to achieve continuous high-frequency shooting of the foam. Each video is about 1 minute long, and the size of the video frame is maintained at 600×800. During this period, the obtained foam images are labeled as 6 corresponding working conditions. Subsequently, the size of the input image is reduced by cropping and downsampling operations, and the image pixel values ​​are scaled to the specified range through Min-Max normalization processing to ensure data consistency. Finally, the data is divided into a training set (60%), a validation set (20%), and a test set (20%) to establish and evaluate the hybrid learning model.

[0056] Furthermore, the acquisition and preprocessing of data in step S1 specifically includes:

[0057] S1.1. Directly inputting multiple 1-minute videos or 600×800 images will cause the model to overflow memory. Therefore, the following strategy is adopted: First, each 1-minute video is cropped into multiple video clips, each video clip contains 10 frames of 600×800 images, and then the 600×800 images are downsampled to 300×300 images;

[0058] S1.2, normalize the image in step S1.1, and finally obtain the processed flotation foam image, which is used to accelerate the convergence speed of the model and improve the stability of training. The specific calculation formula is as follows:

[0059]

[0060] Among them, output is the image pixel value output, input is the image pixel value input, max(input) and min(input) represent the minimum pixel value and maximum pixel value of the image respectively.

[0061] S2. Feature extraction and fusion: First, the feature vector of the foam image is extracted using the left and right branches. The manual feature extraction model of the left branch extracts manual features from the input flotation foam layer image samples to extract rich semantic information contained in the image. The extracted manual features include five manual features: foam color, bubble size, bubble carrying rate, bubble speed and bubble breakage rate. The right branch deep learning model uses the inception-V3 structure of GooLeNet to extract higher-level deep learning features from the foam image that cannot be obtained by the manual feature extraction model. Then, the feature vectors extracted by the two models are fused together to obtain a complex mixed feature vector with rich and comprehensive feature information. Finally, the mixed feature vector is used as the input of 1-DCFN for end-to-end learning, training and classification.

[0062] Specifically, the flotation foam image samples are manually characterized by F h and deep learning features F d Extract and transform the manual features F h and deep learning features F d Fusion generates mixed feature vector F, manual feature F h Including foam color, bubble size, bubble carrying rate, bubble speed and bubble breakage rate, deep learning feature F d Extraction through deep convolutional representation learning models;

[0063] Furthermore, the manual feature F in step S2 h and deep learning features F d The specific steps of extraction are as follows:

[0064] S2.1. Foam color feature extraction: The mean and standard deviation of the red component, green component, and blue component of the foam image RGB space are used as the color features of the foam image;

[0065] S2.2, bubble size feature extraction: using the highlight point extraction method based on illumination modeling, taking the highlight point as the foreground, its surrounding area as the background, and the highlight area at the top of the bubble as the marking point, the watershed algorithm is used to calculate the area generation of the bubble;

[0066] Furthermore, the specific calculation process of step S2.2 is as follows:

[0067] S2.2.1. Highlight point extraction: Since many tiny bubbles (noise) will be attached to normal flotation bubbles, a small-core two-dimensional median filter (such as a 10×10 filter kernel) is introduced to filter the entire foam image to eliminate the influence of noise. The filtering result is expressed as I bIt can be found that after filtering, the tiny bubbles in the foam image are removed and the valley between two connected bubbles becomes darker. In order to estimate the illumination distribution in the foam image, a large-core two-dimensional median filter (such as a 40×40 filter kernel) is introduced to filter the entire foam image. The filtering result is expressed as I d After combining two filters of different scales, the highlight points that are larger than tiny bubbles but smaller than normal flotation bubbles are retained. That is, the result after two median filters retains the main information of the highlight points and flotation bubbles and removes the noise information such as tiny bubbles. The filter kernel size in the illumination modeling is estimated based on the average bubble size. The top of the bubble I s It can be expressed as:

[0068] I s =I d -I b

[0069] Among them, I d Represents the filtering result of the large-kernel two-dimensional median filter, I b represents the filtering result of the small kernel two-dimensional median filter, I s Indicates the top mark point of the bubble;

[0070] S2.2.2. Distribution estimation: Select the log-normal distribution to model the bubble size distribution on the surface of the froth flotation tank, and then use the maximum likelihood method to estimate the log-normal distribution equation to obtain the optimal fit of the bubble size distribution, and use the standard deviation σ of the log-normal distribution equation to obtain the optimal fit of the bubble size distribution. 2 , the average value μ is used as a descriptor of the foam size distribution in the foam image sample.

[0071] In view of the fact that the bubble size distribution on the surface of the foam layer presents the characteristics of lognormal distribution, the lognormal distribution is selected to model the bubble size distribution on the surface of the foam flotation tank. The lognormal distribution is a continuous probability distribution, which is applicable to random variables with normal logarithm distribution. Its probability density function is defined as follows:

[0072]

[0073] where σ 2 is the standard deviation and μ is the mean.

[0074] The log-normal distribution equation is estimated by the maximum likelihood method to obtain the optimal fit of the bubble size distribution. Define [x1...x n ] is a set of independent observations with log-normal distribution, then the likelihood function can be defined as:

[0075]

[0076] Further converted to:

[0077]

[0078] To estimate the lognormal distribution. 2 The likelihood equation for and μ can be written as:

[0079]

[0080]

[0081] The solution of the above formula is:

[0082]

[0083]

[0084] S2.3. Extraction of bubble load ratio features: The percentage of the bright spot area of ​​the bubbles in the foam image to the total area of ​​the foam image is calculated by the calculation formula. The specific calculation formula is as follows:

[0085]

[0086] Where n represents the number of bubbles, L rate Indicates the bubble load rate, M i represents the area of ​​the i-th bubble bright spot in the image, and N represents the total area of ​​the foam image.

[0087] S2.4, bubble velocity feature extraction: using the velocity feature extraction method based on SURF key point matching and macroblock tracking, select N at the top of the first frame of the bubble image. row The foam image block composed of rows is reference B reference ; Then extract a series of N consecutive row row-sized blocks, and finally by finding With B reference The maximum key point matching block in the image is used to calculate the foam velocity feature, and the velocity feature of multiple foam images is extracted to obtain the average foam velocity feature. Represents the image block starting from row i;

[0088] S2.5, bubble breakage rate feature extraction: After obtaining the foam image velocity information through step S2.4, the latter frame image in the image pair is converted to the same position as the previous frame image to obtain the difference image between the two. The bubble breakage rate is determined by calculating the ratio of the number of pixels in the difference image that exceeds a given threshold to the total number of pixels in the foam image;

[0089] S2.6. Deep convolutional representation learning model for deep learning features F dExtraction: The deep learning feature extraction model uses the inception-V3 network. A 2048-component image feature vector containing the complex image features of the input foam image sample is obtained by extracting from the pooling layer 3 of the model;

[0090] S2.7, for manual features F h and deep learning features F d Fusion is performed to obtain the mixed feature vector F = [F h , F d ].

[0091] The feature vector F is used as the input of 1-DCFN, and the image is classified and estimated through network learning. 1-DCFN is a custom lightweight convolutional neural network that improves the model recognition performance by stacking convolutional layers, and accelerates the training process by adding BN layers, improving the convergence speed of the model, and helping to alleviate the problems of gradient disappearance and explosion. The final classification result of the model is output through the category probability P i To determine, the output probability is obtained through the Softmax layer activation function.

[0092] S3. The fused mixed feature vector is used as the input of the 1-DCFN model for processing and classification.

[0093] S4. Train the 1-DCFN model: First, define the 1-DCFN model output value P i The actual corresponding label y of the sample i The cross entropy loss between F , Secondly, a two-layer fully connected classification module is connected in series based on the feature output of the deep convolutional representation learning model, and the classification output of the deep convolutional representation learning model is defined The actual corresponding label y of the sample i The cross entropy loss between D , used to guide the training of the representation model, and finally the predicted output of the 1-DCFN model is used as the soft target of the deep convolutional representation learning model, and the 1-DCFN model output value P is defined i Classification output of the learning model with deep convolutional representation The consistency loss function l between Consistency , and then the Adam algorithm is used to F , l D and l Consistency Make optimizations.

[0094] Furthermore, in step S4, the 1-DCFN model output value P i The actual corresponding label y of the sample i The cross entropy loss between F , the specific expression is as follows:

[0095]

[0096] Among them, Q represents the number of training samples, C represents the number of categories, and l F Represents the 1-DCFN model output value P i The actual corresponding label y of the sample i The cross entropy loss between i (m) represents the probability of category m in the actual label of the sample, log represents the logarithmic function, P i (m) represents the probability that the output category of the 1-DCFN model is m, and They respectively represent the summation of Q and C internal data.

[0097] Furthermore, the deep convolution in step S4 represents the classification output of the learning model The actual corresponding label y of the sample i The cross entropy loss between D , the specific expression is as follows:

[0098]

[0099] Among them, Q represents the number of training samples, C represents the number of categories, and y i (m) represents the probability of category m in the actual label of the sample, log represents the logarithmic function, and Respectively represent the summation of the internal Q and C data, represents the probability of category m output by the deep convolutional representation learning model.

[0100] Furthermore, in step S4, the 1-DCFN model output value P i Classification output of the learning model with deep convolutional representation The consistency loss function l between Consistency , the specific expression is as follows:

[0101]

[0102] Among them, l Consistency Represents the 1-DCFN model output value P i Classification output of the learning model with deep convolutional representation The consistency loss function between them, Q represents the number of training samples, C represents the number of categories, log represents the logarithmic function, and Respectively represent the summation of the internal Q and C data, represents the probability of category m output by the deep convolutional representation learning model, P i(m) represents the probability that the output category of the 1-DCFN model is m.

[0103] The flotation foam image sample is input into the feature mixture learning model. First, a 2048-component image feature vector F is extracted from the pooling layer 3 of the deep convolutional representation learning model. d With the hand-crafted feature F h The mixed feature vector F is obtained by fusion and used as the input of the 1-DCFN model to construct the model classification output P i and the true label y of the image i The cross entropy loss between F , collaboratively optimize the deep convolutional representation learning model and the 1-DCFN model; calculate the classification output of the deep convolutional representation learning model and the true label y of the image i The cross entropy loss between D , minimize the difference between the model classification output and the actual label, to prevent F The gradient disappears, which makes the deep convolution representation learning model difficult to learn; finally, the output of the deep convolution representation learning model defined in the minimization formula With 1-DCFN model output P i The consistency loss function l Consistency , using 1-DCFN to reversely guide the deep convolution representation learning model parameter update. During the entire training process, the Adam algorithm is used to simultaneously optimize the formula

[0104] l=α1l F +α2l D +α3l Consistency

[0105] Among them, α1, α2 and α3 are three weight parameters, α1+α2+α3=1. Once the accuracy of the validation set reaches saturation, the learning rate is reduced by 10 times to improve the convergence effect of the model.

[0106] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A flotation process condition recognition method based on hybrid learning of manual features and deep learning features, characterized by: The following steps are involved: S1. Data acquisition and preprocessing: obtain the video shot by the industrial camera above the flotation tank and normalize the video image; S2, feature extraction and fusion: including manual feature extraction of flotation foam image samples h and deep learning features F d The manual feature F h and deep learning features F d Fusion generates a mixed feature vector F, the manual feature F h Including foam color, bubble size, bubble carrying rate, bubble speed and bubble breakage rate, the deep learning feature F d Extraction through deep convolutional representation learning models; The manual feature F in step S2 h and deep learning features F d The specific steps of extraction are as follows: S2.

1. Foam color feature extraction: The mean and standard deviation of the red component, green component, and blue component of the foam image RGB space are used as the color features of the foam image; S2.2, bubble size feature extraction: using the highlight point extraction method based on illumination modeling, taking the highlight point as the foreground, its surrounding area as the background, and the highlight area at the top of the bubble as the marking point, the watershed algorithm is used to calculate the area generation of the bubble; S2.3, bubble carrying rate feature extraction: the percentage of the bright spot area of ​​the bubble in the foam image to the total area of ​​the foam image is calculated by a formula; S2.4, bubble velocity feature extraction: select N at the top of the first frame of foam image row The foam image block composed of rows is reference B reference ; Then extract a series of N consecutive row row-sized blocks, and finally by finding With B reference The maximum key point matching block in the image is used to calculate the foam velocity feature, and the velocity feature of multiple foam images is extracted to obtain the average foam velocity feature. Represents the image block starting from row i; S2.5, bubble breakage rate feature extraction: after obtaining the foam image velocity information through step S2.4, convert the latter frame image in the image pair to the same position as the previous frame image to obtain the difference image between the two; calculate the ratio of the number of pixels exceeding a given threshold in the difference image to the total pixels of the foam image to determine the bubble breakage rate; S2.

6. Deep convolutional representation learning model for deep learning features F d Extraction: The deep learning feature extraction model uses the inception-V3 network to extract the deep learning feature F d Perform extraction; S2.7, for manual features F h and deep learning features F d Fusion is performed to obtain the mixed feature vector F = [F h ,F d ]; S3, using the fused mixed feature vector as the input of the 1-DCFN model for processing and classification; S4. Train the 1-DCFN model: First, define the 1-DCFN model output value P i The actual corresponding label y of the sample i The cross entropy loss between F , Secondly, a two-layer fully connected classification module is connected in series based on the feature output of the deep convolutional representation learning model, and the classification output of the deep convolutional representation learning model is defined The actual corresponding label y of the sample i The cross entropy loss between D , used to guide the training of the representation model, and finally the predicted output of the 1-DCFN model is used as the soft target of the deep convolutional representation learning model, and the 1-DCFN model output value P is defined i Classification output of the learning model with deep convolutional representation The consistency loss function l between Consistency , and then the Adam algorithm is used to F , l D and l Consistency Make optimizations.

2. According to claim 1, a flotation process condition identification method based on hybrid learning of manual features and deep learning features is characterized by: The acquisition and preprocessing of data in step S1 specifically include: S1.1, each 1-minute video is cropped into multiple video clips, each video clip contains 10 frames of 600×800 images, and then the 600×800 images are downsampled to get 300×300 images; S1.2, normalize the image in step S1.1, and finally obtain the processed flotation foam image, which is used to accelerate the convergence speed of the model and improve the stability of training. The specific calculation formula is as follows: Among them, output is the image pixel value output, input is the image pixel value input, max(input) and min(input) represent the minimum pixel value and maximum pixel value of the image respectively.

3. The method for identifying flotation process conditions based on hybrid learning of manual features and deep learning features according to claim 1 is characterized in that: The specific calculation process of step S2.2 is as follows: S2.2.

1. Highlight point extraction: The entire foam image is filtered by a small-core two-dimensional median filter, and the filtering result is I b , and then introduce a large kernel two-dimensional median filter to filter the entire foam image, and the filtering result is I d , combining the two filtering results, we get the bubble top I s The expression is as follows: I s =I d -I b Among them, I d Represents the filtering result of the large-kernel two-dimensional median filter, I b represents the filtering result of the small kernel two-dimensional median filter, I s Indicates the top mark point of the bubble; S2.2.

2. Distribution estimation: Select the log-normal distribution to model the bubble size distribution on the surface of the froth flotation tank, and then use the maximum likelihood method to estimate the log-normal distribution equation to obtain the optimal fit of the bubble size distribution, and use the standard deviation σ of the log-normal distribution equation to obtain the optimal fit of the bubble size distribution. 2 , the average value μ is used as a descriptor of the foam size distribution in the foam image sample.

4. The method for identifying flotation process conditions based on hybrid learning of manual features and deep learning features according to claim 1 is characterized in that: The specific calculation formula for the bubble load rate feature extraction in step S2.3 is as follows: Where n represents the number of bubbles, L rate Indicates the bubble load rate, M i represents the area of ​​the i-th bubble bright spot in the image, and N represents the total area of ​​the foam image.

5. The method for identifying flotation process conditions based on hybrid learning of manual features and deep learning features according to claim 1 is characterized in that: In step S4, the 1-DCFN model outputs the value P i The actual corresponding label y of the sample i The cross entropy loss between F , the specific expression is as follows: Among them, Q represents the number of training samples, C represents the number of categories, and l F Represents the 1-DCFN model output value P i The actual corresponding label y of the sample i The cross entropy loss between i (m) represents the probability of category m in the actual label of the sample, log represents the logarithmic function, P i (m) represents the probability that the output category of the 1-DCFN model is m, and They respectively represent the summation of Q and C internal data.

6. The method for identifying flotation process conditions based on hybrid learning of manual features and deep learning features according to claim 5 is characterized in that: The deep convolution in step S4 represents the classification output of the learning model The actual corresponding label y of the sample i The cross entropy loss between D , the specific expression is as follows: Among them, Q represents the number of training samples, C represents the number of categories, and y i (m) represents the probability of category m in the actual label of the sample, log represents the logarithmic function, and Respectively represent the summation of the internal Q and C data, represents the probability of category m output by the deep convolutional representation learning model.

7. The method for identifying flotation process conditions based on hybrid learning of manual features and deep learning features according to claim 6 is characterized in that: In step S4, the 1-DCFN model outputs the value P i Classification output of the learning model with deep convolutional representation The consistency loss function l between Consistency , the specific expression is as follows: Among them, l Consistency Represents the 1-DCFN model output value P i Classification output of the learning model with deep convolutional representation The consistency loss function between them, Q represents the number of training samples, C represents the number of categories, log represents the logarithmic function, and Respectively represent the summation of the internal Q and C data, represents the probability of category m output by the deep convolutional representation learning model, P i (m) represents the probability that the output category of the 1-DCFN model is m.

Images