Lightweight feature extraction model and flotation process running state evaluation method

Abstract

The application discloses a lightweight feature extraction model and a flotation process running state evaluation method. The model comprises: a first teacher module configured to generate a first roughing image feature and a first scavenging image feature according to the foam image; a second teacher module configured to generate a first data feature according to the foam data; a student module configured to generate a second roughing image feature according to the first roughing image feature, generate a second scavenging image feature according to the first scavenging image feature, and generate a second data feature according to the first data feature; and a fusion module configured to fuse the second roughing image feature, the second scavenging image feature and the second data feature to generate a first fusion feature, and generate a state grade label according to the first fusion feature; and determine the state grade of the current flotation process running state, so as to provide a more lightweight model deployment and a more competitive application in an actual industrial scene.

Description

Technical Field

[0001] This invention relates to the field of flotation process operation status evaluation, and in particular to a lightweight feature extraction model and a method for evaluating the operation status of flotation processes. Background Technology

[0002] Flotation, as a primary method for mineral separation, has been widely applied in industries such as metallurgy, coal, and chemicals. To ensure mineral resource recovery rates, reduce production costs, and improve overall economic efficiency, timely operational optimization of the production process is necessary to ensure efficient operation under optimal conditions. However, due to operational lags and external environmental interference, the process operation often deviates from the initially set ideal level, affecting the overall economic benefits of the enterprise. Therefore, evaluating the operational status of the flotation process has significant theoretical and practical value.

[0003] In recent years, research on the evaluation of the operational status of industrial production processes has received widespread attention from both academia and industry. The purpose of process monitoring is to distinguish whether a process is operating normally. However, process operational status evaluation, based on the premise that the process is operating normally, further differentiates the quality of the actual production operation, subdividing the normal operational status into multiple levels such as excellent, good, average, and poor. This allows managers and operators to provide a reliable basis for ensuring the production process operates as well as possible and for subsequent operational adjustments.

[0004] During flotation, real-time images of the flotation drum and related process data, such as slurry level, aeration rate, foam size, and saturation, are collected. The images and data complement each other, ensuring the completeness of the information needed for operational status evaluation, both of which are crucial for assessing the quality of the flotation process. Operational status evaluation can be viewed as a pattern classification task. Given the characteristics of the flotation process, this task faces the following challenges: First, during flotation, the amount of raw ore fed and the particle size distribution fluctuate continuously with the production process. Simultaneously, the screening effect and hydrocyclone feed conditions change constantly due to external disturbances such as mechanical vibration, causing frequent fluctuations in the flotation foam characteristics and affecting operator judgment. Second, flotation is a long-term process, and the characteristic differences between its various state levels are relatively small, making the extraction of essential characteristics from different state levels difficult. Third, with the continuous improvement of information acquisition technology, multiple signal sensors are used in the flotation process. The existence of multi-source information generates a large amount of real-time data during the process. This real-time data occupies a significant amount of hardware storage space. Therefore, the storage and computing resources that can be allocated to the evaluation model are very limited, making it difficult to use complex state evaluation models to evaluate the flotation process in real time. Among the three challenges mentioned above, image quality and feature recognition issues severely limit the accuracy of the flotation process's operational state evaluation. Limited storage and computing resources place extremely high demands on the lightweight nature of evaluation models based on image and data information. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides an adaptive queue network service quality management method and system, which solves the technical problem that the storage and computing resources that can be allocated to the evaluation model are very limited, making it difficult to use complex state evaluation models to evaluate the flotation process in real time.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, a first aspect of the present invention provides a lightweight feature extraction model.

[0009] A second aspect of the present invention provides a method for evaluating the operating status of a flotation process based on lightweight heterogeneous information.

[0010] In view of this, a lightweight feature extraction model is proposed according to a first aspect of the embodiments of this application, the model comprising: a first teacher module, a second teacher module, a student module, and a fusion module;

[0011] The first teacher module is configured to: acquire a bubble image, and generate a first coarse-selection image feature and a first scan image feature based on the bubble image;

[0012] The second teacher module is configured to: acquire bubble data and generate a first data feature based on the bubble data;

[0013] The student module is configured to: generate a second coarse-selected image feature based on the first coarse-selected image feature; generate a second scanned image feature based on the first scanned image feature; and generate a second data feature based on the first data feature.

[0014] The fusion module is configured to: fuse the second coarse selection image features, the second scan image features, and the second data features to generate a first fusion feature, and generate a status level label based on the first fusion feature; and determine the status level of the current flotation process based on the status level label.

[0015] According to a second aspect of the embodiments of this application, a method for evaluating the operating status of a flotation process is provided, the method comprising:

[0016] Acquire foam images and obtain foam data based on the foam images;

[0017] The bubble image is input into the first teacher module to generate the first coarse-selection image features and the first scanned image features;

[0018] The foam data is input into the second teacher module to generate the first data feature;

[0019] Generate second coarse image features based on the first coarse image features;

[0020] Generate second scanned image features based on the first scanned image features;

[0021] Generate a second data feature based on the first data feature;

[0022] The second coarse-selected image features, the second scanned image features, and the second data features are input into the fusion module for fusion to generate the first fused feature;

[0023] A status level label is generated based on the first fusion feature;

[0024] The status level of the current flotation process is determined based on the status level label.

[0025] In one implementation, the process of acquiring a foam image and obtaining foam data from the foam image further includes:

[0026] The initial foam images and initial foam data acquired online are normalized respectively; the initial foam images are initial foam images inside the flotation tube acquired in real time by an industrial camera; the initial foam data are data information on the foam state extracted using a foam image analyzer; the data information on the foam state includes foam stability, flow rate, color, and the number of large, medium, and small bubbles.

[0027] In one implementation, the step of generating a state level label based on the first fusion feature includes:

[0028] The first fusion feature extracted from the nth sample is denoted as

[0029] Where P is the number of foam images in each training sample, and J is... I Let n be the feature dimension of the bubble image, where n = 1, 2, ..., N;

[0030] The actual state level label of the first fused feature is obtained using one-hot encoding.

[0031] In one implementation, the actual state level label y of the image features is obtained using one-hot encoding. n Use the following formula:

[0032]

[0033] Where, θ R ={θ R，1 θ R，2 , …, θ R，c} represents the parameters of the SoftMax classifier. This represents the posterior probability that the nth sample belongs to the cth state level.

[0034] In one implementation, the step of inputting the preprocessed foam image into the first teacher module to generate the first coarse-selected image features and the first scanned image features further includes:

[0035] The loss function for the first teacher module is:

[0036]

[0037] Among them, y n For status level labels.

[0038] In one implementation, the step of inputting the second coarse-selected image features, the second scanned image features, and the second data features into a fusion module for fusion and generating a first fused feature includes:

[0039] The second coarse-selected image features and the second scanned image features are fused using STE to generate coarse-selected image fusion features and scanned image fusion features;

[0040] Image fusion features are constructed based on the coarse-selected image fusion features and the scanned image fusion features; the second data features are then self-fused to generate a third data feature.

[0041] The image fusion feature and the second data feature are fused together using CTE to generate the first fusion feature.

[0042] In one implementation, the step of generating a state level label based on the first fusion feature includes:

[0043] The first fusion feature is input into the feedforward layer to generate the second fusion feature;

[0044] The second fused feature is input into the SoftMax classifier to generate the state level label.

[0045] In one implementation, the step of inputting the first fused feature into the feedforward layer and generating the second fused feature further includes:

[0046] The feedforward layer expression is:

[0047]

[0048] in, This is the weight matrix. This is the deviation vector.

[0049] In one implementation, the step of determining the state level of the current flotation process operation state based on the state level label further includes:

[0050] The status level is the index of the largest element in the status level label.

[0051] (III) Beneficial Effects

[0052] The beneficial effects of this invention are:

[0053] This invention provides a lightweight feature extraction model and a method for evaluating the operational status of the flotation process. Utilizing deep learning, it accurately and objectively extracts deep features from flotation foam images and data with temporal information, and adaptively fuses multi-source heterogeneous information, ensuring accurate assessment of the flotation process's operational status. This allows for accurate description of the flotation process's operational status by actual production operators and managers, overcoming the subjectivity and arbitrariness of manual observation and ensuring enterprise production efficiency and economic benefits. Furthermore, it offers a more lightweight model deployment, making it more competitive in real-world industrial applications. It provides data support for the flotation process, offering operational guidance and enabling optimized control of the flotation process. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0055] Figure 1 A block diagram of a lightweight feature extraction model and a flotation process operation status evaluation method provided in this application;

[0056] Figure 2 A flowchart of a lightweight feature extraction model and a flotation process operation status evaluation method provided in this application;

[0057] Figure 3 A schematic diagram of a flotation process for a lightweight feature extraction model and a flotation process operation status evaluation method provided in this application;

[0058] Figure 4 The internal structure diagram of the LSTM for a lightweight feature extraction model and a flotation process operation status evaluation method provided in this application;

[0059] Figure 5 This is a schematic diagram of knowledge distillation learning for a lightweight feature extraction model and a flotation process operation status evaluation method provided in this application. Detailed Implementation

[0060] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0061] This application addresses the problem in existing technologies where limited storage and computing resources allocated to evaluation models hinder real-time evaluation of the flotation process using complex state evaluation models. Based on these reasons, this application provides a lightweight feature extraction model and a method for evaluating the operational state of the flotation process.

[0062] The method and system of the present invention will be further described below with reference to specific embodiments.

[0063] Firstly, such as Figure 1 As shown, this application provides a lightweight feature extraction model, which includes: a first teacher module 100, a second teacher module 200, a student module 300, and a fusion module 400;

[0064] The first teacher module 100 is configured to: acquire a bubble image, and generate a first coarse selection image feature and a first scan image feature based on the bubble image;

[0065] The second teacher module 200 is configured to: acquire bubble data and generate a first data feature based on the bubble data;

[0066] The student module 300 is configured to: generate a second coarse-selected image feature based on the first coarse-selected image feature; generate a second scanned image feature based on the first scanned image feature; and generate a second data feature based on the first data feature.

[0067] The fusion module 400 is configured to: fuse the second coarse selection image features, the second scan image features, and the second data features to generate a first fusion feature, and generate a status level label based on the first fusion feature; and determine the status level of the current flotation process operation based on the status level label.

[0068] This application provides a lightweight feature extraction model, the C3D-Bi-LSTM-Transformer model. This model can adaptively fuse multi-source heterogeneous information, ensuring accurate assessment of the flotation process's operational status. It accurately describes the flotation process's operational status for actual production operators and managers, overcoming the subjectivity and arbitrariness of manual observation and ensuring enterprise production efficiency and economic benefits. Furthermore, it offers a more lightweight model deployment, making it more competitive in real-world industrial applications. It provides data support for the flotation process, offering operational guidance and enabling optimized control of the flotation process. Additionally, based on this model, this application also discloses a method for evaluating the operational status of the flotation process.

[0069] Secondly, such as Figure 2 As shown, this application provides a method for evaluating the operating status of a flotation process, the method comprising:

[0070] S100, acquire a foam image and obtain foam data based on the foam image;

[0071] In step S100, before acquiring the foam image and obtaining foam data based on the foam image, the method further includes: normalizing the initial foam image and initial foam data acquired online; the initial foam image is an initial foam image inside the flotation tube acquired in real time by an industrial camera; the initial foam data is data information on the foam state extracted using a foam image analyzer; the data information on the foam state includes foam stability and flow rate.

[0072] In practical applications, the system hardware of this application mainly consists of a camera, a foam analyzer, sensors, a computer, and other auxiliary components. The camera is primarily used to acquire flotation foam images and requires an independent light source to improve image contrast. The foam analyzer provides shallow features such as foam size and saturation, as well as dynamic features that cannot be provided by a single image, such as foam stability and flow rate. The sensors mainly acquire variables such as slurry level and aeration rate. The simulation experimental environment for evaluating the flotation process operation status is as follows: software platform: PyCharm 2020.1.3; deep learning framework: Tensorflow 2.4.0; hardware platform: i7-10875H CPU and NVIDIA GeForce RTX 3060 Laptop GPU, RAM: 32GB, operating system: Windows 10.

[0073] During the flotation process, the sampling frequencies for various data points differ. X-ray fluorescence analyzer collects data on copper concentrate and tailings grades at a recording frequency of approximately 16 minutes. A froth analyzer collects initial froth images at a recording frequency of approximately a few seconds, and based on the continuous image information, obtains short-term dynamic information such as froth velocity and stability, as well as short-term static information such as froth size and color. Pulp level and aeration rate are recorded at a frequency of approximately a few seconds. To balance long-term and short-term dynamic information from flotation and ensure consistency in the data collection cycle, we use the sampling cycle of concentrate and tailings as the standard, and sample 16 initial froth images and data points at equal intervals within the cycle.

[0074] Scalar data for the flotation process includes flotation froth state variables and process operation variables. Specific variable measurement points are as follows: Figure 3 As shown:

[0075] Foam measurement is crucial in the coarse, fine, and sweep stages of selection, providing information on foam flow rate, the number of foams per square meter (large, medium, and small), foam area, foam color, and foam stability. The x-axis velocity and y-axis velocity (in millimeters per second) are calculated by the position vector difference between two adjacent foams in two consecutive images. Foam area is the number of pixels occupied by the foam in the image, converted to square centimeters. Foam color is categorized by hue, saturation, and brightness. Foam stability is calculated by the ratio of the number of foams in two consecutive images.

[0076] The quality of the selection process can be reflected to some extent in the coarse selection and scanning processes. Therefore, in order to ensure the lightweight nature of the model, only the image features selected in the coarse selection and scanning processes are used in the image feature extraction process.

[0077] The table below only lists the state variables and flotation operation variables related to coarse flotation. The state variables for fine flotation and scavenging flotation are the same as those for coarse flotation and are not listed.

[0078] Table 1 shows the variables in the flotation process:

[0079]

[0080]

[0081] Based on comprehensive economic indicators, the flotation process was divided into four levels: poor, medium, good, and excellent. Finally, 21,276 offline data points and 5,220 online test data points for a specific stable mode were obtained from the historical production database. 80% of the offline data was used as the training set, and 20% as the validation set for model testing.

[0082] S200, the bubble image is input to the first teacher module 100 to generate the first coarse selection image features and the first scan image features;

[0083] In step S200, the step of inputting the preprocessed bubble image into the first teacher module 100 to generate the first coarse-selected image features and the first scanned image features further includes: the loss function of the first teacher module 100 is:

[0084]

[0085] Among them, y n For status level labels.

[0086] In practical applications, the preprocessed bubble image is input into the first teacher module 100, which is a C3D teacher module. After the bubble image is input into the C3D teacher module, the first coarse-selected image features and the first scanned image features are extracted. Since feature extraction in convolutional neural networks mainly uses convolutional layers, activation layers, and pooling layers, in the convolutional layers of a 2D convolutional neural network, the features obtained from the previous layer are convolved in two dimensions of the spatial plane using convolution kernels. The value of a unit at position (x, y) in the j-th feature map of the i-th layer is denoted as... And it is obtained by calculation using equation (2);

[0087]

[0088] Where k is the number of feature maps in the (i-1)th layer, Pi and Q i These represent the size of the spatial dimension of the i-th convolutional kernel. Let b be the value of the k-th convolution kernel on (p, q). ij It represents the bias, and f represents the activation function.

[0089] For 3D convolution, the value of a unit at position (x, y, z) in the j-th feature map of the i-th layer is denoted as... And it is obtained by calculation using equation (3);

[0090]

[0091] Among them, R i Let be the size of the time dimension of the i-th convolutional kernel.

[0092] By adding a time dimension during the 3D convolution process, feature extraction not only considers the spatial features of the foam, but also combines information from multiple sampling times to mine features, which can effectively reduce the impact of external interference in the flotation process on the foam features.

[0093] The main function of the pooling layer is downsampling, which divides the features of the previous layer into several sub-regions and takes the average or maximum value of each sub-region, thereby reducing the number of parameters and the computational load of the network, and also controlling overfitting to some extent.

[0094] S300, the foam data is input into the second teacher module 200 to generate the first data feature;

[0095] In step S300, the preprocessed foam data is input into the second teacher module 200, which is a Bi-LSTM module. After the foam data is input into the Bi-LSTM module, the first data features can be extracted. Since LSTM is generally trained using the features output from the previous time step and the features from the current time step as input, LSTM is suitable for tasks involving time series. Its internal structure is as follows: Figure 4 As shown. An LSTM contains three gating units: a forget gate, an input gate, and an output gate. The forget gate and input gate can be controlled by changing the long-term state unit c. t The information can be selectively retained by the forget gate. t-1 The information from the input gate determines the current state unit c. t What new information is added? The output gate is used to control the long-term cell state c. t The information is output to the current output value h. t The activation vector f of the forget gate at time t in layer l. t I Input gate activation vector and output gate activation vector The expressions are shown in equations (4), (5), and (6):

[0096]

[0097]

[0098]

[0099] Where σ is a non-linear activation function, usually the sigmoid function. This represents the input of layer 1 at time t-1. This represents the input of layer l-1 at time t. These are the weight matrices for the forget gate, input gate, and output gate, respectively. These are the input weight matrices for the forget gate, input gate, and output gate, respectively. These are the bias vectors for the forget gate, input gate, and output gate, respectively. The intermediate state at time t in layer l. It can be obtained through equation (7):

[0100]

[0101] Then, the memory cells of the first layer of the LSTM and the hidden state at time t are... Updated as shown in equations (8) and (9):

[0102]

[0103]

[0104] Bi-LSTM includes forward information hiding states. and backward information hiding state Therefore, compared to LSTM, which only contains hidden states with forward information, Bi-LSTM can more fully utilize time-series information, extract data features for a specific time period, and increase the model's robustness to interference. The final output of Bi-LSTM is the sum of the two hidden states, i.e. The loss function of Bi-LSTM is the same as that in equation (1), which is the cross-entropy loss function, and stochastic gradient descent is used to minimize the loss function J. BiLSTM .

[0105] The initial foam data collected online is preprocessed using Z-score normalization to generate foam data. After the Bi-LSTM network is trained, the Bi-LSTM teacher module M is obtained. T-BiLSTM The foam data is input into the Bi-LSTM teacher module, and the features extracted from the last layer are used as the first data feature and denoted as T. D=[t T-BiLSTM，1 , t T-BiLSTM，2 , ..., t T-BiLSTM，N ] T ,in J D Let n be the dimension of the first data feature, where n = 1, 2, ..., N.

[0106] In this application, the C3D network structure used for extracting image features and the Bi-LSTM network structure used for extracting data features are shown in Tables 2 and 3. In Table 2, C1(32@3×3×3) indicates that there are 32 3×3×3 convolutional kernels in the first layer, and the stride of the convolutional kernels in the network is 1. P1(2×2×2) indicates that the max pooling execution size in the first layer is 2×2×2, and the stride is 2 in the spatial dimension and 1 in the temporal dimension. According to cross-validation, the learning rate of the image feature extraction network module is determined to be 0.0001, the minimum batch size is 64, and after 800 training iterations, the validation set progress of the coarse selection module reaches 98.59%. The validation accuracy of the scanning module is 93.2%. The network structure of the coarse selection and scanning fusion module 400 based on foam images is a simple concatenation of the extracted features on the basis of the first two structures. After training, the validation set accuracy of this network module is 98.8%, which is higher than the classification accuracy of the first two modules. The learning rate of the data feature extraction network module is 0.0001, the minimum batch size is 64, and after 40 training iterations, the validation set accuracy reaches 99.48%.

[0107] Table 2 shows the network structure of the C3D teacher module;

[0108]

[0109] Table 3 shows the network structure of the Bi-LSTM teacher module;

[0110]

[0111] S400, generate second coarse-selected image features based on the first coarse-selected image features;

[0112] In step S400, according to the knowledge distillation algorithm, the student module 300M of the C3D coarse image extraction module is obtained. S-CNN-R And obtain M S-CNN-R Extracted second coarse-selected image features t S-CNN-R .

[0113] Knowledge distillation is mainly divided into response-based knowledge distillation, relation-based knowledge distillation, and feature-based knowledge distillation based on the categories of knowledge extracted. Response-based knowledge modules use the final prediction result of the teacher module as a label to supervise the student module 300 in learning the hidden knowledge contained therein, thus achieving simple and effective module compression. Relation-based knowledge distillation does not learn the result of the teacher module's feature output, but rather the relationships between layers and between sample data. By providing an identity relation mapping, it enables the student module 300 to better learn the relational knowledge of the teacher module. Feature-based knowledge distillation uses the output of the intermediate feature layers of the teacher module as the knowledge to supervise the student module 300. Compared to response-based knowledge modules, feature-based knowledge distillation can significantly reduce the capacity difference between the hidden layers of complex teacher and simple student modules 300, thereby enhancing the expressive power of the student module 300. Generally, the loss of feature-based knowledge distillation can be expressed as:

[0114]

[0115] Among them W T W is a feature parameter of the teacher module. s Φ is a characteristic parameter of the student module. T and Φ s These are the transformation functions for the teacher and student modules, respectively, used to match the dimension of the network feature output. A schematic diagram of feature-based knowledge distillation learning is shown below. Figure 5 As shown.

[0116] To obtain a more lightweight network model, we sought a network model that simultaneously satisfies high accuracy and lightweight design based on the existing network and combined with experiments. The specific operation is as follows: After training the C3D and Bi-LSTM teacher networks by minimizing the cross-entropy loss function, according to the knowledge distillation algorithm, while keeping the number of network layers unchanged, we continuously reduced the number of C3D convolutional kernels and Bi-LSTM nodes to train the C3D and Bi-LSTM student network modules respectively. We calculated the sum of the mean squared errors of the output of each pooling layer in the teacher module and the pooling layer output of the student module 300, and trained the student module 300 by minimizing the mean squared error and the cross-entropy loss function to achieve intermediate layer feature learning. The specific calculation method of the loss function is shown in Equation (11):

[0117]

[0118] Where L represents the total number of layers in the teacher module and the student module.

[0119] Knowledge distillation experiments were conducted on both the C3D-based teacher module and the Bi-LSTM-based teacher module. Considering both accuracy and network model lightweighting, the C3D teacher module (Table 4 structure) was selected as the image feature extraction module, achieving a validation set accuracy of 96.3% for the student module. The Bi-LSTM teacher module (Table 5 structure) was selected as the data feature extraction module, achieving a validation set accuracy of 95.6% for the student model.

[0120] Table 4 shows the network structure of the C3D student module;

[0121]

[0122] Table 5 shows the network structure of the Bi-LSTM student model;

[0123]

[0124] S500, generate second scanned image features based on the first scanned image features;

[0125] In step S500, the student model M of the C3D scanned image extraction module is obtained according to the knowledge distillation algorithm. S-CNN-S M S-CNN-S Extracted second scan image features t S-CNN-S .

[0126] S600, Generate a second data feature based on the first data feature;

[0127] In step S600, according to the knowledge distillation algorithm, the student module M of the Bi-LSTM data extraction module... S-BiLSTM M S-BiLSTM Extracted second data feature t S-BiLSTM .

[0128] S700, the second coarse-selected image features, the second scanned image features, and the second data features are input into the fusion module 400 for fusion to generate the first fusion feature;

[0129] In step S700, the step of inputting the second coarse-selected image features, the second scanned image features, and the second data features into the fusion module 400 for fusion and generating the first fusion feature includes: fusing the second coarse-selected image features and the second scanned image features through STE to generate coarse-selected image fusion features and scanned image fusion features; constructing image fusion features based on the coarse-selected image fusion features and the scanned image fusion features; performing self-fusion of the second data features to generate a third data feature; and performing mutual fusion of the image fusion features and the second data features through CTE to generate the first fusion feature. The step of generating a state level label based on the first fusion feature includes: inputting the first fusion feature into the feedforward layer and generating a second fusion feature; inputting the second fusion feature into the SoftMax classifier and generating the state level label. The step of inputting the first fusion feature into the feedforward layer and generating the second fusion feature also includes: the feedforward layer expression is shown in equation (12):

[0130]

[0131] in, This is the weight matrix. This is the deviation vector.

[0132] In step S700, the fusion module 400 is a Transformer module, which is based on the attention mechanism to fuse image features and data features. When describing features of different state levels, it adaptively assigns different weights to different types of features, realizes the effective fusion of multi-source heterogeneous information, and thus improves the accuracy and reliability of the evaluation model.

[0133] First, the second coarse-selected image features and the second scanned image features are fused using STE to obtain the fused coarse-selected image features. and scan image features Then, the fused features of the coarse-selected image and the fused features of the scanned image are concatenated to obtain the fused image features. The second data feature t S-BiLSTM Self-fusion is performed to obtain the third data feature t D Furthermore, the image features t are fused. I Second data feature t D The heterogeneous information fusion feature, also known as the first fusion feature, is obtained by mutual fusion. in and This is the weight matrix of the mutual fusion network, used to match the dimensions of image fusion features and data fusion features.

[0134] The first fusion feature t after fusion fusionAs the input to the feedforward layer, the feedforward layer expression is shown in Equation (12).

[0135] Will As input to the SoftMax classifier, the resulting state level label from the C3D-Bi-LSTM-Transformer network is denoted as... The loss function of C3D-Bi-LSTM-Transformer is shown in equation (13):

[0136]

[0137] The parameters of the C3D-Bi-LSTM-Transformer network were updated and optimized using the backpropagation (BP) algorithm. After the overall backpropagation fine-tuning was completed, a runtime evaluation model based on the C3D-Bi-LSTM-Transformer was obtained, thus completing offline modeling.

[0138] The trained C3D teacher module was used as the image feature extraction module of the fusion network, and the trained Bi-LSTM teacher module was used as the data feature extraction module. A linear mapping layer was used to unify the dimensions, and the second coarse-selected image features and the second scanned image features were fed into the STE to output the first fused feature. Further, a linear mapping layer was used to unify the image and data dimensions, and the first fused feature, obtained by fusing the image fusion feature and the third data feature, was obtained through the CTE. Finally, the first fused feature was passed through an MLP network and SoftMax to output the classification result. In the training of the C3D-Bi-LSTM-Transformer network, to achieve high accuracy and a lightweight network model, the learning rate was determined to be 0.00001 and the minimum batch size to be 64 based on cross-validation. After 60 rounds of training, the model's validation set progress reached 100%. The fusion network structure consists of one layer of STE, one layer of CTE, and one layer of MLP, with linear mapping used to unify the feature dimensions. The specific structure is shown in Table 6.

[0139] Table 6 shows the network structure of the attention module;

[0140]

[0141]

[0142] S800, generate a state level label based on the first fusion feature;

[0143] In step S800, the step of generating a state level label based on the first fusion feature includes: denoting the first fusion feature extracted from the nth sample as... Where P is the number of foam images in each training sample, and J is... I Let n = 1, 2, ..., N be the feature dimensions of the bubble image; the actual state level label of the first fused feature is obtained using one-hot encoding. The actual state level label y of the image feature is obtained using one-hot encoding. n The calculation is performed using equation (14):

[0144]

[0145] Where, θ R ={θ R，1 θ R，2 , …, θ R，C} represents the parameters of the SoftMax classifier. This represents the posterior probability that the nth sample belongs to the cth state level.

[0146] In practical applications, the Rectified Linear Unit (ReLU) has a higher computational speed than other activation functions in the activation function layer of CNNs, which can significantly improve gradient propagation and is therefore widely used. Its definition is shown in equation (15):

[0147] f(x) = max(x, 0) (15);

[0148] Assuming there are N image training samples covering C state levels, the image samples are pixel-normalized and each pixel is divided by 255. The first fusion feature extracted from the nth sample is denoted as... P is the number of images in each training sample, J I Given image feature dimensions n = 1, 2, ..., N, flattening the image features yields... J I′ =P×J I And use one-hot encoding to mark its actual state level label as Record the status level label output by C3D as... It can be calculated using equation (14).

[0149] The loss function of C3D is shown in equation (1), and stochastic gradient descent is used to minimize the loss function J. C3D After network training, the C3D coarse-selected teacher network module M is obtained. T-CNN-R And C3D scanning teacher network module M T-CNN-S Let the set of features formed by the first coarsely selected image be denoted as... Let the set of features formed by the first scanned image be denoted as in The first coarse selection of image feature dimensions, This represents the first dimension of the scanned image features.

[0150] S900, determine the status level of the current flotation process operation based on the status level label.

[0151] In step S900, the step of determining the status level of the current flotation process operation status according to the status level label further includes: the status level is the index of the largest element in the status level label.

[0152] In practical application scenarios, the current state level is: The index of the largest element in the middle, i.e., if The current process is operating at state level C*. In the online evaluation, the test set contains 5220 samples. Samples 1-1008 belong to state level "Poor," samples 1009-2340 to state level "Medium," samples 2341-3816 to state level "Good," and samples 3817-5220 to state level "Excellent." The evaluation method based on C3D-Bi-LSTM-Transformer achieved an accuracy of 99.1%, a precision of 99.57%, a recall of 99.57%, and an F1 score of 99.57%. This indicates that the online evaluation results based on C3D-Bi-LSTM-Transformer are consistent with the actual operating conditions.

[0153] The student C3D-Bi-LSTM-Transformer model has fewer network parameters (0.9M) than the teacher model (9.2M), yet still maintains high evaluation accuracy. This demonstrates that by rationally utilizing multi-source heterogeneous information, the performance of the runtime evaluation model has been effectively improved. Furthermore, the C3D-Bi-LSTM-Transformer offers a more lightweight model deployment, making it more competitive in real-world industrial applications.

[0154] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims of this application.

Claims

1. A system using a lightweight feature extraction model, characterized in that, The system includes: a first teacher module, a second teacher module, a student module, and a fusion module; The first teacher module is configured to: acquire a bubble image, and generate a first coarse-selection image feature and a first scan image feature based on the bubble image; The second teacher module is configured to: acquire bubble data and generate a first data feature based on the bubble data; The student module is configured to: generate a second coarse-selected image feature based on the first coarse-selected image feature; generate a second scanned image feature based on the first scanned image feature; and generate a second data feature based on the first data feature. The fusion module is configured to: fuse the second coarse selection image features, the second scan image features, and the second data features to generate a first fusion feature, and generate a status level label based on the first fusion feature; and determine the status level of the current flotation process based on the status level label. The step of inputting the second coarse-selected image features, the second scanned image features, and the second data features into the fusion module for fusion and generating the first fusion feature includes: fusing the second coarse-selected image features and the second scanned image features through STE to generate coarse-selected image fusion features and scanned image fusion features; constructing an image fusion feature based on the coarse-selected image fusion feature and the scanned image fusion feature; performing self-fusion of the second data features to generate a third data feature; and performing mutual fusion of the image fusion feature and the second data feature through CTE to generate the first fusion feature.

2. A method for evaluating the operational status of a flotation process based on lightweight heterogeneous information, characterized in that, The method includes: Acquire foam images and obtain foam data based on the foam images; The bubble image is input into the first teacher module to generate the first coarse-selection image features and the first scanned image features; The foam data is input into the second teacher module to generate the first data feature; Generate second coarse image features based on the first coarse image features; Generate second scanned image features based on the first scanned image features; Generate a second data feature based on the first data feature; The second coarse-selected image features, the second scanned image features, and the second data features are input into the fusion module for fusion to generate the first fused feature; The steps of inputting the second coarse-selected image features, the second scanned image features, and the second data features into the fusion module for fusion and generating the first fused feature include: The second coarse-selected image features and the second scanned image features are fused using STE to generate coarse-selected image fusion features and scanned image fusion features; Image fusion features are generated based on the coarse-selected image fusion features and the scanned image fusion features; the second data features are then self-fused to generate a third data feature. The image fusion feature and the third data feature are fused together using CTE to generate the first fusion feature; A status level label is generated based on the first fusion feature; The status level of the current flotation process is determined based on the status level label.

3. The flotation process operation status evaluation method based on lightweight heterogeneous information as described in claim 2, characterized in that, Before acquiring foam images and obtaining foam data from them, the following steps are also included: The initial foam images and initial foam data acquired online are normalized respectively; the initial foam images are initial foam images inside the flotation tube acquired in real time by an industrial camera; the initial foam data are data information on the foam state extracted using a foam image analyzer; the data information on the foam state includes foam stability and flow rate.

4. The flotation process operation status evaluation method based on lightweight heterogeneous information as described in claim 2, characterized in that, The steps for generating a state level label based on the first fusion feature include: From the The first fusion feature extracted from each sample is denoted as... ; in, The number of foam images in each training sample. For the feature dimension of the foam image, ; The actual state level label of the first fused feature is obtained using one-hot encoding.

5. The flotation process operation status evaluation method based on lightweight heterogeneous information as described in claim 4, characterized in that, The actual state level label of the image features is obtained using one-hot encoding. Use the following formula: ； in, These are the parameters of the SoftMax classifier. Indicates the first The sample belongs to the first The posterior probability of each state level.

6. The flotation process operation status evaluation method based on lightweight heterogeneous information as described in claim 2, characterized in that, The step of inputting the foam image into the first teacher module to generate the first coarse-selected image features and the first scanned image features further includes: The loss function for the first teacher module is: ； in, For status level labels.

7. The flotation process operation status evaluation method based on lightweight heterogeneous information as described in claim 6, characterized in that, The steps for generating a state level label based on the first fusion feature include: The first fusion feature is input into the feedforward layer to generate the second fusion feature; The second fused feature is input into the SoftMax classifier to generate the state level label.

8. The flotation process operation status evaluation method based on lightweight heterogeneous information as described in claim 7, characterized in that, The step of inputting the first fused feature into the feedforward layer and generating the second fused feature further includes: The feedforward layer expression is: ； in, , This is the weight matrix. , This is the deviation vector.

9. The flotation process operation status evaluation method based on lightweight heterogeneous information as described in claim 2, characterized in that, The step of determining the status level of the current flotation process operation based on the status level label further includes: The status level is the index of the largest element in the status level label.

Images