Flotation froth dynamic diagnosis and self-adaptive regulation and control system based on multi-mode depth perception and time sequence prediction

Abstract

The invention discloses a flotation froth dynamic diagnosis and self-adaptive regulation and control system based on multi-mode depth perception and time sequence prediction. The flotation froth dynamic diagnosis and self-adaptive regulation and control system aims at solving the problems that in the prior art, the flotation process is not comprehensive in monitoring perception, dynamic prediction is missing, and regulation and control self-adaptability is poor. According to the invention, by deploying a multi-source heterogeneous sensor array, multi-modal data of vision, spectrum, acoustics and the like of foam are synchronously collected; and generating comprehensive foam comprehensive state characterization by using a cross-modal attention fusion network. Modeling is carried out on dynamic evolution of foam by adopting a hierarchical time sequence prediction and anomaly detection network, the future state trend is accurately predicted, and early warning of anomaly is realized. And finally, an intelligent regulation and control agent based on deep reinforcement learning is constructed, the intelligent regulation and control agent autonomously decides optimal process parameter adjustment according to the current state and future prediction, and online learning and optimization are carried out through continuous interaction with the actual process. The beneficiation recovery rate, the grade and the stability of the production process are remarkably improved, and the operation cost is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mineral flotation process monitoring and control, and particularly relates to a flotation froth dynamic diagnosis and self-adaptive regulation system based on multi-modal deep perception and time series prediction. BACKGROUND

[0002] Flotation is one of the most widely used and important separation methods in mineral processing, and its efficiency directly affects the subsequent smelting cost and resource recovery rate. The stability and optimal control of the flotation process are highly dependent on the accurate monitoring and timely regulation of the froth state in the flotation tank. The morphology (such as size, thickness, stability), color, and distribution of the surface attached minerals of the froth are key characteristics that reflect the separation effect and running state of the flotation process.

[0003] The monitoring and regulation of the pre-flotation process mainly faces three levels of technical limitations. First, the limitation of perceived information. Existing monitoring methods mostly rely on single visual image analysis, but visual information is easily disturbed by factors such as industrial site light changes and froth reflections, making it difficult to accurately capture the three-dimensional structure, internal mineral composition, and other deep information of the froth. Even if multiple sensors are deployed, the data fusion technology often stays at the primary level of simple weighting, failing to effectively exploit the complementary relationship between different data sources, and thus failing to form a comprehensive and in-depth understanding of the froth state. Secondly, the lack of dynamic prediction ability. Most methods focus on the static evaluation of the current state of the froth, while flotation itself is a complex dynamic process, and the occurrence of abnormal conditions (such as froth overload or collapse) often has its evolution stage. Due to the lack of effective prediction of this evolution trend, the control system can only respond passively after the problem occurs, missing the best opportunity to take preventive measures, directly affecting the timeliness and effectiveness of the regulation, and making it difficult to ensure the continuous stability of production. Finally, the lack of intelligence and adaptability of regulation decisions. Currently, production regulation still largely relies on manual experience or PID controllers based on fixed rules. Manual regulation has strong subjectivity and response lag, while fixed rule control strategies are difficult to cope with complex and variable conditions such as fluctuations in ore properties, and cannot achieve a dynamic balance between recovery rate, grade, stability, and other optimization goals, resulting in a low level of fine control of the entire process.

[0004] Therefore, how to realize multi-dimensional deep perception of the state of the flotation froth, accurately predict its dynamic evolution trend, and construct an intelligent control strategy that can be self-learned and optimized based on this is a key technical problem to be solved in the current flotation technology field, and has important significance for improving the automation and intelligent level of the flotation process and improving the resource utilization rate and economic benefit. The present application aims to solve the above problems, and an innovative method for dynamically diagnosing, trend predicting and intelligently self-adapting and controlling the flotation froth is constructed by introducing advanced multi-modal deep learning and reinforcement learning technology. SUMMARY

[0005] To solve the above technical problems, the present application provides a flotation froth dynamic diagnosis and intelligent control method based on multi-modal deep perception and time series prediction, which mainly includes the following steps: S1: multi-modal data acquisition, deep feature extraction and fusion step S101: Synchronously acquire multiple modal data of the flotation froth through a multi-source heterogeneous sensor array deployed above the flotation tank. The multi-modal data at least includes: a) high-resolution visual image sequence: capture information such as the morphology (size, shape, distribution) of the froth, color texture, motion speed and stability (such as the frequency of rupture). b) hyperspectral / multispectral image sequence: obtain spectral reflection information of different wavebands on the froth surface, which is used to analyze the type, content and distribution of the mineral attached to the froth surface. c) acoustic signal sequence: In some embodiments, the multi-modal data can also include an acoustic signal sequence. The acoustic signal generated during the froth generation, flow, merger and rupture process is acquired through a high-sensitivity microphone array, which is used to assist in judging the stability, gas content and flow pattern of the froth. d) Process parameter data: Synchronously record process parameters related to the flotation process, such as the aeration amount, pulp liquid level, type and rate of reagent addition, pulp properties (pH value, concentration), etc.

[0006] S102: Preprocess the collected original data of each modality, including but not limited to: a) visual image: image denoising (such as using wavelet transform or non-local mean denoising), illumination compensation, region of interest (ROI) extraction, image enhancement. b) spectral image: spectral correction (radiometric correction, atmospheric correction), noise filtering, spectral feature band selection. c) acoustic signal: signal denoising (such as Wiener filtering, spectral subtraction), feature extraction (such as Mel frequency cepstral coefficient MFCC, sound pressure level). d) Process parameters: outlier rejection, normalization processing.

[0007] S103: For each modality data after preprocessing, a modality-specific deep feature extraction network is constructed and trained to learn the high-level semantic features of each modality data: a) Visual feature extraction: adopt a convolutional neural network (CNN) based architecture, such as improved ResNet, EfficientNet or VisionTransformer (ViT), to extract deep visual features of foam morphology, color, texture, etc. To capture temporal dynamics, a temporal convolution network (TCN) or ConvLSTM layer can be connected after CNN. b) Spectral feature extraction: adopt 1D-CNN, 2D-CNN (for hyperspectral cube) or MLP network combined with attention mechanism to extract spectral features related to mineral composition and content. c) Acoustic feature extraction (if used): If acoustic signals are collected, a CNN or recurrent neural network (RNN, such as LSTM / GRU) based architecture can be used to extract acoustic patterns related to foam dynamic behavior from acoustic features (such as MFCC sequence).

[0008] S104: A cross-modal attention fusion network (CMAFN) is designed and implemented to intelligently fuse the deep features of each modality extracted in S103, generating a unified, information-enhanced foam comprehensive state representation vector. The specific implementation of the CMAFN network can include: a) Inter-modal interaction learning: through multi-head self-attention mechanism or cross-modal attention mechanism, features of different modalities can interact and learn the relevance and complementarity between each other. For example, visual features can guide the weight of spectral features in a specific area, and acoustic features can verify the foam stability observed by vision. b) Dynamic weight allocation: the network can adaptively allocate different fusion weights to features of different modalities according to current input data and task requirements, rather than fixed or simple averaging. c) Feature space alignment (optional): before fusion, features of different modalities can be mapped to a shared latent feature space through nonlinear transformation, to facilitate more effective fusion. Output: fused foam comprehensive state vector , which is a time Comprehensively describes the state of the foam.

[0009] S2: Foam dynamic evolution modeling and early warning of anomalies S201: Based on the foam comprehensive state vector sequence output by S104 , a Hierarchical Spatio-Temporal Prediction and Anomaly Detection Network (HSTP-ADN) is constructed for modeling the complex dynamic evolution of foam state and early warning of anomalies. The network can adopt a Transformer-based Encoder-Decoder architecture, such as Informer, Autoformer, or a specially designed temporal Transformer variant. a) Encoder: Receives the historical sequence of foam integrated state vectors, learns the long-range dependencies and complex patterns of foam state in the time dimension through multi-layer self-attention mechanisms and feedforward networks. b) State decoding and representation: In some embodiments, a decoder branch can be designed to decode the encoder output at the current time to obtain a more refined parametric description of the current foam state (e.g., average size, stability index, key mineral content estimates, etc.). c) Anomaly Detector: Based on the output of the encoder or the output of the state decoder, an anomaly scoring / classification subnetwork is connected. This subnetwork can be based on reconstruction error (for autoencoder type Transformers), prediction error (if short-term prediction is also performed), or directly train a classifier to determine whether the current state deviates from the normal operating range and output the anomaly probability or anomaly type (such as "foam overload", "foam collapse trend", "poor mineralization", etc. predefined categories). The detector should be able to achieve early warning of anomalies.

[0010] S3: Future state evolution trend prediction step S301: Based on the understanding of the historical state sequence by the encoder of the HSTP-ADN, its prediction decoder is used to perform multi-step prediction of key state indicators of the foam in the future. a) Multi-step prediction decoder: This decoder receives the output of the encoder and the query of the time step to be predicted, generating the predicted value of the foam integrated state vector sequence at the future ​b) Key indicators extraction and calibration: extract the key performance indicators (KPIs) of interest to the operator from the predicted comprehensive state vector, such as the predicted average bubble size, stability index, target mineral recovery related indicators, etc. c) Hybrid prediction enhancement: to further improve the robustness and accuracy of the prediction, in some embodiments, the prediction results of the HSTP-ADN can be integrated with the prediction results of one or more auxiliary models (such as ARIMA, SARIMA, etc. statistical models, or SVR, etc. machine learning models). For example, a gating mechanism or another small neural network can be designed to dynamically fuse the predictions from different models, or let the deep model predict the residual of the auxiliary model.

[0011] S4: Deep reinforcement learning based intelligent control decision and online optimization step S401: Construct a Deep Reinforcement Learning (DRL) based Intelligent Control Agent (ICA) to autonomously decide the optimal control actions (such as adjusting the aeration rate, reagent addition rate, etc.) according to the current froth state (from S104 or S201), future state prediction (from S301), and process constraints, and can perform online learning and policy optimization through interaction with the flotation process.

[0012] a) State space (State, ): The input states of the Intelligent Control Agent (ICA) include: i. The current fused comprehensive froth state vector (or its simplified version).

[0013] ii. The future step predicted key froth indicators .

[0014] iii. The current process parameter setpoints (such as aeration rate , reagent addition rate ). iv. Possible abnormal warning signals and their confidence levels (from S201).

[0015] b) Action space (Action, ): The output actions of the Intelligent Control Agent (ICA) are the adjustment amounts or target setpoints of the key process parameters. For example: i. : Adjustment amplitude of aeration rate (can be discrete levels or continuous values).

[0016] ii. : Adjustment of reagent addition rate The design of the action space needs to consider the actual ability of the actuator and the adjustment range allowed by the process.

[0017] c) Reward function ): Carefully designed reward functions are key to the success of DRL. The reward function should reflect the control objectives, for example: i. Mineral Processing Indicator Rewards: Based on actual or estimated improvements in mineral processing indicators such as recovery rate and grade over a period of time after regulation. This may require a supporting "mineral processing effect evaluation model" (which can be a regression model based on historical data or domain knowledge).

[0018] ii. Foam state optimization reward: whether the foam state is closer to the ideal state (such as the ideal size range and stability).

[0019] iii. Abnormal Avoidance Reward: Successfully avoids or mitigates the predicted abnormal state.

[0020] iv. Control cost penalty: Negative rewards are given for frequent or large-scale control actions, as well as excessive consumption of drugs, to encourage smooth and economical control. ,in is the weight coefficient.

[0021] d) DRL algorithm: You can choose a DRL algorithm suitable for continuous or discrete action space, such as DeepQ-Network (DQN) and its variants (RainbowDQN) for discrete actions, and DeepDeterministicPolicyGradient (DDPG), ProximalPolicyOptimization (PPO), SoftActor-Critic (SAC) for continuous action space.

[0022] e) Policy Network and Value Network: An intelligent control agent (ICA) typically consists of a policy network, which maps states to actions, and a value network, which evaluates the value of states or state-action pairs. These networks are implemented using deep neural networks.

[0023] S402: Online learning and policy iteration: a) The DRL intelligent regulation agent can be pre-trained through offline data (historical operation data) in the initial stage, or use the simulation environment for preliminary policy learning. b) After actual deployment, the intelligent regulation agent collects experience through continuous interaction with the real flotation process (performing actions, observing new states, obtaining rewards), and uses these experiences to update its policy network and value network online, thereby continuously optimizing the regulation policy to adapt to changing working conditions and ore properties. c) Design exploration mechanisms (such as epsilon-greedy, noise injection) to balance exploration (try new actions) and exploitation (execute the current optimal action).

[0024] S5: Closed-loop feedback and system iteration steps S501: Send the regulation instructions output by S401 to the actuators of the flotation equipment (such as valve controllers, dosing pump controllers) for implementation.

[0025] S502: After the regulation action is executed, the system returns to S101 to re-collect new multi-modal data, forming a closed-loop "perception-diagnosis-prediction-decision-implementation-re-perception" intelligent control process.

[0026] S503: The new (state, action, reward, next state) samples collected will be used to continuously train and optimize the intelligent regulation agent, HSTP-ADN model (S201, S301), and possibly auxiliary evaluation models. The parameters and structure of the models can be adjusted and iterated periodically or according to performance changes.

[0027] In another aspect, the present application also discloses a flotation froth dynamic diagnosis and intelligent regulation system based on multi-modal deep perception and time series prediction, comprising: A data acquisition and fusion module for synchronously acquiring multi-modal data of the flotation froth, wherein the multi-modal data at least includes a visual image sequence and a spectral image sequence; performing deep feature extraction on the multi-modal data respectively, and fusing the extracted deep features of each modality to generate a unified froth comprehensive state representation vector sequence; A trend prediction module for predicting the future key state indicators of the froth in multiple steps based on the froth comprehensive state representation vector sequence using a time series prediction network; An intelligent regulation module configured with an intelligent regulation agent based on deep reinforcement learning, for constructing an intelligent regulation agent based on deep reinforcement learning, autonomously deciding regulation actions according to the current froth state and the future multi-step predicted key state indicators; the input state space of the intelligent regulation agent includes the current froth comprehensive state representation vector and the future multi-step predicted key state indicators; And an execution module is configured to execute the regulation action and return workflow data to the data acquisition and fusion module to form a closed-loop control process including perception, prediction, decision-making and execution.

[0028] Compared with the prior art, the present application provides clear technical advantages. First, by fusing multi-source sensor data such as vision, spectrum and even acoustics through a deep learning framework, the present application can form a digital representation of the flotation froth state that is much more comprehensive and robust than single-mode analysis, thereby providing a more solid data foundation for diagnosis and decision-making. Second, the present system not only diagnoses the current state, but more importantly, models the dynamic evolution of the froth state to accurately predict its future trend. This prediction capability, combined with intelligent regulation based on reinforcement learning, enables the system to make proactive interventions, transforming traditional lagging response control into preventive optimization control. Finally, the entire method is constructed as a closed-loop learning system, enabling it to continuously optimize itself and iterate control strategies through continuous interaction with actual working conditions. This adaptive capability directly leads to improvements in ore recovery rate and grade, reduction in reagent consumption, and enhancement of the stability of the entire production process, among other practical benefits. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, brief descriptions will be given below to the drawings needed to be used in the embodiments or prior art descriptions.

[0030] Figure 1 is a whole process schematic diagram of a flotation froth dynamic diagnosis and intelligent regulation method based on multi-modal deep perception and time series prediction according to an embodiment of the present application; Figure 2 is a detailed block diagram of a multi-modal data acquisition, deep feature extraction and fusion step (S1) according to an embodiment of the present application; Figure 3 is a joint block diagram of a froth dynamic evolution modeling and early warning step (S2) and a future state evolution trend prediction step (S3) according to an embodiment of the present application; Figure 4 is a schematic diagram of an intelligent regulation decision-making step (S4) according to an embodiment of the present application, showing the interaction process between an intelligent regulation agent (ICA) and a flotation environment; Figure 5 is an internal structure schematic diagram of an exemplary cross-modal attention fusion network (CMAFN) according to an embodiment of the present application; Figure 6 is a structure schematic diagram of an exemplary hierarchical time series prediction and anomaly detection network (HSTP-ADN) according to an embodiment of the present application; Figure 7 is a comparison chart of multi-modal data instances under typical "foam overload" working conditions; Figure 8 is a performance comparison diagram of the present application and traditional control methods; Figure 9 is a human-machine interface (HMI) diagram of the flotation process intelligent diagnosis and control system. DETAILED DESCRIPTION

[0031] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0032] Figure 1 The overall flowchart of the flotation froth dynamic diagnosis and intelligent control method based on multi-modal deep perception and time series prediction according to one embodiment of the present application is shown. Referring to Figure 1 , the method of the present application mainly includes steps S1, S2, S3, S4 and S5. These steps will be described in detail below.

[0033] S1: Multi-modal data acquisition, deep feature extraction and fusion step This step aims to comprehensively obtain data related to the froth state from the flotation tank, and extract high-level semantic features through a deep learning model, and finally fuse into a unified, information-enhanced froth comprehensive state representation. Figure 2 The internal flow of this step is shown in more detail.

[0034] S101: Synchronously acquire multiple modal data of the flotation froth through a multi-source heterogeneous sensor array deployed above the flotation tank.

[0035] In the preferred embodiment of the present application, the sensor array is installed about 1-2 meters above the flotation tank mouth, and is equipped with an industrial-grade protective cover to cope with harsh environments such as splashing of ore pulp, dust and humidity. The data acquisition system is synchronized with the distributed control system (DCS) or programmable logic controller (PLC) system (such as Siemens PCS7 or Rockwell Plant PAx) of the factory, ensuring that the process parameters and sensor data are aligned in time.

[0036] The multi-modal data at least includes: high-resolution visual image sequences: foam images are continuously captured by an industrial-grade CMOS camera (such as Basler a2A1920-51gcPRO) at a rate of 10-30 frames per second, with a resolution of 1920x1080 pixels. These images not only visually reflect the size, shape and distribution of the foam, but also calculate the surface flow velocity field of the foam through the comparison analysis between consecutive frames (for example, using the particle image velocimetry method PIV), thereby quantifying the dynamic stability and flow characteristics of the foam. The rupture frequency of the foam can be obtained by tracking the life cycle of individual foams or foams in a small area.

[0037] Hyperspectral / multi-spectral image sequences: a push-broom or area-array hyperspectral imager (such as Specim FX10) is used to cover a wavelength range of 400-1000 nm. This data is used to identify and quantify key minerals attached to the surface of the foam. For example, in the copper-molybdenum separation process, chalcocite and molybdenite have different spectral absorption characteristics. By analyzing the reflectivity of a specific waveband, the mineralization degree of the foam can be evaluated in real time to determine whether there is "concentrate tailing" or "tailings concentrate" phenomenon, providing direct chemical composition basis for subsequent regulation.

[0038] Acoustic signal sequences: in some embodiments, a high-sensitivity directional microphone array can be deployed near the surface of the foam without physical contact. The dynamic process of the foam is accompanied by acoustic events: small, stable and well-mineralized foams usually produce relatively stable high-frequency acoustic signals; while a large number of foams rapidly merge or break (such as when collapse occurs), significant low-frequency pulse acoustic signals are generated. By analyzing the frequency spectrum and energy changes of the acoustic signals, the gas content, brittleness and stability of the foam can be judged, especially when visual information is severely disturbed by steam or light, acoustic information can provide important supplementary judgment basis.

[0039] Process parameter data: real-time acquisition of parameters closely related to the flotation process from the DCS / PLC system, such as feed quantity (t / h), pulp concentration (%), aeration quantity (m³ / min), agitator speed (rpm), pulp level (m), and instantaneous addition rate (L / min or g / t) of various reagents (such as collectors, frothers, depressants). These data provide a direct working condition background for subsequent state diagnosis and control decisions.

[0040] S102: Preprocessing of the collected original data of each modality.

[0041] The purpose of preprocessing is to eliminate noise and interference from irrelevant variables, standardize the data, and provide high-quality input for subsequent feature extraction.

[0042] Visual image: First, apply guided filtering or non-local means algorithm for denoising to preserve the details of froth edges. Second, apply illumination compensation algorithm based on gray world assumption or Retinex theory to weaken the influence of on-site lighting changes or mine lamp reflection. Then, extract the center region of the flotation tank as the region of interest (ROI) by preset or automatic identification to exclude the influence of fixed structures such as tank wall and overflow weir.

[0043] Spectral image: First, perform radiation calibration and reflectance correction, usually place a standard white board (such as a Teflon plate) as a reference on site to eliminate errors caused by changes in light source intensity and atmospheric absorption. Then, methods such as Savitzky-Golay filtering can be used to smooth the spectral curve and remove noise. To reduce the data dimension and highlight the key information, feature band selection can be based on domain knowledge or principal component analysis (PCA) method.

[0044] Acoustic signal: Use a band-pass filter (such as 300Hz-8000Hz) to filter out low-frequency mechanical noise and high-frequency electromagnetic interference in the factory environment. Then, extract features that can effectively represent the characteristics of the signal, such as Mel-frequency cepstral coefficients (MFCC), frequency spectrum after short-time Fourier transform, or sound pressure level.

[0045] Process parameters: Use 3-sigma principle or local outlier factor (LOF) algorithm to remove outliers caused by sensor jump or communication error. Then, normalize all parameters to the [0,1] interval to avoid different dimensions of parameters dominating the model training.

[0046] S103: For each modality data after preprocessing, construct and train a deep feature extraction network specific to the modality.

[0047] The core of this step is to use the powerful fitting ability of deep learning to automatically learn high-level semantic features from each modality data that are more expressive and robust than manually designed features. a) Visual feature extraction: Preferably, use a convolutional neural network (CNN) pre-trained on large datasets such as ImageNet, such as ResNet-50 or EfficientNet-B3 as the backbone network. Pre-trained weights enable the network to have basic image understanding capabilities. Then, fine-tune the network on a large number of labeled flotation froth images to adapt to the unique features of froth images. To capture the dynamic changes of froth, the sequence of single-frame feature maps extracted by CNN can be input into a temporal convolution network (TCN) or ConvLSTM, thereby obtaining visual dynamic features that integrate spatial and temporal information.

[0048] Spectrum feature extraction: For the spectrum curve of each pixel (one-dimensional data), a one-dimensional convolutional neural network (1D-CNN) can be constructed. Through multiple layers of convolution and pooling operations, this network can effectively capture the local and global patterns of absorption peaks, peak shape slopes, etc. related to mineral composition and content on the spectrum curve. For hyperspectral cubes (three-dimensional data), two-dimensional (2D-CNN) or three-dimensional convolutional neural networks (3D-CNN) can be used for feature extraction, considering both spectral and spatial information.

[0049] Acoustic feature extraction (if used): For acoustic feature sequences (such as MFCC matrices), a recurrent neural network (RNN) based architecture can be used, especially long short-term memory networks (LSTM) or gated recurrent units (GRU). These networks are good at processing time series data and can learn time patterns related to dynamic events such as bubble bursting and merging from acoustic signals.

[0050] S104: Design and implement a cross-modal attention fusion network (CMAFN) to intelligently fuse the deep features of each modality extracted in S103.

[0051] Simple feature concatenation or averaging cannot effectively utilize the complementarity and redundancy between multi-modal data. CMAFN aims to achieve a more intelligent and dynamic fusion. Figure 5 A possible internal structure of CMAFN is illustrated.

[0052] Inter-modal interaction learning: The core of the network is the cross-modal attention mechanism. For example, visual features can be used as "queries" (Query), while spectral and acoustic features are used as "keys" (Key) and "values" (Value). In this way, the network can learn that "when large and round bubbles appear visually (Query), more attention should be paid to the spectral features (Key-Value pair) of the corresponding area to determine whether they are over-mineralized or just large-size bubbles with low or no mineralization. Conversely, the network can focus on analyzing the bubble shape changes in the image at the corresponding time based on the acoustic detection of the breaking sound. This mechanism allows features from different modalities to be mutually calibrated and complementary.

[0053] Dynamic weight distribution: The output of the attention mechanism is essentially a set of dynamically calculated weights. This means that CMAFN can adaptively allocate fusion weights based on the confidence of the current working condition. For example, when the lighting conditions are poor and the image quality is low, the network will automatically reduce the weight of the visual features and rely more on acoustic and process parameter data.

[0054] Feature space alignment (optional): Before attention computation, feature vectors from different feature extractors (e.g. CNN, LSTM) with different dimensions and distributions can be mapped to a shared latent space with a unified dimension through respective independent nonlinear fully connected layers. This helps to improve the efficiency and stability of subsequent fusion computation.

[0055] Finally, CMAFN outputs a fused bubble comprehensive state vector with fixed length (e.g. 512 dimensions) at time . This vector is the unified input for subsequent diagnosis, prediction and control steps, and it condenses all modal information, providing the most comprehensive digital description of the current bubble state.

[0056] In a preferred embodiment, CMAFN can be specifically configured to map visual, spectral, acoustic features to a 512-dimensional shared latent space through independent fully connected layers respectively. The cross-modal attention module can contain 2 Transformer encoder layers, each using 8 multi-head attention, with an internal dimension of 2048 for the feedforward network. With this structure, the network can effectively interact and filter information between different modal features, and finally output a 512-dimensional comprehensive state vector with highly condensed information.

[0057] S2: Bubble dynamic evolution modeling and early abnormality warning step This step is based on the sequence of historical state vectors to model the dynamic behavior of the bubble and to achieve early identification and warning of abnormal conditions. Figure 3 and Figure 6 jointly show the HSTP-ADN network structure used in this step and S3.

[0058] S201: Based on the sequence of bubble comprehensive state vectors output by S104 , a hierarchical time series prediction and anomaly detection network (HSTP-ADN) is constructed.

[0059] Preferably, the network adopts a Transformer-based Encoder-Decoder architecture. The reason for choosing Transformer is that its self-attention mechanism can capture long-range dependencies between any two time points in the sequence, which is crucial for the flotation process because the current state can be influenced by a control action several minutes ago, which is difficult for traditional RNN or TCN to effectively model.a) Encoder: receives a sequence of bubble comprehensive state vectors By multi-head self-attention layers, the encoder is able to learn complex temporal patterns and dependencies within the state sequence, such as the periodic pattern of "foam size first increasing then decreasing", or the causal pattern of "after the increase of aeration volume, the foam stability first rises then falls".

[0060] In a preferred embodiment, the encoder of HSTP-ADN can be stacked by 6 encoder layers, each layer also adopts 8-head self-attention mechanism with model dimension (d_model) of 512. This deep structure enables it to capture complex temporal dependencies from seconds to minutes.

[0061] State decoding and representation: the output of the encoder can be regarded as a deep historical context-aware representation of the current state at time t. This representation can be decoded by a decoder branch (such as a simple multi-layer perceptron MLP) to output more physically interpretable current foam state parameters, such as "average foam diameter", "foam stability index", "concentrate grade estimate", etc. This provides intuitive monitoring indicators for operators.

[0062] Anomaly detector: this detector is the key of this step. It can work based on the representation of the encoder output. One implementation is to train an autoencoder to learn the pattern of state sequence under normal conditions. In real-time monitoring, if the reconstruction error (the difference between input and output) of the current state sequence exceeds the preset threshold, it is judged as abnormal. More preferably, a classifier can be trained to map the representation of the encoder output to predefined abnormal categories (such as "normal", "foam overload trend", "foam collapse risk", "poor mineralization"), and output the probability of each category. The advantage of this method is not only to detect abnormalities, but also to identify the type of abnormalities, which provides the basis for subsequent targeted regulation. For example, when the network output "foam collapse risk" probability continues to rise, even if no obvious changes have been observed by the naked eye, the system should issue a warning.

[0063] S3: future state evolution trend prediction step This step aims to "look ahead" and predict the future state of the foam, providing decision input for preventive and forward-looking control.

[0064] S301: based on the understanding of the historical state sequence by the encoder of HSTP-ADN, its prediction decoder is used to make multi-step prediction of key state indicators of the foam in the future.

[0065] Multi-step prediction decoder: this is another core function of HSTP-ADN. The decoder also receives the output of the encoder, but its task is to generate future predicted value of the froth comprehensive state vector at the next time step For example, the state evolution in the next 5 minutes (with a step of 30 seconds) is predicted.

[0066] Key indicator extraction and calibration: from the predicted future state vector sequence The predicted values of key performance indicators (KPIs) at each future time step, such as the "predicted froth size" and "predicted recovery rate" every half minute in the next 5 minutes, can also be parsed by a shared MLP decoder. This allows operators and control systems to see a future trend curve rather than just a current state point.

[0067] Hybrid prediction enhancement: to further improve the robustness of long-term prediction, in some embodiments, the prediction results of HSTP-ADN can be integrated with one or more simpler statistical models (such as ARIMA). For example, a gating neural network can be designed, whose input is the prediction of HSTP-ADN, the prediction of ARIMA, and the current state features, and whose output is the fusion weight of the two. When the working condition is stable and linear, the network may assign a higher weight to ARIMA; when the working condition is severely fluctuating or nonlinear, it relies more on the prediction of HSTP-ADN. This integration strategy combines the nonlinear fitting ability of deep models and the stability of statistical models.

[0068] S4: Intelligent control decision and online optimization step based on deep reinforcement learning This step is the core calculation and decision unit of the method of the present application, which autonomously decides the optimal control action based on the current state and future prediction to achieve the preset production target. Figure 4 Visually demonstrates the interaction between the intelligent control agent (ICA) and the flotation environment in this step.

[0069] S401: Construct an intelligent control agent (ICA) based on deep reinforcement learning (DRL).

[0070] State space (State, S): the basis for the decision-making of the intelligent control agent, its design is crucial. A comprehensive state should include: The current fused froth comprehensive state vector (or its decoded interpretable parameters).

[0071] Future Key froth indicator sequence predicted for the next step, especially the predicted trend (up, down, or stable).

[0072] Current process parameter set values, such as air charge, reagent addition rate, etc. ​

[0073] Early warning signals from S201 and their confidence. These are concatenated into a long vector as the input state of the DRL intelligent control agent.

[0074] Action space (A): the operations that the intelligent control agent can perform. To ensure smooth control, the action space is usually designed as the “adjustment amount” to the current setting. For example: Air charge adjustment: designed as 5 discrete actions: {large decrease (-10%), small decrease (-2%), keep unchanged (0), small increase (+2%), large increase (+10%)}.

[0075] Reagent addition rate adjustment: similarly designed as discrete adjustment levels for multiple reagents such as collectors, frothers, etc. For actuators that allow continuous control, it can also be designed as a continuous action space, e.g. each action is a normalized adjustment value in the range of [-1, 1].

[0076] Reward function (R): the reward function is used to guide the learning process of the intelligent control agent, and its design needs to consider multiple production objectives: Beneficiation index reward ( ): based on the changes in concentrate grade and recovery rate within a certain period (e.g. 5-10 minutes) after control, evaluated by the fusion features of S104 or the state decoder of S201. Positive reward is given for improvement, and negative reward is given for decline.

[0077] Froth state optimization reward ( ): based on whether the froth state is closer to the ideal range. For example, if the ideal average froth diameter is 15 mm, the current diameter is 20 mm, and an action makes it 18 mm, a positive reward is given.

[0078] Abnormality avoidance reward ( ): if S201 issues a “froth collapse” warning, and the action taken by the intelligent control agent successfully removes the warning at the subsequent time step, a larger positive reward is given.

[0079] Control cost penalty ( ): penalizes the magnitude and frequency of actions. For example, a small negative reward is given for each adjustment, and a larger negative reward is given for large adjustments. This encourages the intelligent control agent to perform more stable and economical control under the premise of ensuring effectiveness. The final total reward is the weighted sum of the above: , where the weight coefficients can be adjusted by process engineers according to production priorities. The weight coefficients The setting of the weights can follow the following guidelines: in the production stage where high yield is pursued, the weight related to full grade / recovery can be increased ; in the stage where stable operation is required and fluctuation of equipment is to be avoided, the weight of state optimization reward can be increased and the weight of regulation cost penalty ; when the nature of the ore being processed is complex and abnormal conditions are prone to occur, the weight of abnormality avoidance reward should be significantly increased . In some advanced embodiments, these weights can even be dynamically adjusted by an upper policy network according to long-term production data and targets, to achieve adaptive balance between different production targets.

[0080] DRL algorithm: for the above discrete action space, a deep Q network (DQN) or its improved version Rainbow can be selected. For continuous action space, the Soft Actor-Critic (SAC) or Proximal Policy Optimization (PPO) algorithm is preferred. SAC algorithm is a particularly suitable choice for continuous control in this scenario due to its good exploration efficiency and stability.

[0081] S402: online learning and policy iteration.

[0082] Offline pre-training: before the system goes online, the intelligent regulation agent can be pre-trained offline using the historical production data accumulated by the factory, including sensor data and manual operation records. This enables the intelligent regulation agent to have a baseline strategy that is not inferior to that of an average operator in the initial stage.

[0083] Online learning: after deployment, the intelligent regulation agent continuously interacts with the real flotation process, constantly generating new (state, action, reward, next state) four-tuple samples and storing them in the experience replay pool. The intelligent regulation agent continuously updates its policy network and value network using these new samples. This means that the intelligent regulation agent can learn autonomously, constantly discovering better control strategies than historical experience, and can adapt to slow changes in ore properties or process drifts caused by equipment wear and tear.

[0084] Exploration and utilization: to avoid falling into local optimum, the intelligent regulation agent needs to balance trying new, uncertain but potentially high-return actions and executing currently known optimal actions. The SAC algorithm endogenously encourages exploration through a framework that maximizes entropy. In DQN, the epsilon-greedy strategy can be used, i.e. with a small probability that decreases with training, randomly selecting an action.

[0085] S5: closed-loop feedback and system iteration steps This step connects all the above modules into a complete, self-running closed-loop intelligent control system. S501: Send the control instruction output by S401 to the actuator of the flotation device.

[0086] The action instruction (such as "small increase in air volume") output by the intelligent control agent (ICA) is translated into specific instructions for the PLC through an industrial communication protocol (such as OPCUA or Modbus TCP / IP), for example, increasing the frequency setting value of the control fan frequency converter from 35 Hz to 36 Hz, or increasing the stroke frequency of the control reagent metering pump from 50 times / min to 55 times / min.

[0087] S502: After the control action is executed, the system returns to S101 to reacquire new multi-modal data, forming a closed loop.

[0088] After the action is executed, the system waits for a fixed time (for example, 30 seconds) to allow the process to respond to the control, and then starts the next round of perception-diagnosis-prediction-decision-execution cycle. This closed loop runs periodically, achieving real-time and continuous intelligent monitoring and control of the flotation process.

[0089] S503: The new (state, action, reward, next state) samples collected will be used for continuous training and optimization.

[0090] All data from online interactions are recorded to form a growing high-value data set. These data are not only used for online updates of the intelligent control agent (ICA), but can also be used periodically for offline retraining and optimization of all deep learning models in S1 to S3 to prevent model performance from declining over time (i.e., model drift), ensuring the long-term stability and advancement of the entire system.

[0091] Comprehensive embodiment: a typical preventive control process To more clearly illustrate how the various parts of the invention work together, a typical complete scenario of proactive intervention through prediction is described below.

[0092] Initial state of the scenario (time t=0): The flotation process appears visually stable, but there is a potential fluctuation in ore properties, leading to excessive mineralization. 1. S1 (perception and fusion): The visual image shows that the foam color has darkened, but is still within the normal range. The CNN extracts the features "foam size is slightly large and color is slightly dark". The hyperspectral image captures key information: the spectral reflection characteristics of specific minerals on the foam surface are significantly enhanced, indicating that the degree of mineralization is rising rapidly, far beyond the optimal range. The high-frequency characteristics of the acoustic signal weaken, and the sound becomes dull, indicating that the foam brittleness has decreased and the flowability has deteriorated. When the CMAFN network is fused, the high spectral features and acoustic features are given higher weights through the attention mechanism, and the final output of the comprehensive state vector A potential risk state of "overload, under-stability" is explicitly pointed out.

[0093] S2&S3 (diagnosis and prediction): S2 (diagnosis and early warning): After receiving the sequence of continuous state vectors, the anomaly detector of the HSTP-ADN outputs an early warning of "foam overload trend" with a confidence of 75%. At this time, the on-site operator may not have noticed the explicit problem. S3 (trend prediction): More critically, the prediction decoder of the HSTP-ADN predicts that, based on the current trend, the foam stability index will decrease by 40% in the next 3-5 minutes without intervention, and there is a high probability of "foam collapse". This prediction result is quantified as a downward curve of future key indicators.

[0094] S4 (decision): The intelligent control agent (ICA) receives a state input containing multi-dimensional information: ① the current "overload" state vector ; ② the predicted curve of future stability that will decrease significantly; ③ the early warning signal of "foam overload trend". After evaluation, the strategy network of the intelligent control agent considers that maintaining the status quo or continuing to increase the reagent will result in a large negative reward value in the "abnormal avoidance reward" of the reward function. Through the evaluation of multiple possible actions, the intelligent control agent decides on an optimal strategy: moderately reduce the aeration volume to reduce the upward buoyancy, and slightly reduce the dosage of the collector to control the mineralization speed. The intelligent control agent outputs specific action instructions: aeration volume down by 5%, collector A addition rate down by 3%.

[0095] S5 (execution and feedback): The instructions are accurately executed through the PLC. In the next few minutes, the system returns to S1 for re-sensing. The new multi-modal data shows that the mineralization degree on the foam surface returns to normal level, the foam fluidity is enhanced, and the acoustic signal becomes clear. The S2 / S3 module confirms that the "foam overload trend" early warning is lifted, and the future stability prediction curve rises to the healthy range. After the reward function calculation, a large positive reward is obtained (mainly from successful abnormal avoidance), and this (state, action, reward, new state) sample is stored in the experience pool, further strengthening the correct decision-making ability of the intelligent control agent in similar situations.

[0096] Through the above closed-loop process, the present application effectively intervenes before the occurrence of catastrophic working conditions (foam collapse) through accurate prediction and intelligent decision-making, avoiding potential production accidents and ensuring the stability of the process.

[0097] Figure 8 The technical principle of the present application is demonstrated, which changes from a lagging response control to an active preventive control.

[0098] The conventional control method represented by the dashed line in the figure is essentially a kind of lag management that intervenes only after an anomaly occurs. When the working condition has a potential fluctuation at time t_1, the conventional system has no awareness of this and can only start taking remedial measures when the key performance indicator has already deviated significantly at time t_2, below the preset key performance indicator safety threshold. However, at this time, the problem has already formed, and at this time, the regulation intervention usually cannot avoid production fluctuations, thereby causing the production process to have a sharp and large fluctuation, and even possibly causing a foam collapse and other serious accidents due to untimely intervention, ultimately damaging the stability and economic benefits of production.

[0099] Compared with the conventional method represented by the dashed line, the method represented by the solid line of the present application can intervene in advance, effectively maintaining the stability of the key performance indicator. At the same time t_1, when the initial signs of working condition fluctuation appear, the prediction model of the present application can predict the downward trend of the future indicator based on the weak early data changes. Therefore, the system does not need to wait for the problem to worsen, but immediately takes a small-range and accurate forward-looking regulation after detecting the downward trend. The effect of this forward-looking preventive intelligent decision is obvious: the entire production process is almost not disturbed, and the key indicator quickly recovers to the ideal optimal state after only a slight fluctuation.

[0100] Finally, it should be noted that the above description is only for the preferred embodiments of the present application and is not intended to limit the protection scope of the present application. Any changes or replacements within the technical scope disclosed by the present application, such as the selection of different modal sensors, the variants of specific deep learning network models, or the specific implementation of reinforcement learning algorithms, should be covered within the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the protection scope of the appended claims.

Claims

1. A flotation foam dynamic diagnosis and intelligent control method based on multimodal depth perception and time series prediction, characterized in that: The following steps are involved: S1: Multimodal data acquisition and fusion step: synchronously acquiring multiple modal data of flotation foam, wherein the multimodal data includes at least a visual image sequence and a spectral image sequence; performing deep feature extraction on the multimodal data, and fusing the extracted deep features of each modality to generate a unified foam comprehensive state representation vector sequence; S3: Future state evolution trend prediction step: Based on the foam comprehensive state representation vector sequence, a time series prediction network is used to perform multi-step prediction of key future state indicators of the foam; S4: Intelligent Control Decision-Making Step: Construct an intelligent control agent based on deep reinforcement learning to autonomously make control decisions based on the current foam state and the key state indicators of the future multi-step prediction. The input state space of the intelligent control agent includes the current foam comprehensive state representation vector and the key state indicators of the future multi-step prediction. S5: Closed-loop feedback step: Execute the control action and return to step S1 to form a closed-loop control process including perception, prediction, decision-making and execution.

2. The method according to claim 1, characterized in that The fusion in step S1 is implemented using a cross-modal attention fusion network, which first maps the extracted deep features of each modality to a shared latent feature space, and then adaptively assigns different fusion weights to the features of different modalities through a cross-modal attention mechanism to achieve intelligent fusion of the deep features of each modality.

3. The method according to claim 1, characterized in that The multimodal data also includes: an acoustic signal sequence collected by a microphone array, and synchronously recorded process parameter data.

4. The method according to claim 1, wherein The time series prediction network in step S3 adopts a Transformer-based architecture to learn the long-range dependency of the foam state in the time dimension.

5. The method according to claim 4, characterized in that The method further includes a foam dynamic evolution modeling and abnormal early warning step S2, which shares the encoder based on the Transformer architecture with step S3; In step S2, the output of the encoder is connected to an anomaly detector, which determines whether the current state deviates from the normal operating range by calculating the reconstruction error output by the encoder or using a classifier and outputs an abnormality warning signal.

6. The method according to claim 5, characterized in that The input state space of the intelligent control agent in step S4 also includes the abnormal warning signal output in step S2.

7. The method according to claim 1, characterized in that The intelligent control agent in step S4 is trained using a composite reward function, which is calculated by weighted summing at least three of the following items: Mineral processing index rewards based on the improvement of mineral processing indexes; Bubble state optimization rewards based on whether the bubble state is close to the ideal state; Anomaly avoidance rewards based on success in avoiding or mitigating predicted anomalies; and regulation cost penalties that penalize the cost or frequency of regulatory actions; Among them, the weight coefficient of each item can be dynamically adjusted according to production priorities.

8. The method according to claim 1, characterized in that The control action output by the intelligent control agent in step S4 is an adjustment amount for one or more key process parameters, and the key process parameters include the inflation volume and / or the agent addition rate.

9. The method according to claim 1, characterized in that The method also includes online learning and strategy iteration steps: the intelligent control agent collects experience through continuous interaction with the real flotation process, and uses the collected experience to update its strategy online so that it can adapt to changing working conditions.

10. A flotation foam dynamic diagnosis and intelligent control system based on multimodal depth perception and time series prediction, characterized in that: include: A data acquisition and fusion module is used to synchronously collect multiple modal data of flotation foam, wherein the multimodal data includes at least a visual image sequence and a spectral image sequence; perform deep feature extraction on the multimodal data respectively, and fuse the extracted depth features of each modality to generate a unified vector sequence representing the comprehensive state of the foam; A trend prediction module is used to make multi-step predictions of key future state indicators of the foam based on the foam comprehensive state representation vector sequence using a time series prediction network; An intelligent control module is configured with an intelligent control agent based on deep reinforcement learning, which is used to build an intelligent control agent based on deep reinforcement learning to autonomously decide on control actions based on the current foam state and the key state indicators of the future multi-step prediction; The input state space of the intelligent control agent includes the current foam comprehensive state representation vector and the key state indicators of the future multi-step prediction; and an execution module for executing the control action and returning the workflow to the data acquisition and fusion module to form a closed-loop control process including perception, prediction, decision-making and execution.

Images