Flotation froth flow rate prediction method and device based on dynamic spatiotemporal attention fusion

Abstract

The application discloses a flotation froth flow rate prediction method and device based on dynamic space-time attention fusion, and relates to the technical field of computer vision. The method comprises the following steps: acquiring original flotation froth video data; preprocessing the original flotation froth video data by taking every 10 frames of images as a set to obtain flotation froth image sequence data; calculating the true value of the froth flow rate at the current moment by adopting an optical flow method, and taking the true value of the froth flow rate at the current moment as a training label; constructing an initial flotation froth flow rate prediction model; training the initial flotation froth flow rate prediction model based on the flotation froth image sequence data and the training label to obtain a trained flotation froth flow rate prediction model; and inputting flotation froth image data to be predicted into the trained flotation froth flow rate prediction model to obtain a flotation froth flow rate prediction result at a future moment. The application can improve the reliability and generalization of the flotation froth flow rate prediction.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and apparatus for predicting the flow velocity of flotation foam based on dynamic spatiotemporal attention fusion. Background Technology

[0002] The flotation froth processing is an industrial process influenced by a combination of factors, and its surface contains rich and complex feature information. However, the flotation process is complex and difficult to monitor in real time. Manual visual inspection is subjective and lagging, thus affecting production efficiency. Therefore, using machines to replace manual labor for accurate and efficient monitoring of the flotation process has become a research focus and an inevitable development trend.

[0003] The introduction of deep learning algorithms has brought a new breakthrough to image processing of flotation foam. It primarily uses a multi-layered, complex neural network structure to extract deep-level abstract features from the data and perform representation and learning. It directly uses images as input, without requiring pre-description of image features, and can automatically update and optimize the weights of each network. Supported by a large amount of training sample data and deep networks, deep learning has demonstrated superior classification results. Compared to traditional methods, deep learning algorithms not only improve the accuracy and robustness of foam physical feature extraction in complex environments, but also provide high-dimensional feature descriptions of foam, thus more comprehensively and accurately describing the complex behavior of the foam surface.

[0004] Currently, the measurement of flotation foam velocity in most industrial processes suffers from inaccurate results and is impossible to measure in some scenarios. Therefore, it is crucial to propose a reliable method for predicting foam velocity. Predicted foam velocity values ​​can provide important decision-making support for relevant industrial applications, promoting efficiency improvements and rational resource utilization in the flotation process. Summary of the Invention

[0005] To address the technical problems of inaccurate flotation foam velocity measurement results and inability to measure velocity in certain scenarios in existing technologies during industrial processes, this invention provides a method and apparatus for predicting flotation foam velocity based on dynamic spatiotemporal attention fusion. The technical solution is as follows:

[0006] On the one hand, a flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion is provided. This method is implemented by a flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion, and includes:

[0007] S1. Obtain raw flotation foam video data; preprocess the raw flotation foam video data into sets of 10 frames each to obtain flotation foam image sequence data;

[0008] S2. Calculate the true value of the foam velocity at the current moment using the optical flow method, and use the true value of the foam velocity at the current moment as the training label.

[0009] S3. Construct an initial flotation foam velocity prediction model;

[0010] S4. Based on the flotation foam image sequence data and the training labels, train the initial flotation foam velocity prediction model to obtain the trained flotation foam velocity prediction model.

[0011] S5. Input the flotation foam image data to be predicted into the trained flotation foam velocity prediction model to obtain the flotation foam velocity prediction results for future times.

[0012] Optionally, step S2 uses optical flow to calculate the true value of the foam flow velocity at the current moment, and uses the true value of the foam flow velocity at the current moment as a training label, including:

[0013] The optical flow method is used to calculate the gray value change of each pixel in two consecutive frames. Combined with the known motion of the camera and the foam surface, the motion field is accurately estimated using the Lucas-Kanade method to obtain the foam flow velocity and foam motion direction. Based on the foam flow velocity and foam motion direction, the overall flow velocity of the foam in the current frame image is obtained through vector merging. The overall flow velocity of the foam in the current frame image is used as the training label.

[0014] Optionally, the initial flotation foam velocity prediction model includes:

[0015] The system includes a contrast-adaptive histogram equalization module, a pipeline embedding module, a dynamic spatiotemporal encoder, and a multilayer perceptron layer.

[0016] Optionally, the limited contrast adaptive histogram equalization module is used to enhance the input original image sequence;

[0017] The pipeline embedding module is used to extract feature vectors from the input image sequence in multiple dimensions, including time, height, and width, and to fuse temporal and spatial information.

[0018] The multilayer perceptron layer is used to extract and transform input features layer by layer.

[0019] Optionally, the dynamic spatiotemporal encoder includes: a spatial encoder, a time encoder, and a dynamic gating unit;

[0020] The spatial encoder is used to extract spatial features from image data.

[0021] The time encoder is used to extract temporal features from image data;

[0022] The dynamic gating unit is used to calculate the correlation between spatial features and temporal features in the image data.

[0023] Optionally, step S4, based on the flotation foam image sequence dataset and the training labels, trains the initial flotation foam flow rate prediction model to obtain a trained flotation foam flow rate prediction model, including:

[0024] S41. Input the flotation foam image sequence data into the limited contrast adaptive histogram equalization module, and obtain multiple local sub-regions by dividing the flotation foam image into blocks; perform histogram equalization processing on the local sub-regions to obtain enhanced image data;

[0025] S42. Input the enhanced image data into the pipeline embedding module, and extract features from the enhanced image data from the time dimension, height dimension and width dimension to obtain the flotation foam image feature vector;

[0026] S43. Simultaneously input the feature vector of the flotation foam image into the dynamic spatiotemporal encoder. Process the feature vector of the flotation foam image through the spatial encoder to obtain the spatial feature vector; process the feature vector of the flotation foam image through the time encoder to obtain the temporal feature vector.

[0027] S44. Input the spatial feature vector and the temporal feature vector into the dynamic gating unit for concatenation to obtain the concatenated features; perform global average pooling on the concatenated features to obtain the fused feature vector.

[0028] S45. Input the fused feature vector into the multilayer perceptron layer, and perform feature extraction and transformation on the fused feature vector layer by layer through multiple neural network layers to obtain a preliminary prediction result of the flotation foam velocity; train the model based on the preliminary prediction result and the constructed loss function to obtain a trained flotation foam velocity prediction model.

[0029] Optionally, the process of inputting the spatial feature vector and the temporal feature vector into the dynamic gating unit for concatenation is represented by the following formulas (1)-(4):

[0030] (1)

[0031] (2)

[0032] (3)

[0033] (4)

[0034] in, Let be the learnable parameters, and represent the spatial gating weight matrix; These are learnable parameters, representing spatial gating bias; Here are the learnable parameters, and here represents the temporal gating weight matrix; These are learnable parameters, representing the timing gate bias; Indicates the first The splicing of spatial and temporal features; This represents the updated spatial characteristics; This represents the updated time series characteristics; Represents the Sigmoid function; This indicates element-wise multiplication; Indicates the first Output weights of the layer dynamic space gating unit; Indicates the first The output weights of the layer dynamic time gating unit and ; express A dimensional real vector space.

[0035] On the other hand, a flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion is provided. This device is applied to the flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion, and the device includes:

[0036] The acquisition unit is used to acquire raw flotation foam video data; the raw flotation foam video data is preprocessed by taking 10 frames of images as a set to obtain flotation foam image sequence data;

[0037] The computing unit is used to calculate the true value of the foam flow velocity at the current moment using the optical flow method, and uses the true value of the foam flow velocity at the current moment as the training label.

[0038] Building blocks are used to construct an initial flotation foam velocity prediction model;

[0039] The training unit is used to train the initial flotation foam flow rate prediction model based on the flotation foam image sequence data and the training labels, so as to obtain the trained flotation foam flow rate prediction model.

[0040] The prediction unit is used to input the flotation foam image data to be predicted into the trained flotation foam velocity prediction model to obtain the flotation foam velocity prediction result at future time.

[0041] Optionally, the computing unit is used for:

[0042] The optical flow method is used to calculate the gray value change of each pixel in two consecutive frames. Combined with the known motion of the camera and the foam surface, the motion field is accurately estimated using the Lucas-Kanade method to obtain the foam flow velocity and foam motion direction. Based on the foam flow velocity and foam motion direction, the overall flow velocity of the foam in the current frame image is obtained through vector merging. The overall flow velocity of the foam in the current frame image is used as the training label.

[0043] Optionally, the initial flotation foam velocity prediction model includes:

[0044] The system includes a contrast-adaptive histogram equalization module, a pipeline embedding module, a dynamic spatiotemporal encoder, and a multilayer perceptron layer.

[0045] Optionally, the limited contrast adaptive histogram equalization module is used to enhance the input original image sequence;

[0046] The pipeline embedding module is used to extract feature vectors from the input image sequence in multiple dimensions, including time, height, and width, and to fuse temporal and spatial information.

[0047] The multilayer perceptron layer is used to extract and transform input features layer by layer.

[0048] Optionally, the dynamic spatiotemporal encoder includes: a spatial encoder, a time encoder, and a dynamic gating unit;

[0049] The spatial encoder is used to extract spatial features from image data.

[0050] The time encoder is used to extract temporal features from image data;

[0051] The dynamic gating unit is used to calculate the correlation between spatial features and temporal features in the image data.

[0052] Optionally, the training unit is used for:

[0053] The flotation foam image sequence dataset is input into the contrast-limited adaptive histogram equalization module. The flotation foam image is divided into blocks to obtain multiple local sub-regions. Histogram equalization is then performed on the local sub-regions to obtain enhanced image data.

[0054] The enhanced image data is input into the pipeline embedding module, and feature extraction is performed on the enhanced image data from the time dimension, height dimension and width dimension to obtain the flotation foam image feature vector;

[0055] The feature vector of the flotation foam image is simultaneously input into the dynamic spatiotemporal encoder. The spatial encoder processes the feature vector of the flotation foam image to obtain the spatial feature vector. The temporal encoder processes the feature vector of the flotation foam image to obtain the temporal feature vector.

[0056] Spatial feature vectors and temporal feature vectors are input into a dynamic gating unit for concatenation to obtain concatenated features; global average pooling is then performed on the concatenated features to obtain a fused feature vector.

[0057] The fused feature vector is input into a multilayer perceptron layer, and the fused feature vector is extracted and transformed layer by layer through multiple neural network layers to obtain a preliminary prediction result of the flotation foam velocity. Based on the preliminary prediction result and the constructed loss function, the model is trained to obtain a trained flotation foam velocity prediction model.

[0058] Optionally, the process of inputting the spatial feature vector and the temporal feature vector into the dynamic gating unit for concatenation is represented by the following formulas (1)-(4):

[0059] (1)

[0060] (2)

[0061] (3)

[0062] (4)

[0063] in, Let be the learnable parameters, and represent the spatial gating weight matrix; These are learnable parameters, representing spatial gating bias; Here are the learnable parameters, and here represents the temporal gating weight matrix; These are learnable parameters, representing the timing gate bias; Indicates the first The splicing of spatial and temporal features; This represents the updated spatial characteristics; This represents the updated time series characteristics; Represents the Sigmoid function; This indicates element-wise multiplication; Indicates the first Output weights of the layer dynamic space gating unit; Indicates the first The output weights of the layer dynamic time gating unit and ; express A dimensional real vector space.

[0064] On the other hand, a flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion is provided. The flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-described methods for predicting flotation foam velocity based on dynamic spatiotemporal attention fusion is implemented.

[0065] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for predicting flotation foam velocity based on dynamic spatiotemporal attention fusion.

[0066] The beneficial effects of the technical solutions provided by the embodiments of the present invention include at least the following:

[0067] This invention proposes a flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion. By establishing a flotation foam velocity prediction model, it achieves accurate prediction of foam velocity at future moments. This invention fundamentally solves the technical problems of inaccurate velocity measurement and inability to measure velocity due to the complex dynamic characteristics of foam in industrial settings. This invention improves the expressive power of features by constructing a pipe embedding module, extracting multi-dimensional features, and fusing feature information. It also constructs a dynamic spatiotemporal encoder to capture local and global structural features in the foam video stream data, as well as dynamic changes and temporal dependencies between video frames, enhancing the model's understanding of spatial and temporal relationships. The predicted foam velocity values ​​obtained using this invention will provide important decision-making basis for related industrial applications, effectively promoting efficiency improvements and rational resource utilization in the flotation process. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a flowchart of a flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion provided by an embodiment of the present invention;

[0070] Figure 2 This is a schematic diagram of the structure of a dynamic spatiotemporal attention fusion model provided in an embodiment of the present invention;

[0071] Figure 3This is a schematic diagram of the structure of the dynamic gating unit provided in an embodiment of the present invention;

[0072] Figure 4 This is a schematic diagram of the Transformer encoder module provided in an embodiment of the present invention;

[0073] Figure 5 This is a block diagram of a flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion provided in an embodiment of the present invention;

[0074] Figure 6 This is a schematic diagram of the structure of a flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion provided in an embodiment of the present invention. Detailed Implementation

[0075] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0076] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0077] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0078] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0079] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0080] This invention provides a method for predicting flotation foam velocity based on dynamic spatiotemporal attention fusion. This method can be implemented by a flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion, which can be a terminal or a server. Figure 1The flowchart shown is for a flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion. The processing flow of this method may include the following steps:

[0081] S1. Obtain raw flotation foam video data; preprocess the raw flotation foam video data into sets of 10 frames each to obtain flotation foam image sequence data.

[0082] The flotation foam types include copper ore foam and phosphate ore foam. The flotation foam image sequence data was divided into training and validation sets in an 8:2 ratio.

[0083] S2. Calculate the true value of the foam velocity at the current moment using the optical flow method, and use the true value of the foam velocity at the current moment as the training label.

[0084] Optionally, the specific implementation process of S2 includes:

[0085] The optical flow method is used to calculate the gray value change of each pixel in two consecutive frames. Combined with the known motion of the camera and the foam surface, the motion field is accurately estimated using the Lucas-Kanade method to obtain the foam flow velocity and foam motion direction. Based on the foam flow velocity and foam motion direction, the overall flow velocity of the foam in the current frame image is obtained through vector merging. The overall flow velocity of the foam in the current frame image is used as the training label.

[0086] Based on the optical flow assumption, it is assumed that each point on the foam surface translates along its surface normal direction in two consecutive image frames. The optical flow method determines the relative motion of each point on the foam surface by tracking the grayscale value changes of each pixel in the image, and infers the pixel's trajectory and velocity information. In one feasible implementation, the optical flow method estimates the displacement of each pixel using the camera's motion model and the relative motion of the foam surface. By calculating the change in grayscale value of each pixel in two adjacent image frames and combining this with the known motion of the camera and the foam surface, the motion field is accurately estimated using the Lucas-Kanade method, further obtaining the foam flow velocity and foam motion direction. The results estimated by the optical flow method provide the spatial distribution information of the foam flow velocity at the current moment, which is used as the training label for the flotation foam flow velocity prediction model.

[0087] In one possible implementation, it is assumed that Indicates the coordinates in the image For a pixel at time t, according to the principle of optical flow, the pixel representation can be obtained through the following formula (1):

[0088] (1)

[0089] in, For foam along Distance of movement in the direction, For foam along Distance of movement in the direction, Let be the time interval; where, based on the expression of formula (1), formula (1) is further processed to obtain the expression of formula (2) as follows:

[0090] (2)

[0091] in, Since it is a second-order infinitesimal, it can be ignored. Substituting formula (2) back into formula (1) yields the following formula (3):

[0092] (3)

[0093] In this case, both sides of formula (3) are divided by... You can obtain: ,in, and The optical flow vector of the bubble image is denoted as ; the optical flow vectors are merged to obtain the result estimated by the optical flow method.

[0094] S3. Construct an initial flotation foam velocity prediction model.

[0095] Among them, such as Figure 2 The diagram shown is a structural schematic of a dynamic spatiotemporal attention fusion model provided in an embodiment of the present invention. The CLAHE module is a Contrast Limited Adaptive Histogram Equalization (CLAHE) module.

[0096] Optionally, the initial flotation foam velocity prediction model includes:

[0097] The system includes a contrast-adaptive histogram equalization module, a pipeline embedding module, a dynamic spatiotemporal encoder, and a multilayer perceptron layer.

[0098] Optionally, a contrast-adaptive histogram equalization module is limited for enhancing the input original image sequence;

[0099] Among them, the pipeline embedding module is used to extract feature vectors from the input image sequence in multiple dimensions of time, height and width, and to fuse temporal and spatial information;

[0100] Among them, the multilayer perceptron layer is used to extract and transform the input features layer by layer.

[0101] Among them, such as Figure 3 The figure shown is a structural schematic diagram of the dynamic gating unit provided in an embodiment of the present invention.

[0102] Optionally, the dynamic spatiotemporal encoder includes: a spatial encoder, a time encoder, and a dynamic gating unit;

[0103] Among them, the spatial encoder is used to extract spatial features from image data;

[0104] Among them, the time encoder is used to extract temporal features from image data;

[0105] The dynamic gating unit is used to calculate the correlation between spatial features and temporal features in image data.

[0106] Among them, the dynamic spatiotemporal encoder is also used to extract the local and global structures in video image data, as well as the dynamic changes and temporal dependencies between video image frames, and adaptively adjust the feature contribution ratio of time and space.

[0107] S4. Based on the flotation foam image sequence dataset and training labels, train the initial flotation foam flow rate prediction model to obtain the trained flotation foam flow rate prediction model.

[0108] Optionally, the specific implementation process of S4 includes S41-S45:

[0109] S41. Input the flotation foam image sequence data into the limited contrast adaptive histogram equalization module, and obtain multiple local sub-regions by dividing the flotation foam image into blocks; perform histogram equalization processing on the local sub-regions to obtain enhanced image data;

[0110] In one feasible implementation, the input flotation foam image is... Divided into There are several local sub-regions, each sub-region being 8×8 in size; among them, Indicates the height of the image. Indicates the width of the image. Indicates the number of channels; calculates the grayscale histogram for each sub-region. And set the cropping threshold to The average number of pixels is defined as When the total number of pixels in all histograms exceeds The portion is truncated and redistributed evenly across all pixels. This process is repeated until all truncated excess pixels are added to the histogram, i.e., until the pixel count reaches a certain threshold. The image is enhanced by using bilinear interpolation to eliminate boundary artifacts between sub-regions.

[0111] S42. Input the enhanced image data into the pipeline embedding module, and extract features from the enhanced image data from the time dimension, height dimension and width dimension to obtain the flotation foam image feature vector;

[0112] In one feasible implementation, non-overlapping spatiotemporal "tubes" are extracted from the input image sequence and linearly projected onto... Space. For dimensions of The pipe, where t represents the length of a single spatiotemporal pipe in the time dimension. This represents the size of a single spacetime tube in the height dimension. Represents the dimension of a single spacetime tube in the width dimension; definition , , ,in, This represents the number of tubes that can be divided along the time dimension. This represents the number of tubes that can be divided along the spatial height dimension. This indicates the number of tubes that can be divided along the width dimension of the space. This represents the total number of frames in the input image sequence. Represents the spatial height of a single frame image. The spatial width of a single frame image is represented; feature vectors of the bubble image are extracted from the time dimension, height dimension and width dimension respectively, and spatiotemporal information is fused in the feature extraction process.

[0113] S43. Simultaneously input the feature vector of the flotation foam image into the dynamic spatiotemporal encoder. Process the feature vector of the flotation foam image through the spatial encoder to obtain the spatial feature vector; process the feature vector of the flotation foam image through the time encoder to obtain the temporal feature vector.

[0114] In one feasible implementation, the input to the spatial encoder is a foam sequence feature matrix. After flattening ,in, , Number of channels. Definition This represents the initial feature vector input to the spatial encoder, resulting in... indivual Afterwards, Perform parallel processing and add position encoding We can then conclude that: , represents the encoded feature vector, where pass Spatial features are obtained after one Transformer encoder layer. .

[0115] In one feasible implementation, the time encoder and the spatial encoder share the input. After transposing, we get ,definition This represents the initial feature vector input to the time encoder, resulting in... indivual Parallel processing is then performed, and timing coding is added. We can then conclude that: This represents the encoded feature vector, where, pass Temporal features are obtained after one Transformer encoder layer. .

[0116] Among them, such as Figure 4 The diagram shown is a structural schematic of the Transformer encoder module provided in an embodiment of the present invention; wherein, the feature vector of the flotation foam image is... In the input multi-head attention mechanism, self-attention is calculated using the following formula (4):

[0117] (4)

[0118] in, Represents the query matrix; Represents the key matrix; Represents a value matrix; where, , , ,in This represents the input feature vector; This indicates a query for the change matrix. Represents the key transformation matrix. Represents the value transformation matrix, The dimension of the attention head is represented. The multi-head attention mechanism divides the input feature vector into multiple groups, calculates the concatenated features after attention for each group, and expresses this as shown in the following formula (5):

[0119] (5)

[0120] in, Indicates the output projection matrix; This represents the output of the multi-head attention layer; This represents the first attention head dimension; Indicates the first One dimension of attention; This indicates a splicing operation.

[0121] Among them, based on the splicing after attention, the processing result of the multi-head attention layer is obtained through residual connection and layer normalization, which is expressed by the following formula (6):

[0122] (6)

[0123] in, This represents the output of the multi-head attention layer; Indicates the first Layer input feature vector; Presentation layer normalization operation.

[0124] The processing results of the multi-head attention layer are input into the feedforward network for further processing, as expressed by the following formula (7):

[0125] (7)

[0126] in, This represents the final output of the encoder module; This represents a feedforward neural network.

[0127] S44. Input the spatial feature vector and the temporal feature vector into the dynamic gating unit for concatenation to obtain the concatenated features; perform global average pooling on the concatenated features to obtain the fused feature vector.

[0128] Optionally, the process of inputting the spatial feature vector and the temporal feature vector into the dynamic gating unit for concatenation is represented by the following formulas (8)-(11):

[0129] (8)

[0130] (9)

[0131] (10)

[0132] (11)

[0133] in, Let be the learnable parameters, and represent the spatial gating weight matrix; These are learnable parameters, representing spatial gating bias; Here are the learnable parameters, and here represents the temporal gating weight matrix; These are learnable parameters, representing the timing gate bias; Indicates the first The splicing of spatial and temporal features; This represents the updated spatial characteristics; This represents the updated time series characteristics; Represents the Sigmoid function; This indicates element-wise multiplication; Indicates the first Output weights of the layer dynamic space gating unit; Indicates the first The output weights of the layer dynamic time gating unit and ; express A dimensional real vector space.

[0134] After processing by the dynamic gating unit, the spatial features are finally obtained. and time series characteristics By splicing the two together, we obtain the spliced ​​features. The concatenated features are then subjected to global average pooling to obtain the fused feature vector. .

[0135] S45. Input the fused feature vector into the multilayer perceptron layer, and perform feature extraction and transformation on the fused feature vector layer by layer through multiple neural network layers to obtain a preliminary prediction result of the flotation foam velocity; train the model based on the preliminary prediction result and the constructed loss function to obtain a trained flotation foam velocity prediction model.

[0136] In one feasible implementation, the multilayer perceptron layer includes an input layer, multiple hidden layers, and an output layer. Each hidden layer undergoes a linear transformation using a weighted sum and a bias term, followed by nonlinear processing with an activation function to further obtain complex features from the input data. The input data begins at the input layer and is processed layer by layer. Each layer calculates a weighted sum, applies a bias term, and performs a nonlinear transformation using an activation function to obtain high-level features. After this multi-layered transformation, the data is converted into a prediction result in the output layer, i.e., the foam flow rate result is expressed as... ,in, This represents the fused feature vector; This represents a multilayer perceptron.

[0137] In this embodiment of the invention, the root mean square error is used as the loss function of the flotation foam velocity prediction model, which is expressed by the following formula (12):

[0138] (12)

[0139] in, The model predicts the first The flow rate of each sample; Indicates the first The true flow rate label for each sample; This represents the root mean square error; N represents the total number of samples.

[0140] S5. Input the flotation foam image data to be tested into the trained flotation foam velocity prediction model to obtain the flotation foam velocity prediction results for future times.

[0141] This invention proposes a flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion. By establishing a flotation foam velocity prediction model, it achieves accurate prediction of foam velocity at future moments. This invention fundamentally solves the technical problems of inaccurate velocity measurement and inability to measure velocity due to the complex dynamic characteristics of foam in industrial settings. This invention improves the expressive power of features by constructing a pipe embedding module, extracting multi-dimensional features, and fusing feature information. It also constructs a dynamic spatiotemporal encoder to capture local and global structural features in the foam video stream data, as well as dynamic changes and temporal dependencies between video frames, enhancing the model's understanding of spatial and temporal relationships. The predicted foam velocity values ​​obtained using this invention will provide important decision-making basis for related industrial applications, effectively promoting efficiency improvements and rational resource utilization in the flotation process.

[0142] Figure 5 This is a block diagram of a flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion, provided in an embodiment of the present invention. This device is used in a flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion. (Refer to...) Figure 5 The device includes an acquisition unit 510, a calculation unit 520, a construction unit 530, a training unit 540, and a prediction unit 550. Wherein:

[0143] The acquisition unit 510 is used to acquire raw flotation foam video data; and to preprocess the raw flotation foam video data by taking 10 frames of images as a set to obtain flotation foam image sequence data.

[0144] The computing unit 520 is used to calculate the true value of the foam flow velocity at the current moment using the optical flow method, and uses the true value of the foam flow velocity at the current moment as the training label.

[0145] Building unit 530 is used to build an initial flotation foam velocity prediction model;

[0146] Training unit 540 is used to train the initial flotation foam flow rate prediction model based on the flotation foam image sequence data and the training labels to obtain the trained flotation foam flow rate prediction model.

[0147] The prediction unit 550 is used to input the flotation foam image data to be predicted into the trained flotation foam velocity prediction model to obtain the flotation foam velocity prediction result at future time.

[0148] Optionally, the computing unit 520 is used for:

[0149] The optical flow method is used to calculate the gray value change of each pixel in two consecutive frames. Combined with the known motion of the camera and the foam surface, the motion field is accurately estimated using the Lucas-Kanade method to obtain the foam flow velocity and foam motion direction. Based on the foam flow velocity and foam motion direction, the overall flow velocity of the foam in the current frame image is obtained through vector merging. The overall flow velocity of the foam in the current frame image is used as the training label.

[0150] Optionally, the initial flotation foam velocity prediction model includes:

[0151] The system includes a contrast-adaptive histogram equalization module, a pipeline embedding module, a dynamic spatiotemporal encoder, and a multilayer perceptron layer.

[0152] Optionally, the limited contrast adaptive histogram equalization module is used to enhance the input original image sequence;

[0153] The pipeline embedding module is used to extract feature vectors from the input image sequence in multiple dimensions, including time, height, and width, and to fuse temporal and spatial information.

[0154] The multilayer perceptron layer is used to extract and transform input features layer by layer.

[0155] Optionally, the dynamic spatiotemporal encoder includes: a spatial encoder, a time encoder, and a dynamic gating unit;

[0156] The spatial encoder is used to extract spatial features from image data.

[0157] The time encoder is used to extract temporal features from image data;

[0158] The dynamic gating unit is used to calculate the correlation between spatial features and temporal features in the image data.

[0159] Optionally, the training unit 540 is used for:

[0160] The flotation foam image sequence data is input into the contrast-limited adaptive histogram equalization module. The flotation foam image is divided into blocks to obtain multiple local sub-regions. Histogram equalization is performed on the local sub-regions to obtain enhanced image data.

[0161] The enhanced image data is input into the pipeline embedding module, and feature extraction is performed on the enhanced image data from the time dimension, height dimension and width dimension to obtain the flotation foam image feature vector;

[0162] The feature vector of the flotation foam image is simultaneously input into the dynamic spatiotemporal encoder. The spatial encoder processes the feature vector of the flotation foam image to obtain the spatial feature vector. The temporal encoder processes the feature vector of the flotation foam image to obtain the temporal feature vector.

[0163] Spatial feature vectors and temporal feature vectors are input into a dynamic gating unit for concatenation to obtain concatenated features; global average pooling is then performed on the concatenated features to obtain a fused feature vector.

[0164] The fused feature vector is input into a multilayer perceptron layer, and the fused feature vector is extracted and transformed layer by layer through multiple neural network layers to obtain a preliminary prediction result of the flotation foam velocity. Based on the preliminary prediction result and the constructed loss function, the model is trained to obtain a trained flotation foam velocity prediction model.

[0165] Optionally, the process of inputting the spatial feature vector and the temporal feature vector into the dynamic gating unit for concatenation is represented by the following formulas (1)-(4):

[0166] (1)

[0167] (2)

[0168] (3)

[0169] (4)

[0170] in, Let be the learnable parameters, and represent the spatial gating weight matrix; These are learnable parameters, representing spatial gating bias; Here are the learnable parameters, and here represents the temporal gating weight matrix; These are learnable parameters, representing the timing gate bias; Indicates the first The splicing of spatial and temporal features; This represents the updated spatial characteristics; This represents the updated time series characteristics; Represents the Sigmoid function; This indicates element-wise multiplication; Indicates the first Output weights of the layer dynamic space gating unit; Indicates the first The output weights of the layer dynamic time gating unit and ; express A dimensional real vector space.

[0171] This invention proposes a flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion. By establishing a flotation foam velocity prediction model, it achieves accurate prediction of foam velocity at future moments. This invention fundamentally solves the technical problems of inaccurate velocity measurement and inability to measure velocity due to the complex dynamic characteristics of foam in industrial settings. This invention improves the expressive power of features by constructing a pipe embedding module, extracting multi-dimensional features, and fusing feature information. It also constructs a dynamic spatiotemporal encoder to capture local and global structural features in the foam video stream data, as well as dynamic changes and temporal dependencies between video frames, enhancing the model's understanding of spatial and temporal relationships. The predicted foam velocity values ​​obtained using this invention will provide important decision-making basis for related industrial applications, effectively promoting efficiency improvements and rational resource utilization in the flotation process.

[0172] Figure 6 This is a schematic diagram of a flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion may include the above-mentioned... Figure 5 The flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion is shown. Optionally, the flotation foam velocity prediction device 610 based on dynamic spatiotemporal attention fusion may include a first processor 2001.

[0173] Optionally, the flotation foam velocity prediction device 610 based on dynamic spatiotemporal attention fusion may also include a memory 2002 and a transceiver 2003.

[0174] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0175] The following is combined with Figure 6 The components of the flotation foam velocity prediction device 610 based on dynamic spatiotemporal attention fusion are described in detail below:

[0176] The first processor 2001 is the control center of the flotation foam flow rate prediction device 610 based on dynamic spatiotemporal attention fusion. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0177] Optionally, the first processor 2001 can perform various functions of the flotation foam velocity prediction device 610 based on dynamic spatiotemporal attention fusion by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0178] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 6 CPU0 and CPU1 are shown in the diagram.

[0179] In a specific implementation, as one example, the flotation foam velocity prediction device 610 based on dynamic spatiotemporal attention fusion may also include multiple processors, for example... Figure 6 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0180] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0181] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the flotation foam flow rate prediction device 610 based on dynamic spatiotemporal attention fusion. Figure 6 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0182] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0183] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 6 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0184] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the flotation foam velocity prediction device 610 based on dynamic spatiotemporal attention fusion. Figure 6 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0185] It should be noted that, Figure 6 The structure of the flotation foam velocity prediction device 610 based on dynamic spatiotemporal attention fusion shown in the figure does not constitute a limitation on the router. Actual flotation foam velocity prediction devices based on dynamic spatiotemporal attention fusion may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0186] Furthermore, the technical effect of the flotation foam velocity prediction device 610 based on dynamic spatiotemporal attention fusion can be referred to the technical effect of the flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion described in the above method embodiments, and will not be repeated here.

[0187] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.

[0188] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0189] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0190] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0191] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0192] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0193] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0194] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0195] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0196] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0197] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0198] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0199] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for predicting flotation foam velocity based on dynamic spatiotemporal attention fusion, characterized in that, The method includes: S1. Obtain raw flotation foam video data; preprocess the raw flotation foam video data into sets of 10 frames each to obtain flotation foam image sequence data; S2. Calculate the true value of the foam velocity at the current moment using the optical flow method, and use the true value of the foam velocity at the current moment as the training label. S3. Construct an initial flotation foam velocity prediction model; The initial flotation foam velocity prediction model includes: Limit contrast adaptive histogram equalization module, pipeline embedding module, dynamic spatiotemporal encoder and multilayer perceptron layer; S4. Based on the flotation foam image sequence data and the training labels, train the initial flotation foam velocity prediction model to obtain the trained flotation foam velocity prediction model. Specifically, S4 involves training the initial flotation foam flow rate prediction model based on the flotation foam image sequence dataset and the training labels to obtain a trained flotation foam flow rate prediction model, including: S41. Input the flotation foam image sequence data into the limited contrast adaptive histogram equalization module, and obtain multiple local sub-regions by dividing the flotation foam image into blocks; perform histogram equalization processing on the local sub-regions to obtain enhanced image data; S42. Input the enhanced image data into the pipeline embedding module, and extract features from the enhanced image data from the time dimension, height dimension and width dimension to obtain the flotation foam image feature vector; S43. Simultaneously input the feature vector of the flotation foam image into the dynamic spatiotemporal encoder. Process the feature vector of the flotation foam image through the spatial encoder to obtain the spatial feature vector; process the feature vector of the flotation foam image through the time encoder to obtain the temporal feature vector. S44. Input the spatial feature vector and the temporal feature vector into the dynamic gating unit for concatenation to obtain the concatenated features; perform global average pooling on the concatenated features to obtain the fused feature vector. S45. Input the fused feature vector into the multilayer perceptron layer, and perform feature extraction and transformation layer by layer through multiple neural network layers to obtain a preliminary prediction result of the flotation foam velocity; train the model based on the preliminary prediction result and the constructed loss function to obtain a trained flotation foam velocity prediction model. S5. Input the flotation foam image data to be predicted into the trained flotation foam velocity prediction model to obtain the flotation foam velocity prediction results for future times.

2. The flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion according to claim 1, characterized in that, The S2 method uses optical flow to calculate the true value of the foam flow velocity at the current moment, and uses the true value of the foam flow velocity at the current moment as the training label, including: The optical flow method is used to calculate the gray value change of each pixel in two consecutive frames. Combined with the known motion of the camera and the foam surface, the motion field is accurately estimated using the Lucas-Kanade method to obtain the foam flow velocity and foam motion direction. Based on the foam flow velocity and foam motion direction, the overall flow velocity of the foam in the current frame image is obtained through vector merging. The overall flow velocity of the foam in the current frame image is used as the training label.

3. The flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion according to claim 1, characterized in that, The limited contrast adaptive histogram equalization module is used to enhance the input original image sequence. The pipeline embedding module is used to extract feature vectors from the input image sequence in multiple dimensions, including time, height, and width, and to fuse temporal and spatial information. The multilayer perceptron layer is used to extract and transform input features layer by layer.

4. The flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion according to claim 1, characterized in that, The dynamic spatiotemporal encoder includes: a spatial encoder, a time encoder, and a dynamic gating unit; The spatial encoder is used to extract spatial features from image data. The time encoder is used to extract temporal features from image data; The dynamic gating unit is used to calculate the correlation between spatial features and temporal features in the image data.

5. The flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion according to claim 1, characterized in that, The process of inputting the spatial feature vector and the temporal feature vector into the dynamic gating unit for concatenation is represented by the following formulas (1)-(4): （1） （2） （3） （4） in, Let be the learnable parameters, and represent the spatial gating weight matrix; These are learnable parameters, representing spatial gating bias; Here are the learnable parameters, and here represents the temporal gating weight matrix; These are learnable parameters, representing the timing gate bias; Indicates the first The splicing of spatial and temporal features; This represents the updated spatial characteristics; This represents the updated time series characteristics; Represents the Sigmoid function; This indicates element-wise multiplication; Indicates the first Output weights of the layer dynamic space gating unit; Indicates the first The output weights of the layer dynamic time gating unit and ; express A real vector space.

6. A flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion, wherein the flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion is used to implement the flotation foam velocity prediction method based on dynamic spatiotemporal attention fusion as described in any one of claims 1-5, characterized in that, The device includes: The acquisition unit is used to acquire raw flotation foam video data; the raw flotation foam video data is preprocessed by taking 10 frames of images as a set to obtain flotation foam image sequence data; The computing unit is used to calculate the true value of the foam flow velocity at the current moment using the optical flow method, and uses the true value of the foam flow velocity at the current moment as the training label. Building blocks are used to construct an initial flotation foam velocity prediction model; The training unit is used to train the initial flotation foam flow rate prediction model based on the flotation foam image sequence data and the training labels, so as to obtain the trained flotation foam flow rate prediction model. The prediction unit is used to input the flotation foam image data to be predicted into the trained flotation foam velocity prediction model to obtain the flotation foam velocity prediction result at future time.

7. A flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion, characterized in that, The flotation foam velocity prediction device based on dynamic spatiotemporal attention fusion includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 5.

Images