A foam image representation learning method based on learnable prototype aggregation and visual pattern association mining

Abstract

The application discloses a kind of based on learnable prototype aggregation and visual pattern association mining froth image representation learning method, belong to mineral flotation process monitoring technical field.The method first extracts the local deep feature of froth image using convolutional neural network;Then a set of learnable visual pattern prototypes is constructed, and the soft assignment matrix between each local feature and prototype is calculated;Then the local features are weighted and aggregated according to the soft assignment weight, to form a typical visual pattern representation;Again, according to the response distribution similarity of different visual patterns in image space, a pattern relationship graph is constructed, and the co-occurrence relationship and structural association between patterns are mined through graph convolution network;Finally, the enhanced representation of the whole image is obtained by averaging aggregation, and joint optimization is carried out by combining classification loss and prototype diversity loss.The application effectively solves the difficulty problem of representation caused by local adhesion, boundary blur and insufficient pattern association in froth image, and improves the accuracy and robustness of flotation condition recognition.

Description

Technical Field

[0001] This invention relates to the field of mineral flotation process monitoring technology, specifically to a foam image representation learning method based on learnable prototype aggregation and visual pattern association mining. Background Technology

[0002] Mineral processing is a crucial step in mineral resource processing. Its purpose is to separate target minerals from gangue minerals based on the differences between them, thereby improving the utilization rate of mineral resources. Foam flotation, due to its high separation efficiency and wide applicability to various minerals, has become one of the most widely used mineral processing methods.

[0003] The basic principle of froth flotation is to utilize the differences in the physicochemical properties of different mineral surfaces. Under the action of flotation reagents, target mineral particles adhere to the surface of air bubbles and float with them, thus separating the target mineral from gangue minerals. First, the collected raw ore undergoes crushing and ball milling processes to form mineral particles suitable for flotation. Then, the mineral particles are mixed with water in a mixer to form a slurry. Flotation reagents such as frothers, collectors, and activators are added to the slurry according to process requirements and thoroughly mixed in a stirred tank. The frother promotes bubble formation and dispersion, while the collectors and activators enhance the hydrophobicity of the target mineral surface. Finally, the mixed slurry is fed into a flotation cell, where a rotating impeller injects air to form bubbles. After the bubbles collide with the mineral particles, the hydrophobic target minerals adhere to the bubble surface and rise with the bubbles to the slurry, forming a froth layer, which is then scraped off by a scraper, achieving the collection of the target mineral. The hydrophilic minerals remain in the slurry and are eventually discharged from the flotation cell, forming tailings.

[0004] In recent years, machine vision technology has been gradually applied to the monitoring of the flotation process. This method uses camera equipment to acquire real-time images of the flotation cell and uses image processing algorithms to analyze the visual features of the flotation process, such as color, size, and texture, thereby achieving automatic monitoring of the flotation process.

[0005] Foam images commonly exhibit phenomena such as foam adhesion and blurred boundaries, making it difficult to accurately delineate the morphological boundaries of local regions. Traditional deterministic region segmentation methods typically rely on explicit boundary information. When foam edges are unclear or multiple foam regions are adhered to each other, oversegmentation or undersegmentation problems easily occur, thus affecting the reliability of subsequent feature extraction. In actual foam images, a particular flotation condition is often not determined by a single foam visual feature, but by the co-occurrence of multiple local visual patterns forming a specific combination. For example, under different flotation conditions, local features such as foam morphology, texture roughness, and adhered regions often exhibit different co-occurrence patterns and structural relationships.

[0006] To address the shortcomings in visual representation caused by foam adhesion, blurred boundaries, and mixed local states in foam images, as well as the insufficient modeling of the correlations between different local visual patterns, this invention proposes a flotation condition identification method based on learnable prototype aggregation and visual pattern association mining. This method can summarize and represent local visual features in foam images using learnable typical visual pattern prototypes, and utilize a soft allocation mechanism to characterize the degree of correlation between local regions and different visual patterns. Simultaneously, based on the visual pattern allocation results, a visual pattern association graph is constructed to further explore the co-occurrence relationships and structural associations between different visual patterns, thereby obtaining a more discriminative foam image representation and improving the accuracy and stability of flotation condition identification. Summary of the Invention

[0007] To overcome the above-mentioned technical problems, this invention provides a foam image representation learning method based on learnable prototype aggregation and visual pattern association mining.

[0008] To achieve the above technology, the following steps are included: The preprocessed flotation process foam image is input into the foam image representation learning model to obtain an enhanced foam image representation; the foam image representation learning model executes the following sequentially: The preprocessed foam image is encoded using a convolutional neural network. Construct a prototype of a learnable visual pattern that is updated along with the network parameters during training; Calculate the similarity between the encoding operation results and the learnable visual pattern prototype to construct a soft assignment matrix; A weighted aggregation operation is performed on the soft assignment matrix and the encoding operation results to obtain the visual pattern representation matrix, and the pattern relation matrix under different modes is calculated. The pattern relation matrix is ​​then constructed by adding self-loops to the pattern relation matrix and performing symmetric normalization. The visual pattern relation matrix is ​​input into the graph convolutional network to obtain a global enhanced representation of the entire bubble image.

[0009] Furthermore, the encoding operation includes: The preprocessed foam image is input into the foam image feature encoding module to obtain the feature map; The feature map is expanded into a set of local feature vectors according to its spatial location.

[0010] Furthermore, the constructed learnable visual pattern prototype, which is updated along with the network parameters during training, is represented as follows: , ∈ ,in, Indicates the number of visual pattern prototypes. Indicates the feature dimension; The learnable visual pattern prototype is used to characterize the local appearance state of the feature map of the bubble image. The feature dimension It has the same dimensions as the feature map.

[0011] Furthermore, the step of calculating the similarity between the encoding operation result and the learnable visual pattern prototype to construct the soft assignment matrix includes: Based on the constructed learnable visual pattern prototypes, the similarity between each local feature and each visual pattern prototype is calculated. ,all Construct a soft allocation matrix, denoted as ,in, This represents the total number of local features.

[0012] Furthermore, the step of performing a weighted aggregation operation on the soft allocation matrix and the encoding operation result to obtain the visual pattern representation matrix includes: In obtaining the soft allocation matrix After that, regarding the first For each visual pattern prototype, the soft-assigned weights of all local features corresponding to the visual pattern prototypes are calculated and weighted summed to obtain the visual pattern representation. The visual pattern representation matrix is ​​obtained by combining and splicing all visual pattern representations, and is represented as follows: , ∈ .

[0013] Furthermore, the step of calculating the pattern relation matrix under different modes includes: Calculate co-occurrence relationships under different patterns The expression is as follows: In the formula, Indicates the first Soft-assigned column vectors under each visual mode; Indicates the first Soft-assigned column vectors under each visual mode; All co-occurrence relationships form a pattern relation matrix, represented as follows: , .

[0014] Furthermore, the expression for constructing the visual pattern relation matrix by adding self-loops to the pattern relation matrix and performing symmetric normalization is as follows: In the formula, This represents the visual pattern relation matrix of the final input graph convolutional network; Let be a degree matrix, and its diagonal elements be... ,in, For Kronecker delta, when The value is 1 if the condition is met, otherwise it is 0. Represents the identity matrix.

[0015] Furthermore, in the step of inputting the visual pattern relation matrix into the graph convolutional network to obtain the global enhanced representation of the entire bubble image, the graph convolutional network is set as a two-layer graph convolutional structure.

[0016] Furthermore, the method of inputting the visual pattern relation matrix into the graph convolutional network to obtain the global enhanced representation of the entire bubble image is as follows: Visual pattern representation matrix Relationship matrix with normalized visual patterns Input a two-layer convolutional network, where the expression for the first layer's graph convolution is as follows: The expression for the second-layer graph convolution is as follows: In the formula, and These are the output features of the first and second layers; It is a non-linear activation function; and The matrix in the middle represents the learnable parameter matrix of the first and second layer graph convolutions; After graph convolutional association mining, the model obtains an updated visual pattern representation. The average aggregation method is used to aggregate the representations of all visual patterns, as shown in the following expression: In the formula, This represents the th element after graph convolution update. Visual pattern representation, Indicates the number of visual pattern prototypes.

[0017] Furthermore, the method also includes: The bubble image representation learning model is trained using the total loss; The expression for the total loss is as follows: In the formula, Represents the classification loss function; This indicates the loss of prototype diversity. To adjust the balance between the weights of classification loss and prototype diversity loss; the expression for the classification loss function is as follows: In the formula, Indicates the total number of operating condition categories. An index representing the total number of operating condition categories; Indicates the first Real labels for similar working conditions; Output results for the model. ,in and These represent the weight matrix and bias term of the classifier, respectively. The expression for prototype diversity loss is as follows: In the formula, Denotes the Frobenius norm; Let be the Gram matrix of the prototype matrix, where This represents a learnable visual pattern prototype. Represents the identity matrix.

[0018] Beneficial effects of the present invention 1. This invention introduces learnable visual pattern prototypes to model frequently occurring and meaningful local states in foam images. Each visual pattern prototype can automatically learn a typical visual pattern during training, and group local regions scattered in different spatial locations but with similar morphological features into the same typical visual pattern. In this way, the model no longer relies solely on local convolutional features or single whole-image features, but can use typical visual patterns as intermediate representation units to represent key local states in foam images, thereby improving the representation capability and interpretability of fine-grained features in foam images.

[0019] 2. To address the common problems in actual flotation processes, such as foam sticking together, blurred boundaries, and mixed states in local areas, this invention employs a learnable prototype soft assignment mechanism. This mechanism calculates the membership degree of each local feature to different visual pattern prototypes, resulting in a soft assignment matrix. Compared to the traditional hard assignment method, this invention's soft assignment reduces the impact of the assumption of a single local region on subsequent representations, and reduces information loss caused by blurred boundaries and local mixing. This provides a more reliable foundation for constructing visual pattern relationship graphs and obtaining enhanced representations of the entire foam image.

[0020] 3. This invention constructs a visual pattern relationship graph based on the similarity of response distributions of different visual patterns in the image space within the soft allocation matrix, and utilizes a graph convolutional network to perform information propagation and feature updates on this graph. Through this process, the model can not only obtain representations of individual visual patterns, but also mine co-occurrence relationships and structural associations between different visual patterns, thereby obtaining a more comprehensive representation of foam images. Attached Figure Description

[0021] Figure 1 This is a flowchart of the steps of the present invention; Figure 2 This is a network structure diagram of the foam image representation learning method based on learnable prototype aggregation and visual pattern association mining of the present invention. Detailed Implementation

[0022] The present invention will be further described in detail below with reference to specific embodiments.

[0023] See Figure 1 and Figure 2 A foam image representation learning method based on learnable prototype aggregation and visual pattern association mining includes the following steps: S1. Foam Image Data Acquisition and Preprocessing: First, by installing an industrial camera and auxiliary light source directly above the flotation cell, foam images during the flotation process are acquired in real time. The acquired foam images are used to reflect the appearance characteristics of the foam layer, such as color, size, and texture, under different flotation conditions.

[0024] Subsequently, on-site operators categorized the acquired flotation images according to the actual flotation conditions, obtaining the raw flotation flotation image dataset. The flotation image dataset then underwent preprocessing, specifically including the following steps: The original bubble images are cropped: during the training phase, random cropping is used to obtain 224×224×3 images as network input; during the validation and testing phases, center cropping or fixed cropping is used to obtain images of the same size. To enhance the model's adaptability to fluctuations in ambient lighting, data augmentation operations such as horizontal flipping, vertical flipping, and Gaussian blur are performed on the images during the training phase. The cropped image pixels are normalized to meet the requirements of subsequent model inputs; Finally, after the above preprocessing, the foam image samples are divided into training set, validation set and test set according to the ratio of 60%, 20% and 20% for subsequent model training.

[0025] S2. Constructing a foam image representation learning model based on learnable prototype aggregation and visual pattern association mining: This model consists of a foam image feature encoding module, a learnable prototype soft assignment module, a visual pattern aggregation module, and a visual pattern association mining module. The foam image feature encoding module extracts local deep features with spatial distribution information from the input foam image. The learnable prototype soft assignment module constructs a set of trainable visual pattern prototypes and calculates the membership degree of each local feature to each visual pattern prototype. The visual pattern aggregation module weights and aggregates the local features according to the soft assignment results, forming several visual pattern representations. The visual pattern association mining module constructs a visual pattern relationship graph based on the similarity of the response distribution of different visual patterns in the image space, and performs information propagation and feature updating of the visual pattern representations under the constraints of this relationship matrix to mine the co-occurrence relationships and structural associations between different visual patterns, enhancing the perception ability of each visual pattern representation to other related patterns. Finally, the updated visual pattern representations are averaged and aggregated to obtain an enhanced representation of the entire foam image.

[0026] Specifically, the visual pattern in this invention refers to a typical local representation pattern in the feature space represented by a learnable prototype after the foam image is feature-encoded, rather than a fixed region obtained by manually dividing or traditionally segmenting the original image. The visual pattern is used to characterize the response features of recurring local appearance states in the foam image in the feature space, including but not limited to fine foam, coarse foam, foam adhesion areas, and transition areas between different states.

[0027] Furthermore, S2 specifically includes the following steps: S2.1, Foam Image Feature Encoding: The first few stages of ResNet50 are used as the foam image feature encoding module to extract the deep features of the foam image after S1 preprocessing. and These represent the height and width of the input image, respectively. The specific steps are as follows: Image Input foam image feature encoding module The resulting embedded feature map is represented as: ,in, and These represent the height and width of the feature map, respectively. Representing feature dimension, This represents the feature map obtained after feature encoding of a foam image. Each spatial location in the feature map corresponds to a local region in the original foam image. This local region may contain local visual states such as small foam, large foam, foam adhesion areas, and transition areas between different states. That is, using ResNet50, with an input image size of 224×224×3, a feature map of size 14×14×1024 is obtained after feature encoding. The feature map is then... Local feature sets are expanded according to spatial location and consist of local feature vectors: ,in, Indicates the first Local feature vectors, ; Represents the total number of local features. The specific model architecture of the foam image feature encoding module is shown in Table 1: Table 1: Architecture of Foam Image Feature Encoding Module S2.2 Constructing a learnable visual pattern prototype: This invention is based on the obtained local feature set Build a set of learnable visual pattern prototypes Modeling is performed on typical local states that occur frequently in foam images and have significance for identifying working conditions. Indicates the number of visual pattern prototypes.

[0028] The visual pattern prototypes in this invention are neither fixed templates pre-set by humans nor fixed regions obtained by traditional image segmentation methods. Instead, they are trainable vectors that are updated together with the network parameters during model training. Each visual pattern prototype is used to represent a typical local appearance state (such as foam size, texture roughness, degree of adhesion, etc.) in the feature map of a foam image.

[0029] S2.3, Soft Assignment of Local Features and Visual Pattern Prototypes: In actual flotation processes, foam images commonly exhibit problems such as foam adhesion, blurred boundaries, and mixed states in local areas, making accurate segmentation of local foam regions difficult. Traditional methods based on manual segmentation or deterministic region division typically require clear foam boundaries and tend to directly classify a region into a single category. When foam edges are blurred, multiple foams are adhered to each other, or a local region simultaneously possesses multiple foam state characteristics, hard assignment methods often fail to accurately describe the complex visual attributes of the region. They easily force continuously transitioning or mixed-state local regions into a single visual mode, thereby weakening the multi-modal information contained in the local region and affecting the reliability of subsequent visual mode aggregation. To address this issue, this invention employs a learnable prototype soft assignment mechanism, calculating the membership degree of each local feature to different visual mode prototypes to obtain a soft assignment matrix. The specific steps are as follows: First, based on the constructed learnable visual pattern prototype P The similarity between each local feature and each visual pattern prototype is calculated to obtain the degree of membership of each local region to different visual pattern prototypes. The calculation formula is as follows: In the formula, Indicates the first The local eigenvectors of the nth local feature vector pair The degree of membership of a visual pattern prototype; Indicates the first A visual pattern prototype; Indicates the first A visual pattern prototype; Hyperparameters that control the smoothness of soft assignment; Next, the membership degrees of all local feature vectors to the visual pattern prototype are reassembled into a soft assignment matrix, represented as: ; S2.4 Visual Pattern Aggregation Based on Soft Allocation Matrix: After obtaining the soft allocation matrix... Then, based on the membership weights of local features to each visual pattern prototype, the local feature set is... We perform weighted aggregation to obtain a visual pattern representation, that is, for the first... Given a visual pattern prototype, all local features are weighted and summed according to their soft-assigned weights on that prototype to obtain the representation of the visual pattern in the current bubble image. The calculation formula is as follows: In the formula, Indicates the first A representation of a visual pattern prototype; The visual pattern representation matrix is ​​obtained by combining all visual pattern representations: ; Through this step, the original set of local features is transformed into a set of visual pattern representations; each visual pattern representation no longer corresponds to a single fixed position in the image, but is formed by the weighted sum of multiple local regions in the whole image that are related to the visual pattern; therefore, the visual pattern aggregation module can summarize similar local visual information scattered in different spatial locations of the foam image, thereby enhancing the representation ability of typical local states of the foam image.

[0030] S2.5, Construction of Visual Pattern Relationship Graph: Based on Visual Pattern Representation Matrix Furthermore, a visual pattern relationship diagram is constructed to describe the co-occurrence relationships and structural associations among different visual patterns.

[0031] Specifically: Record The Listed as , which means The local feature vector of the i-th spatial location belongs to the i-th Membership distribution of visual pattern prototypes Reserved The activation strength information of two visual pattern prototypes at different spatial locations is analyzed. If the allocation of two visual patterns at various spatial locations is similar, it indicates that they are highly correlated in the current image. Based on this, further analysis using the allocation matrix... Constructing a visual pattern relationship diagram, then the first... The visual pattern and the first The co-occurrence relationship between visual patterns is defined as follows: In the formula, Indicates the first Soft-assigned column vectors under each visual mode; Indicates the first Soft-assigned column vectors under each visual mode; All co-occurrence relationships form a pattern relation matrix, represented as: This matrix reflects the degree of co-activation of different visual modes in the current bubble image. To ensure that each visual mode retains its own information during subsequent graph convolution propagation, a self-loop is added to the mode relation matrix and symmetric normalization is performed. The calculation formula is as follows: In the formula, This represents the visual pattern relation matrix of the final input graph convolutional network; Let be a degree matrix, and its diagonal elements be... ,in, For Kronecker delta, when The value is 1 if the condition is met, otherwise it is 0. Represents the identity matrix; S2.6 Visual Pattern Association Mining and Bubble Image Enhancement Representation Acquisition: The visual pattern representation matrix... Relationship matrix with normalized visual patterns The input visual pattern association mining module performs feature mining and feature updating on the visual pattern relationship graph. A two-layer convolutional network is used to propagate information on the pattern relationship graph. The calculation processes of the first and second layer graph convolutions are as follows: First layer graph convolution: Second layer graph convolution: In the formula, and These are the output features of the first and second layers; It is a non-linear activation function; and The matrix in the middle represents the learnable parameter matrix of the first and second layer graph convolutions; After graph convolutional association mining, the model obtains an updated visual pattern representation. This representation integrates co-occurrence relationships and structural associations between different visual modes, reflecting the combination relationships between local visual modes in the foam image. Since the ultimate goal of this invention is to obtain a unified representation of the entire foam image, rather than a node representation of a single visual mode, it is necessary to aggregate all updated visual mode representations to obtain an enhanced representation of the entire foam image. This invention uses an average aggregation method to aggregate the representations of all visual patterns, as shown in the following expression: In the formula, This represents the th element after graph convolution update. Visual pattern representation.

[0032] S3. Model Training and Joint Optimization: Enhanced representation based on S7 output This step introduces a working condition category supervision signal and a prototype diversity regularization constraint, and jointly optimizes all module parameters through backpropagation, enabling the model to learn typical visual patterns and associated structures with discriminative and interpretable characteristics. During the model training phase, to ensure the obtained enhanced representations... It has the ability to distinguish working conditions, and a working condition category supervision signal is introduced during the training phase; specifically, it will enhance the representation. Input a classification prediction layer for training, and obtain the predicted probabilities of the work condition categories: ,in and These represent the weight matrix and bias term of the classifier, respectively. To determine the model's output, the classification loss function is defined as follows: In the formula, Indicates the total number of operating condition categories. An index representing the total number of operating condition categories; Indicates the first Real labels for similar working conditions; To reduce redundancy between different prototypes (typical visual patterns) and enhance the discriminative power of visual patterns, a prototype diversity loss is further introduced, expressed as follows: In the formula, Denotes the Frobenius norm; The Gram matrix of the prototype matrix is ​​used to characterize the pairwise correlation between prototypes. This loss is close to the identity matrix, which keeps the correlation between different prototypes as low as possible, thereby encouraging different prototypes to learn mutually distinguishable prototypes and avoiding multiple prototypes from converging to similar patterns. In summary, the total loss during model training is defined as: ,in, A balance coefficient is used to adjust the weights between classification loss and prototype diversity loss.

[0033] To address the challenges of visual representation in foam images caused by foam adhesion, blurred boundaries, and mixed local states, as well as the insufficient utilization of the correlations between different local visual patterns, this invention summarizes and represents frequently occurring local visual features with discriminative significance in foam images using learnable prototypes. This forms typical visual patterns, and a soft assignment mechanism characterizes the similarity between local regions and different typical visual patterns. Simultaneously, a visual pattern relationship graph is constructed based on the visual pattern assignment results to further explore the co-occurrence relationships and structural associations between different typical visual patterns, thereby obtaining a more discriminative foam image representation. This method can learn typical visual patterns with discriminative significance while more meticulously characterizing the membership relationships of local regions to different visual patterns, and further explore the co-occurrence relationships and structural associations between different visual patterns, improving the efficiency and accuracy of mineral flotation condition identification.

[0034] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A foam image representation learning method based on learnable prototype aggregation and visual pattern association mining, characterized in that, Includes the following steps: The preprocessed flotation process foam image is input into the foam image representation learning model to obtain an enhanced foam image representation; the foam image representation learning model executes the following sequentially: The preprocessed foam image is encoded using a convolutional neural network. Construct a prototype of a learnable visual pattern that is updated along with the network parameters during training; Calculate the similarity between the encoding operation results and the learnable visual pattern prototype to construct a soft assignment matrix; A weighted aggregation operation is performed on the soft assignment matrix and the encoding operation results to obtain the visual pattern representation matrix, and the pattern relation matrix under different modes is calculated. The pattern relation matrix is ​​then constructed by adding self-loops to the pattern relation matrix and performing symmetric normalization. The visual pattern relation matrix is ​​input into the graph convolutional network to obtain a global enhanced representation of the entire bubble image.

2. The foam image representation learning method based on learnable prototype aggregation and visual pattern association mining according to claim 1, characterized in that: The encoding operation includes: The preprocessed foam image is input into the foam image feature encoding module to obtain the feature map; The feature map is expanded into a set of local feature vectors according to its spatial location.

3. The foam image representation learning method based on learnable prototype aggregation and visual pattern association mining according to claim 2, characterized in that: The constructed learnable visual pattern prototype, which is updated along with the network parameters during training, is represented as follows: , ∈ ,in, Indicates the number of visual pattern prototypes. Indicates the feature dimension; The learnable visual pattern prototype is used to characterize the local appearance state of the feature map of the bubble image. The feature dimension It has the same dimensions as the feature map.

4. The foam image representation learning method based on learnable prototype aggregation and visual pattern association mining according to claim 3, characterized in that: The step of calculating the similarity between the encoding operation result and the learnable visual pattern prototype to construct the soft assignment matrix includes: Based on the constructed learnable visual pattern prototypes, the similarity between each local feature and each visual pattern prototype is calculated. ,all Construct a soft allocation matrix, denoted as ,in, This represents the total number of local features.

5. The foam image representation learning method based on learnable prototype aggregation and visual pattern association mining according to claim 4, characterized in that: The step of performing a weighted aggregation operation on the soft-assignment matrix to obtain the visual pattern representation matrix includes: In obtaining the soft allocation matrix After that, regarding the first For each visual pattern prototype, the soft-assigned weights of all local features corresponding to the visual pattern prototypes are calculated and weighted summed to obtain the visual pattern representation. The visual pattern representation matrix is ​​obtained by combining and splicing all visual pattern representations, and is represented as follows: , ∈ .

6. The foam image representation learning method based on learnable prototype aggregation and visual pattern association mining according to claim 5, characterized in that: The steps for calculating the pattern relation matrix under different patterns include: Calculate co-occurrence relationships under different patterns The expression is as follows: In the formula, Indicates the first Soft-assigned column vectors under each visual mode; Indicates the first Soft-assigned column vectors under each visual mode; All co-occurrence relationships form a pattern relation matrix, represented as follows: , .

7. The foam image representation learning method based on learnable prototype aggregation and visual pattern association mining according to claim 6, characterized in that: The expression for constructing the visual pattern relation matrix by adding self-loops to the pattern relation matrix and performing symmetric normalization is as follows: In the formula, This represents the visual pattern relation matrix of the final input graph convolutional network; Let be a degree matrix, and its diagonal elements be... ,in, For Kronecker delta, when The value is 1 if the condition is met, otherwise it is 0. Represents the identity matrix.

8. The foam image representation learning method based on learnable prototype aggregation and visual pattern association mining according to claim 7, characterized in that: In the process of inputting the visual pattern relation matrix into a graph convolutional network to obtain a global enhanced representation of the entire bubble image, the graph convolutional network is set as a two-layer graph convolutional structure.

9. The foam image representation learning method based on learnable prototype aggregation and visual pattern association mining according to claim 8, characterized in that: The method of inputting the visual pattern relation matrix into the graph convolutional network to obtain the global enhanced representation of the entire foam image is as follows: Visual pattern representation matrix Relationship matrix with normalized visual patterns Input a two-layer convolutional network, where the expression for the first layer's graph convolution is as follows: The expression for the second-layer graph convolution is as follows: In the formula, and These are the output features of the first and second layers; It is a non-linear activation function; and The matrix in the middle represents the learnable parameter matrix of the first and second layer graph convolutions; After graph convolutional association mining, the model obtains an updated visual pattern representation. The average aggregation method is used to aggregate the representations of all visual patterns, as shown in the following expression: In the formula, This represents the th element after graph convolution update. Visual pattern representation, Indicates the number of visual pattern prototypes.

10. The foam image representation learning method based on learnable prototype aggregation and visual pattern association mining according to claim 1, characterized in that: The method further includes: The bubble image representation learning model is trained using the total loss; The expression for the total loss is as follows: In the formula, Represents the classification loss function; This indicates the loss of prototype diversity. To adjust the balance between the weights of classification loss and prototype diversity loss; the expression for the classification loss function is as follows: In the formula, Indicates the total number of operating condition categories. An index representing the total number of operating condition categories; Indicates the first Real labels for similar working conditions; Output results for the model. ,in and These represent the weight matrix and bias term of the classifier, respectively. The expression for prototype diversity loss is as follows: In the formula, Denotes the Frobenius norm; Let be the Gram matrix of the prototype matrix, where This represents a learnable visual pattern prototype. Represents the identity matrix.

Images