Bubble recognition method and device based on deep learning and storage medium

Abstract

The embodiment of the invention provides a bubble recognition method and device based on deep learning and a storage medium, and belongs to the field of image analysis of microbubble groups in the gas-liquid reaction process, and the method comprises the steps: collecting original pictures of bubbles; enhancing the features of the bubbles in the original picture to generate a preprocessed picture; distributing a training data set and a test data set based on the preprocessed picture; constructing a deep learning convolutional neural network model; training the deep learning convolutional neural network model based on the training data set and the test data set; and based on the trained deep learning convolutional neural network model, identifying the bubbles. According to the method, the deep learning convolutional neural network model is trained by using the original bubble pictures shot by the non-intrusive method, the trained deep learning convolutional neural network model can greatly improve the precision and accuracy of bubble group size and distribution identification, and the processing time is shortened.

Description

Technical Field

[0001] This invention relates to the field of image analysis of microbubble clusters during gas-liquid reactions, and specifically to a method, apparatus, and storage medium for bubble recognition based on deep learning. Background Technology

[0002] Gas-liquid two-phase reactions, as complex chemical processes occurring between gaseous and liquid media, are not only widespread in nature but also play a vital role in industrial production and daily life. These reactions involve diverse chemical processes such as acid-base neutralization, redox reactions, and hydrogenation, and are widely applied in environmental protection (e.g., removal of harmful gases from industrial emissions) and the pharmaceutical field (e.g., drug preparation and drug metabolism). In short, gas-liquid reactions are an important type of chemical reaction, playing a crucial role in industrial production and daily life.

[0003] In gas-liquid reactions, bubbles may form due to the generation or release of gases. These bubbles generate and rise in the liquid, and their size and distribution, among other hydrodynamic properties, have a significant impact on the gas-liquid reaction. Identifying bubble size and distribution can provide information about gas-liquid interactions, including reaction rate, reaction efficiency, mass transfer processes, reaction kinetics, and product distribution. Therefore, in experimental and industrial applications, accurately identifying and monitoring bubble size and distribution helps optimize reaction conditions, improve reaction efficiency, and promote the research and application of gas-liquid reactions.

[0004] Currently, there are two main methods for identifying bubble size and size distribution: invasive and non-invasive methods. Invasive methods typically require introducing one or more sensors or probes into the gas-liquid reaction system to measure bubble parameters. This method can provide relatively accurate and detailed bubble information in some cases, especially for measuring specific bubble properties such as bubble diameter and velocity. However, in practice, invasive methods may interfere with the gas-liquid reaction system, particularly for complex and sensitive reaction systems, potentially affecting the accuracy of the reaction. In contrast, non-invasive methods primarily identify and measure bubbles indirectly, such as using optical microscopy and high-speed photography to capture bubble morphology and dynamic changes, or using acoustic sensors and fiber optic sensors to measure changes in the sound or light fields generated by bubbles. These methods are not only simple to operate and do not interfere with the reaction system, maintaining the accuracy of the reaction, but also provide rich bubble information, offering strong support for gas-liquid reaction kinetics research. Furthermore, these methods are relatively simple and easy to implement, applicable to most experimental conditions.

[0005] In the process of realizing this invention, the inventors of this application discovered that the above-mentioned solutions of the prior art have the following defects: due to the complexity and diversity of gas-liquid reaction systems and the different reaction conditions, non-invasive methods are often less accurate than invasive methods in measuring certain bubble characteristics. Summary of the Invention

[0006] The purpose of this invention is to provide a device that, by combining deep learning and non-invasive methods, can measure the size and distribution of bubble swarms under different gas-liquid two-phase reaction conditions, significantly improving the accuracy and precision of bubble swarm size and distribution identification and shortening data processing time.

[0007] To achieve the above objectives, embodiments of the present invention provide a bubble recognition method based on deep learning. The method includes: acquiring original images of bubbles; enhancing the features of the bubbles in the original images to generate preprocessed images; allocating training and testing datasets based on the preprocessed images; constructing a deep learning convolutional neural network model; training the deep learning convolutional neural network model based on the training and testing datasets; and identifying the size and distribution of a bubble cluster based on the trained deep learning convolutional neural network model.

[0008] Optionally, the original images of the collected bubbles include: real images of the bubble cluster under actual working conditions obtained by non-invasive methods.

[0009] Optionally, the bubbles in any of the original images are undistorted and have clearly visible outlines.

[0010] Optionally, enhancing the features of the bubbles in the original images of the original image dataset includes: background correction to correct the unevenness of background illumination in the original images; and noise removal to remove background noise in the original images.

[0011] Optionally, based on the preprocessed image, allocating the training dataset and the test dataset includes: labeling the corresponding annotation information of the bubbles in the preprocessed image to determine the annotation dataset; and dividing the annotation dataset into a first preset proportion as the training dataset and a second preset proportion as the test dataset, wherein the first preset proportion is greater than the second preset proportion.

[0012] Optionally, the corresponding annotation information includes one or more of the following: the category of the bubble and its border.

[0013] Optionally, the construction of the deep learning convolutional neural network model includes: performing transfer learning based on the Lenet-5 model, wherein the Lenet-5 model includes 2 convolutional layers, 2 pooling layers, 2 fully connected layers, and 1 output layer; the deep learning convolutional neural network model includes the 2 convolutional layers, 2 pooling layers, and the first fully connected layer of the Lenet-5 model; and the deep learning convolutional neural network model replaces the last fully connected layer of the Lenet-5 model with a new fully connected layer whose output size is within a preset division scale range.

[0014] Optionally, before training the deep learning convolutional neural network model based on the training dataset and the test dataset, the method further includes: batch processing the images in the training dataset and the test dataset before feeding the training dataset and the test dataset into the deep learning convolutional neural network model, including batch adjusting the width and height of the images to 32 pixels × 32 pixels.

[0015] Optionally, before recognizing the bubble based on the trained deep learning convolutional neural network model, the method further includes: calculating the performance evaluation metrics of the trained deep learning convolutional neural network model; and when the performance evaluation metrics are good, the trained deep learning convolutional neural network model is good, and the training of the deep learning convolutional neural network model is terminated, wherein the performance evaluation metrics include one or more of the following: recall, precision, F1 score, and mean precision; and when the recall, precision, and F1 score are within a preset range, the recall, precision, and F1 score are good performance evaluation metrics.

[0016] On the other hand, the present invention provides a bubble recognition device based on deep learning, the device including a memory and a processor, the processor being configured to execute the bubble recognition method based on deep learning according to any one of the preceding claims of this application to recognize the bubble.

[0017] On the other hand, the present invention provides a machine-readable storage medium storing instructions for causing a machine to execute the deep learning-based bubble recognition method described in any of the preceding claims.

[0018] Through the above technical solution, the original bubble images acquired using a non-invasive method are preprocessed and used to train the deep learning convolutional neural network model based on Lenet-5 model transfer learning provided by this invention. Based on the trained deep learning convolutional neural network model, this invention can provide a reliable method for measuring the size and distribution of microbubble groups. Through this method, this invention can be applied to systems with high gas content during gas-liquid reactions, with high processing accuracy, short processing time, and high robustness. This invention can also adjust the neural network framework according to actual needs, and is not only applicable to bubble groups with regular shapes, but also to shape recognition in other cases with irregular shapes.

[0019] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0021] Figure 1 This is a flowchart of a bubble recognition method based on deep learning provided in an embodiment of the present invention.

[0022] Figure 2 This is a schematic diagram of the original image of a bubble cluster acquired using a non-invasive method, provided in an embodiment of the present invention.

[0023] Figure 3 This is a flowchart illustrating the allocation of training and testing datasets based on the preprocessed images provided in this embodiment of the invention.

[0024] Figure 4 This is a schematic diagram of the network structure of a deep learning convolutional neural network model based on Lenet-5 model transfer learning provided in an embodiment of the present invention.

[0025] Figure 5 This is a schematic diagram of a bubble cluster image identified after deep learning processing, provided in an embodiment of the present invention.

[0026] Figure 6 This is a histogram diagram illustrating the size and distribution of the bubble cluster provided in an embodiment of the present invention. Detailed Implementation

[0027] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0028] Figure 1 This is a flowchart of the bubble recognition method based on deep learning provided in an embodiment of the present invention. (See attached diagram.) Figure 1 As shown, the deep learning-based bubble recognition method includes:

[0029] Step S10: Collect the original image of the bubble;

[0030] In some embodiments of the present invention, real images of bubble clusters under actual working conditions are obtained by non-invasive methods, such as... Figure 2 The image shown is a schematic diagram of the original bubble cluster image acquired using a non-invasive method according to an embodiment of the present invention. Optionally, the bubbles in any of the original images should be distortion-free and have clearly visible outlines. Noise and blurry bubbles should be avoided. See reference... Figure 2 As shown, Figure 2 Bubbles with unclear outlines are not included in the statistics of this invention.

[0031] In some embodiments of the present invention, the preferred non-invasive method involves using a high-performance CCD (Charge-Coupled Device) camera to continuously capture images of moving bubble clusters under different actual working conditions, thereby acquiring high-resolution original images of the microbubble clusters with clear edges. Preferably, the total number of bubbles with no distortion and clearly visible outlines is greater than 10,000. Optionally, the 10,000 bubbles may include: capturing at least 100 original bubble images under different actual working conditions using a high-performance CCD camera, with each original bubble image containing at least 100 bubbles with no distortion and clearly visible outlines.

[0032] In some embodiments of the present invention, to ensure the ability to capture the shape, boundaries, and dynamic changes of the bubble, the high-performance CCD camera required for capturing the original bubble image preferably has a resolution greater than 380,000 pixels; the illumination required for normal operation is preferably below 0.01x, ensuring that even in darker environments, the CCD camera can capture enough light to form a clear image, avoiding blurring or distortion of the original bubble image due to insufficient light; the target surface size is 1 / 2 inch; to capture the rapid movement of the bubble, the frame rate of the high-performance camera should match the actual speed of the bubble's movement. In this embodiment of the present invention, the appropriate frame rate range of the high-performance CCD camera is preferably determined according to the actual speed of the bubble's movement. This ensures that the frame rate is high enough to capture the complete movement trajectory of the bubble, while avoiding excessive redundant data due to an excessively high frame rate.

[0033] Step S20: Enhance the features of the bubbles in the original image to generate a preprocessed image;

[0034] In some embodiments, the original image can be enhanced using various image processing techniques. This embodiment of the invention preferably employs background correction and noise removal as image processing methods to enhance bubble features. In actual bubble photography, factors such as ambient light, light source position, and equipment performance can lead to inconsistent lighting intensity in the background area of ​​the original bubble image, resulting in localized areas of excessively high or low brightness. Optionally, background correction typically uses various image processing techniques to balance the brightness in the image, making the overall illumination more uniform. This embodiment of the invention preferably uses a block-based background difference method to eliminate or correct the unevenness of background illumination in the original image, thereby enhancing the contrast between the bubble and the background, making it easier to identify and extract bubble features. Furthermore, in actual bubble photography, discrete and isolated pixels or pixel blocks (noise) often appear in the background of the original bubble image due to equipment noise, shooting environment, transmission process, etc. Optionally, noise removal typically uses various algorithms to detect and remove these noise points, thereby improving image quality. In this embodiment of the invention, median filtering and morphological opening and closing operations are preferred to remove background noise in the original image, thereby making the image look smoother and more delicate, improving the visual effect of the image. At the same time, it can remove noise that may cover up bubble features, making the bubble features more prominent, so that it is easier to identify and extract bubble features.

[0035] Step S30: Based on the preprocessed images, allocate training and testing datasets;

[0036] Figure 3 This is a flowchart illustrating the allocation of training and testing datasets based on the preprocessed images provided in this embodiment of the invention. (See attached document.) Figure 3 As shown, the flowchart for allocating the training and test datasets based on the preprocessed images includes:

[0037] Step S31: Label the corresponding annotation information of the bubbles in the preprocessed image to determine the labeled dataset;

[0038] In some embodiments, the corresponding annotation information of bubbles in each preprocessed image is manually annotated using OpenCV or Matlab. Optionally, the corresponding annotation information includes one or more of the following: the bubble category and the border. Preferably, the category is classified according to the size of the bubble. Preferably, the bubble size is identified as 50-500 micrometers. Optionally, bubbles in the 50-500 micrometer range can be divided into 90 categories at 5-micrometer intervals. For example, bubbles with a size between 50 and 54 micrometers are classified into one category, bubbles with a size between 55 and 59 micrometers are classified into another category, bubbles with a size between 495 and 500 micrometers are classified into yet another category, and so on.

[0039] Step S32: Divide the labeled dataset into a first preset proportion as the training dataset and a second preset proportion as the test dataset, wherein the first preset proportion is greater than the second preset proportion.

[0040] In some embodiments of the present invention, it is preferable to randomly allocate 70% of the images in the labeled dataset as the training dataset and the remaining 30% as the test dataset.

[0041] Step S40: Construct a deep learning convolutional neural network model;

[0042] In some embodiments, since retraining a deep learning convolutional neural network model requires a lot of time, a suitable convolutional neural network model can be transferred based on a mature convolutional neural network model, such as R-CNN, Faster R-CNN, Mask R-CNN, Lenet-5, ResNet101, etc. Figure 4 This is a schematic diagram of the network structure of a deep learning convolutional neural network model based on Lenet-5 model transfer learning provided in an embodiment of the present invention. (See attached diagram.) Figure 4 As shown, in this embodiment of the invention, a suitable deep learning neural network model is preferably built based on the classic Lenet-5 model and transfer learning techniques. The Lenet-5 model includes two convolutional layers, two pooling layers, two fully connected layers, and one output layer. Figure 4 As shown, the deep learning convolutional neural network model provided in this embodiment of the invention includes an input layer, a hidden layer, and an output layer. The input layer inputs the classified training and test data into the convolutional neural network. The hidden layer forms the main framework of the deep learning convolutional neural network model, including two convolutional layers and two pooling layers of the Lenet-5 model. When an image is input into the convolutional neural network, its feature map can be obtained through the convolutional and pooling layers in the hidden layer. The output layer consists of flattening, a fully connected layer, and an output layer. The flattening operation transforms the multidimensional input into a single dimension, serving as a transition between the convolutional and fully connected layers. The fully connected layer maps the input features to the output result, such as... Figure 4As shown, the fully connected layer includes the first fully connected layer of the Lenet-5 model and a new fully connected layer. The new fully connected layer is a new fully connected layer whose output size is within a preset division scale range after the deep learning convolutional neural network model replaces the last fully connected layer of the Lenet-5 model. The fully connected layer is used to output the classification of images, while the output layer is used to output the results of the deep learning convolutional neural network model.

[0043] In some embodiments, such as Figure 4 As shown, before step S50, the deep learning-based bubble recognition method further includes: before feeding the training dataset and the test dataset into the deep learning convolutional neural network model, i.e., before... Figure 4 The input layer performs batch processing on the images in the training dataset and the test dataset. Optionally, the batch processing includes batch adjusting the width and height of the images in the training dataset and the test dataset. In this embodiment of the invention, the width and height of the images are preferably batch adjusted to 32 pixels × 32 pixels.

[0044] Step S50: Train the deep learning convolutional neural network model based on the training dataset and the test dataset;

[0045] In some embodiments, training the deep learning convolutional neural network model based on the training dataset and the test dataset includes: inputting the batch-adjusted training dataset and the test dataset into the constructed deep learning convolutional neural network model, setting appropriate model parameters, and training the deep learning convolutional neural network model. Optionally, the model parameters include: initial weights and biases and a learning rate. Preferably, in this embodiment of the invention, the initial weights and biases of the deep learning convolutional neural network are set to random values, and the learning rate is set to 0.001 to 0.01, thereby avoiding overfitting and underfitting during training.

[0046] In some embodiments, before step S60, the deep learning-based bubble recognition method further includes: calculating the performance evaluation index of the trained deep learning convolutional neural network model; and when the performance evaluation index is good, the trained deep learning convolutional neural network model is good, and the training of the deep learning convolutional neural network model ends. Optionally, the performance evaluation index includes one or more of the following: recall, precision, F1 score, and mean average precision (mAP). When the recall, precision, and F1 score are within a preset range, the recall, precision, and F1 score are considered good performance evaluation indexes. A good convolutional neural network model should have a recall, precision, and F1 score less than 1. Preferably, the preset range in this embodiment is 85% to 100%.

[0047] In some embodiments, the formulas for calculating recall, precision, and F1 score are as follows:

[0048]

[0049] In this system, TP stands for True Positives, representing the number of samples that are actually positive and predicted as positive by the model. FN stands for False Negatives, representing the number of samples that are actually positive but predicted as negative by the model. FP stands for False Positives, representing the number of samples that are actually negative but predicted as positive by the model. Therefore, recall measures the proportion of samples correctly predicted as positive out of all actual positive samples, while precision measures the proportion of samples predicted as positive by the model that are actually positive. The F1 score is the harmonic mean of recall and precision, used to comprehensively evaluate the model's performance in both recall and precision. When both recall and precision are high, the F1 score will also be high, indicating good model performance.

[0050] Step S60: Based on the trained deep learning convolutional neural network model, identify the size and distribution of the bubble cluster.

[0051] In some embodiments, after a well-trained deep learning convolutional neural network model has been established, the image to be predicted, collected under actual working conditions requiring bubble recognition, is input into the deep learning convolutional neural network model. The model identifies bubble features in the image through a multi-layer convolutional neural network to abstractly extract the semantic concept of bubbles. Then, pooling layers and fully connected layers are used to optimize the bubble feature information. Finally, supervised learning is used to identify and statistically analyze the size and distribution of the bubble clusters in the image. The actual working conditions refer to a gas-liquid two-phase reaction that generates microbubble clusters. Optional and suitable gas-liquid systems include: nitrogen, oxygen, hydrogen, carbon dioxide, water, sodium dodecyl sulfate (SDS) aqueous solution, glycerol, 1,4-butanediol, ionic liquids, etc. Specifically, Figure 5 This is a schematic diagram of a bubble cluster image identified after deep learning processing, provided in an embodiment of the present invention. (See attached image.) Figure 2 and Figure 5 As shown, the embodiments of the present invention identify bubbles of different sizes as much as possible, and preferably identify bubbles without distortion and with clearly visible outlines, while avoiding the identification of noisy and blurry bubbles. Figure 6 This is a histogram diagram illustrating the size and distribution of the bubble clusters provided in an embodiment of the present invention. (See attached image.) Figure 6 As shown, in a preferred embodiment of the present invention, the recognition results output by the deep learning convolutional neural network model of the image to be predicted are post-processed to change the size and distribution of the bubble clusters, as shown in the figure. Figure 6 The histogram shown is presented in the image.

[0052] Specifically, in Example 1 of this invention, the liquid in the gas-liquid system is set to a sodium dodecyl sulfate solution at a temperature of 25°C. Using visualization technology and a microscope, the size of microbubbles within the liquid is photographed at the outlet of the gas-liquid mixture to obtain an image to be predicted. This image is then input into a trained deep learning convolutional neural network model for bubble recognition and detection. Using the deep learning-based bubble recognition method of this invention, the average size of the microbubbles in the image to be predicted is 60 micrometers. Compared to traditional bubble recognition methods such as probe methods, laser particle size analyzers, and image recognition, this embodiment of the invention exhibits smaller measurement errors, higher accuracy, and faster speed.

[0053] Specifically, in Example 2 of this invention, the liquid in the gas-liquid system is replaced with a 30 wt.% aqueous glycerol solution, while other conditions remain the same. Using the deep learning-based bubble recognition method of this invention, the microbubble size in the image to be predicted is output as 50-300 micrometers, and the bubble distribution follows a normal distribution. Compared to traditional bubble recognition methods such as probe methods, laser particle size analyzers, and image recognition, the bubble size measurement error of this embodiment is smaller, more accurate, and faster.

[0054] Specifically, in Example 3 of this invention, the liquid in the gas-liquid system is replaced with a 0.2 mol / L aqueous solution of N-methyldiethanolamine (MDEA), while other conditions remain the same. Using the deep learning-based bubble recognition method of this invention, the microbubble size in the image to be predicted is output as 50-190 micrometers, and the bubble distribution follows a normal distribution. Compared to traditional bubble recognition methods such as probe methods, laser particle size analyzers, and image recognition, the bubble size measurement error of this embodiment is smaller, more accurate, and faster.

[0055] On the other hand, the present invention provides a bubble recognition device based on deep learning, the device including a memory and a processor, the processor being configured to execute the bubble recognition method based on deep learning according to any one of the preceding claims of this application to recognize the bubble.

[0056] The embodiments of the deep learning-based bubble recognition device and the deep learning-based bubble recognition method described above are similar and will not be repeated here.

[0057] The deep learning-based bubble recognition device includes a processor and a memory. The memory stores program units, which are executed by the processor to achieve the corresponding functions.

[0058] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured, and bubble detection is achieved by adjusting kernel parameters.

[0059] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0060] This invention provides a storage medium storing a program that, when executed by a processor, implements the deep learning-based bubble recognition method.

[0061] This invention provides a processor for running a program, wherein the program executes the deep learning-based bubble recognition method during runtime.

[0062] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: a bubble recognition method based on deep learning, the method comprising: acquiring original images of bubbles; enhancing the features of the bubbles in the original images to generate preprocessed images; allocating training and testing datasets based on the preprocessed images; constructing a deep learning convolutional neural network model; training the deep learning convolutional neural network model based on the training and testing datasets; and identifying the size and distribution of a bubble cluster based on the trained deep learning convolutional neural network model.

[0063] Optionally, the original images of the collected bubbles include: real images of the bubble cluster under actual working conditions obtained by non-invasive methods.

[0064] Optionally, the bubbles in any of the original images are undistorted and have clearly visible outlines.

[0065] Optionally, enhancing the features of the bubbles in the original images of the original image dataset includes: background correction to correct the unevenness of background illumination in the original images; and noise removal to remove background noise in the original images.

[0066] Optionally, based on the preprocessed image, allocating the training dataset and the test dataset includes: labeling the corresponding annotation information of the bubbles in the preprocessed image to determine the annotation dataset; and dividing the annotation dataset into a first preset proportion as the training dataset and a second preset proportion as the test dataset, wherein the first preset proportion is greater than the second preset proportion.

[0067] Optionally, the corresponding annotation information includes one or more of the following: the category of the bubble and its border.

[0068] Optionally, the construction of the deep learning convolutional neural network model includes: performing transfer learning based on the Lenet-5 model, wherein the Lenet-5 model includes 2 convolutional layers, 2 pooling layers, 2 fully connected layers, and 1 output layer; the deep learning convolutional neural network model includes the 2 convolutional layers, 2 pooling layers, and the first fully connected layer of the Lenet-5 model; and the deep learning convolutional neural network model replaces the last fully connected layer of the Lenet-5 model with a new fully connected layer whose output size is within a preset division scale range.

[0069] Optionally, before training the deep learning convolutional neural network model based on the training dataset and the test dataset, the method further includes: batch processing the images in the training dataset and the test dataset before feeding the training dataset and the test dataset into the deep learning convolutional neural network model, including batch adjusting the width and height of the images to 32 pixels × 32 pixels.

[0070] Optionally, before recognizing the bubbles based on the trained deep learning convolutional neural network model, the method further includes: calculating the performance evaluation metrics of the trained deep learning convolutional neural network model; and when the performance evaluation metrics are good, the trained deep learning convolutional neural network model is considered good, and the training of the deep learning convolutional neural network model is terminated. The performance evaluation metrics include one or more of the following: recall, precision, F1 score, and mean precision; and when the recall, precision, and F1 score are within a preset range, the recall, precision, and F1 score are considered good performance evaluation metrics. The devices mentioned in this document can be servers, PCs, tablets, mobile phones, etc.

[0071] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: a bubble recognition method based on deep learning, the method comprising: acquiring original images of bubbles; enhancing the features of the bubbles in the original images to generate preprocessed images; allocating training datasets and test datasets based on the preprocessed images; constructing a deep learning convolutional neural network model; training the deep learning convolutional neural network model based on the training dataset and the test dataset; and identifying the size and distribution of a bubble cluster based on the trained deep learning convolutional neural network model.

[0072] Optionally, the original images of the collected bubbles include: real images of the bubble cluster under actual working conditions obtained by non-invasive methods.

[0073] Optionally, the bubbles in any of the original images are undistorted and have clearly visible outlines.

[0074] Optionally, enhancing the features of the bubbles in the original images of the original image dataset includes: background correction to correct the unevenness of background illumination in the original images; and noise removal to remove background noise in the original images.

[0075] Optionally, based on the preprocessed image, allocating the training dataset and the test dataset includes: labeling the corresponding annotation information of the bubbles in the preprocessed image to determine the annotation dataset; and dividing the annotation dataset into a first preset proportion as the training dataset and a second preset proportion as the test dataset, wherein the first preset proportion is greater than the second preset proportion.

[0076] Optionally, the corresponding annotation information includes one or more of the following: the category of the bubble and its border.

[0077] Optionally, the construction of the deep learning convolutional neural network model includes: performing transfer learning based on the Lenet-5 model, wherein the Lenet-5 model includes 2 convolutional layers, 2 pooling layers, 2 fully connected layers, and 1 output layer; the deep learning convolutional neural network model includes the 2 convolutional layers, 2 pooling layers, and the first fully connected layer of the Lenet-5 model; and the deep learning convolutional neural network model replaces the last fully connected layer of the Lenet-5 model with a new fully connected layer whose output size is within a preset division scale range.

[0078] Optionally, before training the deep learning convolutional neural network model based on the training dataset and the test dataset, the method further includes: batch processing the images in the training dataset and the test dataset before feeding the training dataset and the test dataset into the deep learning convolutional neural network model, including batch adjusting the width and height of the images to 32 pixels × 32 pixels.

[0079] Optionally, before recognizing the bubble based on the trained deep learning convolutional neural network model, the method further includes: calculating the performance evaluation metrics of the trained deep learning convolutional neural network model; and when the performance evaluation metrics are good, the trained deep learning convolutional neural network model is good, and the training of the deep learning convolutional neural network model is terminated, wherein the performance evaluation metrics include one or more of the following: recall, precision, F1 score, and mean precision; and when the recall, precision, and F1 score are within a preset range, the recall, precision, and F1 score are good performance evaluation metrics.

[0080] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0085] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0086] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0088] The technical solution of this application states that "the acquisition, transmission, storage, use, and processing of data all comply with the relevant provisions of national laws and regulations" and "it should be noted that in the embodiments of this application, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has used or necessarily used the solution."

[0089] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A bubble recognition method based on deep learning, characterized in that, The method includes: Capture the original image of the bubble; Enhance the features of the bubbles in the original image to generate a preprocessed image; Based on the preprocessed images, training and test datasets are allocated; Constructing deep learning convolutional neural network models; The deep learning convolutional neural network model is trained based on the training dataset and the test dataset; and Based on the trained deep learning convolutional neural network model, the size and distribution of bubble clusters are identified.

2. The bubble recognition method according to claim 1, characterized in that, The original images of the collected bubbles include: real images of bubble clusters under actual working conditions obtained through non-invasive methods.

3. The bubble recognition method according to claim 1, characterized in that, The bubbles in any of the original images are undistorted and their outlines are clearly visible.

4. The bubble recognition method according to claim 1, characterized in that, The enhancement of the features of the bubbles in the original images of the original image dataset includes: Background correction corrects the unevenness of background lighting in the original image; and Noise removal: Remove background noise from the original image.

5. The bubble recognition method according to claim 1, characterized in that, Based on the preprocessed images, the allocation of training and testing datasets includes: Label the corresponding annotation information of the bubbles in the preprocessed image to determine the labeled dataset; and The labeled dataset is divided into a first preset proportion as the training dataset and a second preset proportion as the test dataset, wherein the first preset proportion is greater than the second preset proportion.

6. The bubble recognition method according to claim 5, characterized in that, The corresponding annotation information includes one or more of the following: the type of the bubble and its border.

7. The bubble recognition method according to claim 1, characterized in that, The construction of the deep learning convolutional neural network model includes: transfer learning based on the Lenet-5 model, wherein... The Lenet-5 model includes two convolutional layers, two pooling layers, two fully connected layers, and one output layer. The deep learning convolutional neural network model includes two convolutional layers, two pooling layers, and a first fully connected layer of the Lenet-5 model; and The deep learning convolutional neural network model replaces the last fully connected layer of the Lenet-5 model with a new fully connected layer whose output size is within a preset division scale range.

8. The bubble recognition method according to claim 1, characterized in that, Before training the deep learning convolutional neural network model based on the training dataset and the test dataset, the method further includes: Before feeding the training dataset and the test dataset into the deep learning convolutional neural network model, the images in the training dataset and the test dataset are batch processed, including batch adjusting the width and height of the images to 32 pixels × 32 pixels.

9. The bubble recognition method according to claim 1, characterized in that, Before recognizing the bubble based on the trained deep learning convolutional neural network model, the method further includes: Calculate the performance evaluation metrics of the trained deep learning convolutional neural network model; and When the performance evaluation metrics are good, the trained deep learning convolutional neural network model is considered good, and training of the deep learning convolutional neural network model ends. The performance evaluation metrics include one or more of the following: recall, precision, F1 score, and mean precision; and When the recall rate, precision rate, and F1 score are within a preset range, the recall rate, precision rate, and F1 score are good performance evaluation indicators.

10. A bubble recognition device based on deep learning, characterized in that, The device includes a memory and a processor configured to perform the deep learning-based bubble recognition method according to any one of claims 1-9 to recognize the bubble.

11. A machine-readable storage medium having instructions stored thereon for causing a machine to perform the deep learning-based bubble recognition method according to any one of claims 1-9.

Images