Emulsion droplet identification system, method, computer device and storage medium

By using UNet neural network to identify emulsion droplets, combined with preprocessing and mask image generation, the problem of low recognition difficulty and accuracy of emulsion droplets in the prior art is solved, and high precision and strong generalization ability are achieved Identify effects.

CN114639101BActive Publication Date: 2025-02-07SHAANXI UNIV OF SCI & TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210278340.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-21
Publication Date
2025-02-07
Estimated Expiration
2042-03-21

AI Technical Summary

Technical Problem

Existing emulsion droplet recognition methods are difficult to accurately identify, especially when images are overfitted or underexposed, and the traditional methods have poor robustness and generalization capabilities.

Method used

Multiple rounds of training are used for UNet neural network, combining preprocessing and mask image generation, and the recognition of emulsion droplets is achieved through deep learning.

Benefits of technology

It improves the recognition accuracy and generalization ability of emulsion droplets, and can be end-to-end batch recognition in images of various backgrounds and categories, solving the problem of low recognition difficulty and accuracy of traditional methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114639101B_ABST
    Figure CN114639101B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of emulsion droplet recognition, and relates to an emulsion droplet recognition method, comprising the following steps: obtaining an original emulsion image, preprocessing the original emulsion image, and obtaining an emulsion sequence image; processing the emulsion sequence image to generate a mask image; using the obtained mask image and the emulsion sequence image together as an emulsion image data set, inputting the emulsion image data set into a UNet neural network for multiple rounds of training, obtaining optimal weight parameters, and obtaining a trained emulsion droplet recognition model; using the trained emulsion droplet recognition model to recognize an emulsion image to be recognized, and outputting an emulsion microscopic image droplet recognition result diagram. The emulsion droplet recognition method proposed by the present invention is based on a UNet network structure, improves the recognition accuracy of emulsion droplets, and has a strong generalization ability, thereby realizing automatic recognition of emulsion droplets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of emulsion droplet recognition, and relates to an emulsion droplet recognition system, method, computer equipment and storage medium. Background Art

[0002] Observing the size and distribution of emulsion particles is a prerequisite for analyzing the stability of emulsions. It is very important for mastering the microscopic characteristics of emulsions and guiding the emulsification and demulsification processes of solutions. The first step in studying the particle size of emulsions is to be able to accurately identify the emulsion droplets. Emulsion images taken under a microscope are easily disturbed by the equipment, generate noise, and have a complex background, making them difficult to identify.

[0003] There are two main existing methods for identifying emulsion droplets: One is to use a global threshold binarization algorithm to identify emulsion droplets. The basic idea is to first find the maximum and minimum values ​​of all pixels in the image, and take the midpoint as the threshold. Set the pixels in the image below a certain threshold to black, and the others to white. However, if the image is overfitted or underexposed, the image may be completely black or completely white, making it impossible to identify emulsion droplets.

[0004] The second method is to perform edge detection on emulsion droplets. The purpose of edge detection is to find a set of pixels in the image with drastic brightness changes. These sets are often the outlines of the target. However, in practice, when detecting emulsion micrographs, the edge changes are not as simple and obvious as expected, and the algorithm process is more complicated, with high requirements for the initial image. The algorithm has poor robustness, different success rates when identifying different types of emulsions, and the algorithm is not very generalizable. Summary of the invention

[0005] The present invention provides an emulsion droplet recognition system, method, computer equipment and storage medium, which solve the problems of difficult and low precision in existing emulsion droplet recognition.

[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0007] A method for identifying emulsion droplets comprises the following steps:

[0008] Step 1, obtaining an original emulsion image, preprocessing the original emulsion image, and obtaining an emulsion sequence image;

[0009] Step 2: Processing the emulsion sequence image to generate a mask image;

[0010] Step 3: The obtained mask image and the emulsion sequence image obtained in step 1 are used together as an emulsion image dataset, and the emulsion image dataset is input into the UNet neural network for multiple rounds of training to obtain the optimal weight parameters and obtain a trained emulsion droplet recognition model;

[0011] Step 4: Use the trained emulsion droplet recognition model to identify the emulsion image to be identified, and output the emulsion microscopic image droplet recognition result image.

[0012] Furthermore, in step 1, original images of different types of emulsions are obtained using a microscope.

[0013] Furthermore, in step 1, the preprocessing is used to perform data expansion on the original emulsion image, and the data expansion method is horizontal flipping, vertical flipping, random cropping, rotation or adding noise.

[0014] Further, in step 2, the emulsion sequence image is annotated using the Labelme software, the emulsion droplet region annotation name is set as target, and a json file corresponding to the emulsion sequence image is generated;

[0015] Generate a mask image using the json file and the emulsion sequence image obtained in step 1.

[0016] Furthermore, in step 3, the UNet neural network includes 5 downsampling layers and 5 upsampling layers. The last downsampling layer is connected to the first upsampling layer. There is a skip connection chain between the first four downsampling layers and the last four upsampling layers. The last upsampling layer is connected to the fully connected layer to output the recognition result.

[0017] Furthermore, in step 3, the activation function used by the UNet neural network is the Relu function.

[0018] Further, in step 3, when the UNet neural network performs the nth round of training, the classification result data of each pixel point of the emulsion sequence image in the training set is first obtained, and the classification result data and the value of each pixel point of the mask image obtained in step 2 are calculated using the loss function to obtain the loss value;

[0019] The loss value is back-propagated to the UNet neural network for parameter update, and then the UNet network model with updated parameters is used to perform forward inference on the validation set images, and the MIOU value is calculated using the inference result and the mask image obtained in step 2;

[0020] The calculated MIOU value is compared with the highest MIOU value in the previous n-1 rounds, and the network structure and model parameters with the highest MIOU value are saved as the optimal emulsion droplet recognition model.

[0021] The present invention also discloses an emulsion droplet recognition system, comprising:

[0022] A data preprocessing module is used to preprocess the original emulsion image to obtain an emulsion sequence image;

[0023] The mask image generation module is used to annotate the emulsion sequence image and generate a json file containing the annotation name and the annotation position; and generate a mask image using the json file and the emulsion sequence image;

[0024] A UNet neural network module is used to use the obtained mask image and the emulsion sequence image as an emulsion image data set, input the emulsion image data set into the UNet neural network for training, obtain the optimal weight parameter, and obtain a trained emulsion droplet recognition model;

[0025] The image detection module is used to identify droplets in the emulsion microscopic image, input the emulsion microscopic image into the trained emulsion droplet recognition model, and obtain the emulsion microscopic image droplet recognition result map.

[0026] The present invention also discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the emulsion droplet identification method when executing the computer program.

[0027] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the emulsion droplet identification method are implemented.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention discloses an emulsion droplet recognition method, which can make full use of the advantages of deep learning. After one-time labeling, emulsion images of various backgrounds and categories can be end-to-end batch recognized. It has strong universality and generalization ability, and solves the problem that traditional image processing methods are single-scene and need to design specific solutions for each type of scene, eliminating the design and implementation of complex algorithms and improving work efficiency. On the other hand, traditional image processing methods pay less attention to information, have low recognition accuracy and poor segmentation effect. The emulsion droplet recognition method proposed in the present invention is based on the UNet network structure, which has a large number of channel connections between upsampling and downsampling, can effectively utilize the deep and shallow features of the image, retain the underlying details of the image, improve the recognition accuracy of emulsion droplets, and has strong generalization ability, realizing automatic recognition of emulsion droplets.

[0030] Furthermore, when saving the final network recognition model, only the network model with the highest MIOU value is retained, which not only ensures the accuracy and reliability of the recognition model, but also avoids excessive storage space occupation caused by retaining too many models, as well as the extra workload of ultimately screening the optimal model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the process of the present invention;

[0032] Figure 2 It is the Unet neural network structure diagram of the present invention;

[0033] Figure 3 The original microscopic image of the emulsion tested in the present invention;

[0034] Figure 4 This is a Unet network detection result diagram of the emulsion microscopic image of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0038] like Figure 1 As shown, the present invention discloses a method for identifying emulsion droplets, comprising the following steps:

[0039] Step 1, obtaining an original emulsion image, preprocessing the original emulsion image, and obtaining an emulsion sequence image;

[0040] Step 2: Processing the emulsion sequence image to generate a mask image;

[0041] Step 3: The obtained mask image and the emulsion sequence image obtained in step 1 are used together as an emulsion image dataset, and the emulsion image dataset is input into the UNet neural network for multiple rounds of training to obtain the optimal weight parameters and obtain a trained emulsion droplet recognition model;

[0042] Step 4: Use the trained emulsion droplet recognition model to identify the emulsion image to be identified, and output the following: Figure 4 The emulsion micrograph is shown with droplet identification results.

[0043] The specific steps are as follows:

[0044] 1. Dataset Preparation

[0045] like Figure 3 As shown, the collected emulsion microscopic image is used as the emulsion original image, and the emulsion original image is preprocessed to obtain the emulsion sequence image;

[0046] The specific preprocessing is: use Python's opencv library to perform batch horizontal and vertical flipping, cropping or adding noise to the images for data enhancement to enhance the robustness of the network model.

[0047] Use Labelme software to annotate the data and set the emulsion droplet annotation name as target. After the annotation is completed, a corresponding json file will be generated for each image.

[0048] Use Python language and OpenCV library to batch process the original emulsion microscopy images and the corresponding JSON files to generate mask images.

[0049] Then, the emulsion microscopy image and the generated mask image with the same name are placed in two folders, named img and label, and input into the UNet network for training.

[0050] In layman's terms, the "mask map" is a "standard answer" given manually. During the training process, the network determines the accuracy of its current training results by comparing the "mask map". The difference between the predicted value and the mask map (true value) after each round of training is calculated through the loss function. After the calculation, the parameters are updated and the next round of training is carried out until it gets closer and closer to the true value. Therefore, the emulsion image dataset consists of two parts, one is the original emulsion image, and the other is the generated mask map.

[0051] 2. Network model construction

[0052] The UNet deep learning network framework is widely used for semantic segmentation. It does not require a lot of data, so it can be easily and efficiently applied to various scenarios. The UNet network is mainly composed of downsampling encoding, upsampling decoding, and skip connections. Downsampling encoding performs convolution, normalization, and maximum pooling operations on the input image in the network to reduce the image size and extract shallow features of different scales of the image. Upsampling decoding uses interpolation and convolution methods to restore the image size of the extracted features of different scales of the image and extract the deep features of the image. Skip connections fuse the features of different scales extracted from each downsampling with the corresponding features extracted from the upsampling, combine deep and shallow features, refine the image, and then perform recognition.

[0053] As attached Figure 2 As shown, the UNet neural network established by the present invention for identifying emulsion droplets consists of 5 downsampling layers and 5 upsampling layers. Each downsampling layer will perform two convolution operations, and the first four downsampling layers are connected by a pooling layer. Each upsampling layer will also perform two convolutions, and the last four upsampling layers are connected by a deconvolution layer. There are 4 skip connection chains between the first 4 downsampling layers and the last 4 upsampling layers. The 5th downsampling layer is connected to the 1st upsampling layer, and the 5th upsampling is followed by a fully connected layer to output the recognition result. The activation function used in this network is Relu.

[0054] The upsampling layer is a process of extracting features from the emulsion microscopic image using a large number of convolution kernels. In the network of this invention example, the size of the convolution kernel used is 3x3, and the step size is 1. The convolution kernel matrix and the emulsion microscopic image matrix perform mathematical operations to convert the pixels in the 3x3 window in the original image into 1 pixel. After the operation is completed, the convolution kernel slides one step to the right and continues the operation. In this network structure, the padding value is 1, so the size of the output image remains unchanged after the convolution operation. The number of convolution kernels increases by 1 times after each upsampling layer.

[0055] The pooling layer is located between the two downsampling layers. The present invention uses the maximum pooling method, with a pooling kernel size of 2x2 and a step size of 2. The pooling layer can reduce the dimension of the extracted feature information. On the one hand, it makes the feature map smaller, simplifies the network calculation complexity, and avoids overfitting to a certain extent. On the other hand, it compresses the features and extracts the main features. After the pooling operation, the length and width of the feature map will be reduced by half.

[0056] The activation function is a function that runs on neurons and is responsible for mapping the input of the neuron to the output. The activation function introduces nonlinear factors to the neuron, allowing the neural network to arbitrarily approximate any nonlinear function. Compared with other activation functions, the Relu activation function has no saturation region, no gradient vanishing problem, no complex exponential operations, simple calculation, high efficiency, and fast actual convergence speed.

[0057] The upsampling layer restores the extracted feature vector map to an image. The upsampling layer first crops the feature vector map extracted by the corresponding downsampling layer to the same size as the upsampling layer image, then splices it, and then extracts features through two convolution operations. The convolution kernel size used in this process is 3x3, and the step size is 1. The operation of this layer combines the shallow and deep features of the image, and the restored image is more accurate. The number of convolution kernels in each upsampling layer is the same as the number of convolution kernels in the corresponding downsampling layer.

[0058] Deconvolution is located between two upsampling layers. The image is expanded using interpolation. The size of the deconvolution kernel is 2x2 and the step size is 2. After this operation, the length and width of the image will be doubled.

[0059] The fully connected layer is located after the last upsampling layer, which divides the image into two categories: foreground and background, and combines the masks of the output images, which is the final recognition result. This layer uses a convolution kernel of size 1x1.

[0060] As attached Figure 2 As shown, the UNet neural network established by the present invention for identifying emulsion droplets consists of 5 downsampling layers and 5 upsampling layers. Each downsampling layer will perform two convolution operations, and the first four downsampling layers are connected by pooling layers. Each upsampling layer will also perform two convolution operations, and the last four upsampling layers are connected by deconvolution layers. There are 4 skip connection chains between the first 4 downsampling layers and the last 4 upsampling layers. An emulsion microscopic image is input into the UNet network, and the downsampling layer extracts features from the emulsion microscopic image.

[0061] The present invention uses a convolution kernel of size 3x3 and step length 1 to perform convolution operation with the matrix of the input emulsion microscopic image, and performs 2 convolution operations in total in each layer. Each time through convolution, the obtained feature map needs to be nonlinearly mapped using the Relu activation function. After the first downsampling layer is completed, the extracted feature information is reduced in dimension by using the pooling layer to extract the main features, so that the length and width of the feature map become half of the original, simplifying the network calculation complexity and preventing overfitting during training. Downsampling is performed 5 times in total, the number of convolution kernels for the first time is 64, and then for each layer, the number of convolution kernels doubles compared to the previous layer, and by the 5th layer, the number of convolution kernels is 1024. A scale is added for each pooling layer, and there are 4 pooling layers in total. Adding the scale of the original image, there are 5 scales in total, so this structure realizes multi-scale feature recognition of images.

[0062] The last downsampling layer is connected to the first upsampling layer. During upsampling, the first step is to crop the feature map obtained by downsampling at the corresponding level, and then splice it with the feature map obtained by the previous upsampling layer, and then use a convolution kernel with a size of 3x3 and a step size of 1 to perform two convolution operations to extract features. After each convolution, the Relu activation function needs to be used to perform nonlinear mapping on the obtained feature map. After the two convolution operations are completed, a deconvolution operation is required, that is, the feature map is expanded by interpolation so that the image gradually returns to the input size. The size of the deconvolution kernel used in the present invention is 2x2, and the step size is 2. After each upsampling layer, the number of convolution kernels is reduced by one time compared to the previous layer. By the fifth layer, the number of convolution kernels changes from 1024 in the first layer to 64. The last upsampling layer is connected to two fully connected layers, and the fully connected layer uses a convolution kernel with a size of 1x1 to classify each pixel in the emulsion microscopic image and distinguish between background pixels and droplet pixels.

[0063] 3. Model Training

[0064] Use the emulsion sequence images and the corresponding json files to generate mask images, and then make an emulsion dataset, which contains emulsion sequence images and mask images. The emulsion sequence images and the corresponding mask images are included, and the emulsion sequence images and mask images are used to generate training sets, validation sets, and test sets in a ratio of 7:2:1.

[0065] Regarding the segmentation of the training set, validation set, and test set, when generating the mask image, the default name of the image is the same as the original image name. When segmenting these three parts, they are divided into 7:2:1 according to the image name. For example, if the image is named "Img1", it corresponds to two images, one is the original image and the other is the mask image. That is, an original image and a mask image can be regarded as a whole, one for training and learning, and the other for comparing and observing the accuracy of network learning.

[0066] The prepared emulsion microscopic image and the generated mask image are input into the built UNet network, and the Dropout method is used for training. The training is stopped when the number of training times reaches the preset threshold. After each training, the MIOU value in the model obtained by this round of training is compared with the highest MIOU value in history. MIOU (Intersection-over-Union) is a standard metric for semantic segmentation, which refers to the calculation of the intersection-over-union ratio of two sets. In this case, it is the ratio of the intersection and union of the pixel values ​​of the mask image generated by the annotation and the pixel values ​​of the predicted image obtained by network training. The value is between 0 and 1, and the closer to 1, the more accurate the prediction result. The obtained accuracy is compared with the best historical accuracy in the model (accuracy refers to the specific indicator MIOU), and then the emulsion droplet recognition model with the highest MIOU value is taken as the final result.

[0067] The formula for MIOU is:

[0068]

[0069] Among them, k is the number of categories, i represents the true value, j represents the predicted value, and p ii It means predicting i as i,p ij Indicates predicting i as j,p ji It means predicting j as i.

[0070] In this invention, the loss function used by the UNet neural network is:

[0071]

[0072] Among them, L is the cross entropy calculation function, M represents the number of categories, and y c It is a one-hot vector with two values: 0 and 1. If the predicted category after network training is the same as the true category, the value is 1, otherwise it is 0. c Represents the probability that the predicted value belongs to category c.

[0073] The loss function is used to evaluate the gap between the emulsion image generated during network training and the mask image generated by manual annotation, which will be fed back to the network for parameter update during the training process.

[0074] 4. Example Effect

[0075] In this example, a total of 1500 emulsion microscopic images were collected and constructed, of which 1050 images were used as a training set, 300 images were used as a validation set, and 150 images were used as a test set. The model with the highest accuracy on the validation set was selected as the final model and applied to the test set. The average performance of the model on the test set is shown in the following table:

[0076] Dataset Training duration Loss MIOU value Predict image time Emulsion microscopy images 14.5h 0.1562 0.8031 0.068s

[0077] Intuitively, according to the values ​​of the Loss function and the MIOU in the above table, the emulsion droplets on the image can be basically identified with high accuracy and good recognition effect.

[0078] In this application example, when an emulsion microscopic image is input into the network, the network forward propagates and outputs the classification result of each pixel in the emulsion microscopic image, and then compares it with the pixel value of the mask image generated by manual annotation, and uses the loss function to calculate the difference value. The error value is back-propagated to the network using the chain rule. During back-propagation, the optimizer uses Adam to update the parameters in the network, such as the convolution kernel value and bias, and determines whether to save this round of network model according to the MIOU value. At this point, one round of training is completed. In the example of the present invention, the total number of iterations of the UNet network model epochs is 150, and the initial learning rate is set to 5e -4 , the learning rate is halved every 10 epochs during training.

[0079] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the emulsion droplet identification method are implemented. The computer storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO), etc.), optical storage (such as CD, DVD, BD, HDD, etc.), and semiconductor storage (such as ROM, EPRO M, EEPRO M, non-volatile memory (NANDFLASH), solid state drive (SSD)), etc.

[0080] In an exemplary embodiment, a computer device is also provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the emulsion droplet identification method when executing the computer program. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0081] The emulsion droplet recognition method proposed in the present invention is a method that directly outputs an image mask from an input emulsion microscopic original image. The upsampling layer of the UNet network is connected to the downsampling layer, so that the network can fuse the shallow and deep features of the image, effectively retain the image details, and perform higher-precision detection.

[0082] The present invention can make full use of the advantages of the semantic segmentation model UNet network, realize end-to-end emulsion droplet recognition, and solve the problem that the traditional image processing method is applicable to a single scene. On the other hand, the traditional image processing method has complex operation steps, the algorithm is too targeted, and the universality is weak. It cannot solve the problem in batches and has low work efficiency. The emulsion droplet recognition method proposed in the present invention not only improves the recognition accuracy, but also has a strong generalization ability, and realizes the automatic recognition of emulsion droplets.

[0083] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A method for identifying emulsion droplets, characterized in that: The following steps are involved: Step 1, obtaining an original emulsion image, preprocessing the original emulsion image, and obtaining an emulsion sequence image; Step 2: Processing the emulsion sequence image to generate a mask image; Step 3: The obtained mask image and the emulsion sequence image obtained in step 1 are used together as an emulsion image dataset, and the emulsion image dataset is input into the UNet neural network for multiple rounds of training to obtain the optimal weight parameters and obtain a trained emulsion droplet recognition model; Step 4: Use the trained emulsion droplet recognition model to recognize the emulsion image to be recognized, and output the emulsion microscopic image droplet recognition result image; The UNet neural network established for identifying emulsion droplets consists of 5 downsampling layers and 5 upsampling layers; each downsampling layer performs two convolution operations, and the first four downsampling layers are connected by pooling layers; each upsampling layer also performs two convolution operations, and the last four upsampling layers are connected by deconvolution layers; there are 4 skip connection chains between the first 4 downsampling layers and the last 4 upsampling layers; an emulsion microscopic image is input into the UNet network, and the downsampling layer extracts features from the emulsion microscopic image; A convolution operation is performed with a convolution kernel of size 3x3 and a step size of 1 and the matrix of the input emulsion microscopic image. A total of 2 convolution operations are performed in each layer. Each time a convolution is performed, the Relu activation function is used to perform nonlinear mapping on the obtained feature map. After the first downsampling layer is completed, the pooling layer is used to reduce the dimension of the extracted feature information and extract the main features, so that the length and width of the feature map are reduced to half of the original, simplifying the network calculation complexity and preventing overfitting during training. Downsampling is performed 5 times in total. The number of convolution kernels for the first time is 64. After each layer, the number of convolution kernels doubles compared to the previous layer. By the 5th layer, the number of convolution kernels is 1024. A scale is added after each pooling layer. There are 4 pooling layers in total. Together with the scale of the original image, there are 5 scales in total, realizing multi-scale feature recognition of the image. The last downsampling layer is connected to the first upsampling layer; during upsampling, the first step is to crop the feature map obtained by downsampling at the corresponding level, and then concatenate it with the feature map obtained by the previous upsampling layer, and then use a convolution kernel of size 3x3 and a step size of 1 to perform two convolution operations to extract features; after each convolution, the Relu activation function is required to perform nonlinear mapping on the obtained feature map; after two convolution operations, a deconvolution operation is required, that is, the feature map is expanded by interpolation so that the image gradually returns to the input size; the size of the deconvolution kernel used is 2x2, and the step size is 2; after each upsampling layer, the number of convolution kernels is reduced by half compared to the previous layer. By the fifth layer, the number of convolution kernels changes from 1024 in the first layer to 64; the last upsampling layer is connected to two fully connected layers, and the fully connected layer uses a convolution kernel of size 1x1 to classify each pixel in the emulsion microscopy image and distinguish between background pixels and droplet pixels.

2. The method for identifying emulsion droplets according to claim 1, characterized in that: In step 1, different types of emulsion original images are obtained using a microscope.

3. The method for identifying emulsion droplets according to claim 1, characterized in that: In step 1, the preprocessing is used to perform data expansion on the original emulsion image, and the data expansion method is horizontal flipping, vertical flipping, random cropping, rotation or adding noise.

4. The method for identifying emulsion droplets according to claim 1, characterized in that: In step 2, the emulsion sequence images are annotated using the Labelme software, the emulsion droplet region annotation name is set as target, and a json file corresponding to the emulsion sequence images is generated; Generate a mask image using the json file and the emulsion sequence image obtained in step 1.

5. The method for identifying emulsion droplets according to claim 1, characterized in that: In step 3, when the UNet neural network performs the nth round of training, the classification result data of each pixel point in the emulsion sequence image in the training set is first obtained, and the classification result data and the value of each pixel point of the mask image obtained in step 2 are calculated using the loss function to obtain the loss value; The loss value is back-propagated to the UNet neural network for parameter update, and then the UNet network model with updated parameters is used to perform forward inference on the validation set images, and the MIOU value is calculated using the inference result and the mask image obtained in step 2; The calculated MIOU value is compared with the highest MIOU value in the previous n-1 rounds, and the network structure and model parameters with the highest MIOU value are saved as the optimal emulsion droplet recognition model.

6. An emulsion droplet recognition system for implementing the emulsion droplet recognition method according to any one of claims 1 to 5, characterized in that: include: A data preprocessing module is used to preprocess the original emulsion image to obtain an emulsion sequence image; The mask image generation module is used to annotate the emulsion sequence images and generate a json file containing the annotation name and the annotation position; Generate a mask image using the json file and the emulsion sequence image; A UNet neural network module is used to use the obtained mask image and the emulsion sequence image as an emulsion image data set, input the emulsion image data set into the UNet neural network for training, obtain the optimal weight parameter, and obtain a trained emulsion droplet recognition model; The image detection module is used to identify droplets in the emulsion microscopic image, input the emulsion microscopic image into the trained emulsion droplet recognition model, and obtain the emulsion microscopic image droplet recognition result map.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the emulsion droplet identification method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the emulsion droplet identification method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Blood cell automatic detection and recognition method

    CN110032985A

  • Corn disease grading method and system and computer equipment

    CN111553240A