A method for measuring complex background colonies based on deep learning

Abstract

The present invention discloses a method for measuring the size of colonies in a culture dish under a complex background, the method comprising the following steps: using a designed deep learning network feature extractor to expand a sample image, and then completing the measurement of the size of the colonies in the culture dish by applying the designed deep learning network. This method can accurately measure the colonies in biological culture dishes. First, an image of the colony area in the culture dish is collected by a visual system device, and then the size of the colonies in the colony area is measured: the collected colony sample is amplified by a sample of the colony image in a designed adversarial network model, and the adversarial network model is migrated to the detection network of key points in the colony area to enhance the detection capability of key points. The present invention can accurately measure the sizes of colonies of various sizes, and provide a fast and accurate measurement technology for the cultivation of colonies in organisms.

Description

Technical Field

[0001] The invention discloses a method for measuring the size of a colony in the field of measurement based on machine vision, particularly in the biological field. Background Art

[0002] Determination of total colony count is a common testing requirement in industries such as medicine, food, and agriculture. In the food industry, it is particularly important for assessing food hygiene and quality. Currently, methods commonly used to determine total colony count in food include the chromogenic culture method and the electrical impedance method. Both rely on manual calculation and measurement, resulting in high labor costs and low efficiency. Summary of the Invention

[0003] In response to the shortcomings of existing total colony count and measurement methods, the present invention provides a method for measuring colonies in a complex background based on deep learning.

[0004] Compared with existing methods for counting and measuring total colony counts, this colony test method has high robustness and accuracy in measuring colony size, and also greatly reduces the number of colony samples required. The method mainly includes the following steps:

[0005] Step 1: Colony area acquisition: Use the visual optical system device to photograph the colony area in the biological culture dish and obtain an image sample of the colony area.

[0006] Step 2: Colony Sample Amplification. 1) First, group the collected colony images into a set A = {a1, s2, …, an}. Then, collect images of similar scenes into a set B = {b1, b2, …, bn}. 2) Construct the adversarial learning network sample. An adversarial learning sample group is randomly selected from sample set A and sample set B. This sample group is then fed into a data-augmented adversarial network. The designed data-augmented adversarial network consists of a generator and two discriminators. Generators and discriminators are randomly combined, and one of the discriminators is conditioned using a structural similarity loss function (as shown in Equation 1). The generator is a 2K-layer network, where each of the first K layers consists of a 3x3 convolutional layer and a 1x1 convolutional layer. Every two layers contain a dropout layer and use ELU activation. The last K layers undergo dimensionality increase, with each layer adding and multiplying the features obtained from each of the previous K layers. Both discriminators are K+1-layer networks. Discriminator 1 uses a deep residual network structure with a fully connected layer at the K+1th layer. Discriminator 2 uses a pyramid network structure and shares weights with the first K layers of the generator. The K+1th layer is also fully connected. 3) Network training on colony data. First, keep the generator unchanged and train discriminator 1 and discriminator 2. When discriminators 1 and 2 converge, keep discriminator 1 and discriminator 2 unchanged and train the generator. When the generator network converges, finally train the generator, discriminator 1 and discriminator 2 together. When the overall network converges, a large number of colony area samples are generated.

[0007] Formula 1

[0008] Step 3: Colony area key point detection network. 1) Label the colony data: Select typical colony images from the dataset generated in step 2 for data calibration. First, label the colony area, coloring the colony area black and the rest white. Then, based on prior geometric knowledge, process the labeled image to obtain the key points of each colony, such as Figure 22) Construct a colony region keypoint detection network. The colony image and corresponding keypoints from 1) are input into the keypoint detection network for training. The designed keypoint detection network is a 2K+2 layer model. The first K layers share weights with the first K layers of the generator network in the data augmented adversarial network, and the last K layers perform a dimensionality increase operation. Each dimensionality increase operation includes a 3x3 convolution operation, an ELU convolution operation, and a dimensionality increase operation. In the last K layers, each N=2n (n is 1, 2, ..., K / 2) layer is added to the first 2n layers, that is: F(y)=H(x)+x, where x is 2n (n is 1, 2, ..., K / 2); in the last K layers, each N=2n+1 (n is 1, 2, ..., K / 2-1) layer is multiplied with the 2n+1 (n is 1, 2, ..., K / 2-1) layer in the first K layer network, that is: W(z)=H(v)*v, where v is 2n+1 (n is 1, 2, ..., K / 2-1), and full convolution operations are performed on the K-1 layers to obtain F s1 And perform convolution operation with convolution kernel of 3x3, ELU convolution operation and dimension increase operation, and then perform full convolution operation on the Kth layer to obtain F s2 , for F s1 and F s2 Add and perform full convolution operation to get F o , and finally to F s1 and F o Add and perform full convolution operation to get G o , that is, G o =g(F o , F s2 ), where g represents the full convolution operation. 3) Input the sample image into the keypoint detection network, and the network output is the keypoint of the corresponding image.

[0009] Step 4: Accurately measure the size of the colony. The key points of the colony area can be obtained from step 3, and the length R of the colony area can be obtained from formula 2, which is the measured length of the colony.

[0010] Formula 2

[0011] where x i 、y i is the key point of the colony area, 、 , λ is the adjustment coefficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is the bacterial colony image to be measured collected by the present invention.

[0013] Figure 2 It is an image of the key point of the present invention.

[0014] Figure 3 It is the binary labeled image of the present invention. DETAILED DESCRIPTION

[0015] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0016] Compared with existing methods for counting and measuring total colony counts, this colony test method has high robustness and accuracy in measuring colony size, and also greatly reduces the number of colony samples required. The method mainly includes the following steps:

[0017] Step 1: Colony area acquisition: Use the visual optical system device to photograph the colony area in the biological culture dish and obtain an image sample of the colony area.

[0018] Step 2: Colony Sample Amplification. 1) First, group the collected colony images into a set A = {a1, s2, …, an}. Then, collect images of similar scenes into a set B = {b1, b2, …, bn}. 2) Construct the adversarial learning network sample. An adversarial learning sample group is randomly selected from sample set A and sample set B. This sample group is then fed into a data-augmented adversarial network. The designed data-augmented adversarial network consists of a generator and two discriminators. Generators and discriminators are randomly combined, and one of the discriminators is conditioned using a structural similarity loss function (as shown in Equation 1). The generator is a 2K-layer network, where each of the first K layers consists of a 3x3 convolutional layer and a 1x1 convolutional layer. Every two layers contain a dropout layer and use ELU activation. The last K layers undergo dimensionality increase, with each layer adding and multiplying the features obtained from each of the previous K layers. Both discriminators are K+1-layer networks. Discriminator 1 uses a deep residual network structure with a fully connected layer at the K+1th layer. Discriminator 2 uses a pyramid network structure and shares weights with the first K layers of the generator. The K+1th layer is also fully connected. 3) Network training on colony data. First, keep the generator unchanged and train discriminator 1 and discriminator 2. When discriminators 1 and 2 converge, keep discriminator 1 and discriminator 2 unchanged and train the generator. When the generator network converges, finally train the generator, discriminator 1 and discriminator 2 together. When the overall network converges, a large number of colony area samples are generated.

[0019] Formula 1

[0020] Step 3: Colony area key point detection network. 1) Label the colony data: Select typical colony images from the dataset generated in step 2 for data calibration. First, label the colony area, coloring the colony area black and the rest white. Then, based on prior geometric knowledge, process the labeled image to obtain the key points of each colony, such as Figure 2 2) Construct a colony region keypoint detection network. The colony image and corresponding keypoints from 1) are input into the keypoint detection network for training. The designed keypoint detection network is a 2K+2 layer model. The first K layers share weights with the first K layers of the generator network in the data augmented adversarial network, and the last K layers perform a dimensionality increase operation. Each dimensionality increase operation includes a 3x3 convolution operation, an ELU convolution operation, and a dimensionality increase operation. In the last K layers, each N=2n (n is 1, 2, ..., K / 2) layer is added to the first 2n layers, that is: F(y)=H(x)+x, where x is 2n (n is 1, 2, ..., K / 2); in the last K layers, each N=2n+1 (n is 1, 2, ..., K / 2-1) layer is multiplied with the 2n+1 (n is 1, 2, ..., K / 2-1) layer in the first K layer network, that is: W(z)=H(v)*v, where v is 2n+1 (n is 1, 2, ..., K / 2-1), and full convolution operations are performed on the K-1 layers to obtain F s1 And perform convolution operation with convolution kernel of 3x3, ELU convolution operation and dimension increase operation, and then perform full convolution operation on the Kth layer to obtain F s2 , for F s1 and F s2 Add and perform full convolution operation to get F o , and finally to F s1 and F o Add and perform full convolution operation to get G o , that is, G o =g(F o , F s2 ), where g represents the full convolution operation. 3) Input the sample image into the keypoint detection network, and the network output is the keypoint of the corresponding image.

[0021] Step 4: Accurately measure the size of the colony. The key points of the colony area can be obtained from step 3, and the length R of the colony area can be obtained from formula 2, which is the measured length of the colony.

[0022] Formula 2

[0023] where x i 、y i is the key point of the colony area, 、 , λ is the adjustment coefficient.

[0024] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A deep learning-based method for measuring colony length in complex backgrounds. This method proposes a colony size measurement method that combines an adversarial learning network with a key point detection network. First, the adversarial learning network is used to amplify the colony area sample; Then, the weights of the feature extraction part in the adversarial network are transferred to the key point detection network in the colony area; The method comprises the following steps: Step 1: Colony area acquisition: Use the visual light path system device to shoot the colony area in the biological culture dish and obtain an image sample of the colony area; Step 2: Amplification of colony samples: 1) First, the collected colony images are classified into set A = {a1, a2, ..., an}, and then some images of similar scenes are collected and classified into set B = {b1, b2, ..., bn}; 2) Construct adversarial learning network samples: Randomly take an image from sample set A and sample set B to form an adversarial learning sample group, and input it into the data amplified adversarial network. The designed data amplified adversarial network contains a generator and two discriminators. The generator and discriminator are randomly combined, and one of the discriminators is adjusted using a structural similarity loss function. The structural similarity loss function is shown in formula (1). The generator is a 2K-layer network, in which each layer of the first K layers contains a 3x3 convolution layer and a 1x1 convolution layer, and the interval between two layers contains a Dropout. t layer, and use ELU for activation, and the last K layers of the network are dimensionality-enhanced, and each layer adds and multiplies the feature layers obtained in each of the previous K layers; both discriminators are K+1 layer networks, one of which discriminator 1 uses a deep network with a residual network structure, and the K+1th layer is a fully connected network layer, and the other discriminator 2 uses a pyramid network structure and shares the weights of the first K layers in the generator, and the K+1th layer is a fully connected network layer; 3) Colony data network training: First, keep the generator unchanged and train discriminators 1 and 2. When discriminators 1 and 2 converge, then keep discriminators 1 and 2 unchanged and train the generator. When the generator network converges, finally train the generator, discriminator 1 and discriminator 2 together. When the overall network converges, a large number of colony area samples are generated; Step 3: Colony region key point detection network: 1) Label the colony data: Select typical colony images from the dataset generated in step 2 for data calibration. First, the colony region is labeled, with the colony area colored black and the rest colored white. Then, based on prior geometric knowledge, the labeled image is processed to obtain the key points of each colony unit. 2) Construct a key point detection network for the colony area: Input the colony image and the corresponding key points in 1) into the key point detection network for training. The designed key point detection network is a 2K+2 layer model, in which the first K layers of the key point detection network share the weights of the first K layers of the generator network in the data augmented adversarial network, and the last K layers perform a dimensionality increase operation. Each layer of dimensionality increase includes a convolution operation with a convolution kernel of 3x3, an ELU convolution operation, and a dimensionality increase operation. In the last K layers, each N=2n layer is added to the first 2n layers, that is: F( y)=H(x)+x, where x is 2n; n is 1, 2, ..., K / 2; each N=2n+1 layer in the last K layers is multiplied by the 2n+1 layer in the first K layers, that is: W(z)=H(v)*v, where v is 2n+1; n is 1, 2, ..., K / 2-1; perform full convolution operation on the K-1 layers to obtain Fs1 and perform convolution operation with a convolution kernel of 3x3, ELU convolution operation and dimensionality increase operation, then perform full convolution operation on the Kth layer to obtain Fs2, add Fs1 and Fs2 and perform full convolution operation to obtain F o , and finally to F s1 and F o Add and perform full convolution operation to get G o , that is, G o =g(F o , F s2 ), where g represents a full convolution operation; 3) the sample image is input into the key point detection network, and the network output is the key point of the corresponding image; Step 4: Accurate measurement of colony size: The key points of the colony area can be obtained from step 3, and the length R of the colony area can be obtained from formula (2), which is the measured length of the colony; where x i ,y i is the key point of the colony area, λ is the adjustment coefficient.

Images