High-throughput spotting instrument liquid drop identification method based on convolutional neural network and de-noising diffusion probability model

Abstract

The invention discloses a high-flux sample application instrument liquid drop automatic detection method based on the combination of a residual network and a diffusion model, and belongs to the technical field of computer vision and automatic control, and the method comprises the steps: 1, carrying out the denoising processing of an image through the combination of the diffusion model, so as to enhance the definition and accuracy of the characteristics of liquid drops; 2, using a residual network to extract droplet image features in a sample application result, emphasizing the size and the position of the droplet through an attention module, and finally judging whether the size and the position of the droplet meet the expectation or not. Aiming at the detection requirement of the target liquid drop in the high-flux sample application instrument, the automatic operation without manual intervention is realized, the detection precision (the comprehensive detection error is less than or equal to 0.3%) can be ensured, the processing speed of more than 30 frames per second is realized, the sample application efficiency is obviously improved, and the operation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the fields of image processing and deep learning, and in particular to a droplet recognition method applied to a high-throughput spotter. Background Art

[0002] With the continuous development of chemical experiments and research, high-throughput spotters have been widely used in many fields such as droplet sample analysis and chemical reagent screening. However, traditional droplet recognition methods mainly rely on manual visual observation or image processing techniques based on feature engineering. These methods are sensitive to factors such as noise, light changes, and irregular droplet shapes in droplet samples, which easily lead to problems of insufficient recognition accuracy and low processing efficiency. Summary of the Invention

[0003] The present invention is to solve the above-mentioned deficiencies existing in the prior art, and proposes a droplet recognition method for a high-throughput spotter based on a denoising diffusion probability model, an attention mechanism, and a convolutional neural network, so as to improve the accuracy and processing efficiency of droplet recognition, meet the requirements of high-throughput spotters for the high efficiency and precision of droplet sample processing in experiments, and thus ensure more accurate droplet recognition results in the processing of large-scale experimental data.

[0004] In order to overcome the deficiencies of the prior art, the present invention adopts the following technical solutions:

[0005] A droplet detection method for a high-throughput spotter based on a residual network and a diffusion model according to the present invention is characterized by including the following steps:

[0006] Step 1: After obtaining a series of droplet images formed by ejecting droplets at different positions on the detection surface of the high-throughput spotter and performing normalization processing, a droplet image set is obtained , where represents the droplet image at the th position, and is the total number of images;

[0007] Step 2: Use forward noise addition and backward denoising to process to obtain a target purified image set ; where is the target purified image at the th position;

[0008] Step 3: Build an image classification network based on ResNet-18, which sequentially includes: a backbone network module, an attention mechanism module CBAM, and a classification module, and process to obtain the judgment result of droplet detection in ;

[0009] Step 4: If , it means that the droplets in the target field of view meet the requirements;

[0010] If , it means that the droplets in the target field of view do not meet the requirements; otherwise, it means that there are no droplets in the target field of view, where and represent two thresholds.

[0011] Another feature of the droplet detection method of the high-throughput spotting instrument based on the residual network and the diffusion model according to the present invention is that the step 2 includes:

[0012] Step 2.1: Gradually add noise to using Equation (1) to obtain the th droplet image at the t-th time step , so as to obtain a set of droplet images at the total time step :

[0013] (1)

[0014] In Equation (1), represents the noise coefficient at the t-th time step, and , represents the diffusion function at the t-th time step, and , represents the diffusion intensity at the initial time step, represents the total time step, is the shape parameter, represents the noise coefficient at the t-th step that follows the standard normal distribution;

[0015] Step 2.2: Gradually remove noise from using Equation (2) to obtain the th purified droplet image at the t-th time step , so as to obtain a set of purified droplet images at the total time step :

[0016] (2)

[0017] In Equation (2), is the noise recovery coefficient at the t-th step that follows the Gaussian distribution;

[0018] Step 2.3: Use Equation (3) to construct the reconstruction error loss of the th image And perform minimization to obtain the optimal purified droplet image set at the total time step :

[0019] (3)

[0020] In formula (3), represents the expectation of the noise variable , is the square of the two-norm;

[0021] Step 2.4, from the optimal purified droplet image at the initial time step in is used as the target purified image at the th position , thereby obtaining the target purified image set . .

[0022] Furthermore, the said step 3 includes:

[0023] Step 3.1, Input it into the backbone network module, and after being processed by the convolutional layer, pooling layer and residual block in sequence, obtain the target droplet high-dimensional feature map , where H, W, and C respectively represent the height, width, and number of channels of

[0024] Step 3.2, the attention mechanism module CBAM includes: a channel attention unit and a spatial attention unit, which are used to extract the size and edge position part of the target droplet in the target droplet high-dimensional feature map , thereby converting into the corresponding potential feature ;

[0025] Step 3.3, the classification module uses formula (7) to obtain the judgment result of droplet detection in ;

[0026] (7)

[0027] In formula (7), are respectively the weights and biases of the classifier, represents the potential feature value at position and channel t.

[0028] Furthermore, the said step 3.2 includes:

[0029] Step 3.2.1. The channel attention unit calculates the channel attention map of using Equation (4): :

[0030] (4)

[0031] In Equation (4), represents the fully connected layer, is the Sigmoid activation function, represents the eigenvalue at position and channel t, represents taking the maximum value over the dimensions of the entire spatial position;

[0032] Step 3.2.2. The spatial attention unit calculates the spatial attention map of using Equation (5): :

[0033] (5)

[0034] In Equation (4), represents taking the maximum value among all channels;

[0035] Step 3.2.3. Obtain the potential features including droplet size and edge position using Equation (6): :

[0036] (6)

[0037] In Equation (6), is the Hadamard product.

[0038] An electronic device according to the present invention includes a memory and a processor, characterized in that the memory is used to store a program for supporting the processor to execute the droplet recognition method of the high-throughput spotter, and the processor is configured to execute the program stored in the memory.

[0039] A computer-readable storage medium according to the present invention, characterized in that a computer program stored on the computer-readable storage medium executes the steps of the droplet recognition method of the high-throughput spotter when run by a processor.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] 1. The present invention can achieve fast target droplet detection and analysis, reduce the demand for human resources, and can achieve real-time recognition and processing in experiments related to high-throughput spotters, meeting the requirements of high-efficiency experiments.

[0042] 2. By introducing a denoising diffusion probability model, the present invention can further improve the image quality during the denoising process, making the recognition of target droplets more accurate and effectively dealing with droplet samples under different experimental conditions.

[0043] 3. During the training process of the denoising diffusion probability model of the present invention, by introducing a noise coefficient in the form of cosine , compared with the traditional linear form, the destruction of information is slower, and better training effects can be obtained.

[0044] 4. The present invention introduces an attention mechanism module. By focusing on the size and edge position information of the target droplet, it can complete the droplet recognition task on the high-throughput spotter more accurately than ordinary neural networks.

[0045] 5. The deep learning model adopted by the present invention can be further trained and optimized to adapt to different types of droplet samples and different experimental scenarios, providing good scalability for subsequent scientific research and engineering applications.

[0046] In summary, by combining the deep convolutional neural network with the attention mechanism and the denoising diffusion probability model, the present invention can detect the position information, size and shape of droplets in real time and accurately. Especially during the training process, by optimizing and improving the hyperparameters , the speed of information destruction is significantly reduced, and better training effects are obtained, and the accuracy of target droplet recognition in the spotting results is significantly improved during the denoising and generation processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 is a schematic flow diagram of the forward diffusion and reverse denoising processes in the denoising diffusion probability model;

[0048] Figure 2 is a comparison diagram of the optimized hyperparameters and the traditional linear form;

[0049] Figure 3 is a comparison diagram of the effects of the traditional linear form (upper) and the optimized (lower);

[0050] Figure 4 is a schematic diagram of the network structure of ResNet-18;

[0051] Figure 5 is a schematic diagram of the attention mechanism module CBAM;

[0052] Figure 6 is a schematic diagram of the channel attention unit structure in CBAM;

[0053] Figure 7 is a schematic diagram of the spatial attention unit structure in CBAM. Detailed implementation mode

[0054] In this embodiment, a method for automatically identifying droplets in a high-throughput spotter based on deep learning combines the denoising ability of the diffusion model, the deep feature extraction ability of the residual network, and the attention mechanism for the first time to automatically detect whether the size and position of the target droplets in the spotting results meet the expectations, providing more accurate automatic analysis capabilities for the high-throughput spotter. The method is carried out according to the following steps:

[0055] Step 1: After obtaining a series of droplet images formed by spraying droplets at different positions on the detection surface of the high-throughput spotter and performing normalization processing, a droplet image set is obtained , where represents the droplet image at the th position, is the total number of images;

[0056] Step 2: As shown in Figure 1 , by performing forward noise addition and reverse denoising on , a target purified image set is obtained; where is the target purified image at the th position;

[0057] In the droplet detection task of the high-throughput spotter, the micron-scale target droplet images are easily affected by noise, which may lead to errors and inaccuracies in the droplet detection process. To further improve the accuracy and robustness of droplet recognition, by simulating the inverse process of the noise data distribution, the model can generate higher-quality image representations, thereby enhancing the droplet recognition ability.

[0058] Step 2.1: Use Equation (1) to perform step-by-step forward noise addition on to obtain the th droplet image at the t-th time step , thereby obtaining the th droplet image set under the total time step :

[0059] (1)

[0060] In Equation (1), represents the noise coefficient at the t-th time step, and , represents the diffusion function at the t-th time step, and , represents the diffusion intensity at the initial time step, represents the total time step, is the shape parameter, which is selected in this embodiment , prevent the noise coefficient from being too small near , represents the noise coefficient at the t-th step following a standard normal distribution.

[0061] The goal of the forward noise addition process is to gradually transform the original image into a noisy image. In the classical denoising diffusion probability model process, a series of fixed linear are used as the noise coefficient. Although the linear noise coefficient works well for high-resolution images, it performs poorly for low-resolution images. Adding a large amount of noise at the beginning of the diffusion step will seriously damage the information of the image and is not conducive to subsequent learning. In this embodiment, the micron-scale target droplets on the high-throughput spotter are relatively small in the camera's field of view. Therefore, a cosine-form noise coefficient is adopted in this embodiment to start adding noise more slowly. See the comparison schematic diagram in Figure 2 and Figure 3 .

[0062] Step 2.2, use Equation (2) to perform step-by-step reverse denoising on to obtain the purified droplet image at the t-th time step, thereby obtaining the purified droplet image set at the total time step :

[0063] (2)

[0064] In Equation (2), is the noise recovery coefficient at the t-th step following a Gaussian distribution.

[0065] Step 2.3, use Equation (3) to construct the reconstruction error loss of the image and perform minimization to obtain the optimal purified droplet image set at the total time step : :

[0066] (3)

[0067] In Equation (3), represents the expectation of the noise variable , is the square of the two-norm;

[0068] Step 2.4, select the optimal purified droplet image at the initial time step in as the target purified image at the -th position , thus obtaining the target purified image set .

[0069] Step 3: Build an image classification network based on ResNet-18, which successively includes: a backbone network module, an attention mechanism module CBAM, and a classification module;

[0070] Step 3.1, Input into the backbone network module, and the specific network structure is shown in Figure 4 . After being successively processed by an initial convolutional layer (7×7 convolution, stride 2), a max pooling layer (3×3 pooling, stride 2), and four residual modules, a target droplet high-dimensional feature map is obtained, where H, W, and C respectively represent the height, width, and number of channels of . In this embodiment, , respectively represent the standard height and width of the target purified image , and 512 is the number of channels of the output ;

[0071] Step 3.2: The attention mechanism module CBAM includes: a channel attention unit and a spatial attention unit, which are used to extract the size and edge position part of the target droplet in the target droplet high-dimensional feature map , thereby converting into the corresponding potential feature . The module schematic diagram is shown in Figure 5 , Figure 6 , Figure 7 :

[0072] Step 3.2.1: The channel attention unit calculates the channel attention map of using Equation (4):

[0073] (4)

[0074] Step 3.2.2: The spatial attention unit calculates the spatial attention map of using Equation (5):

[0075] (5)

[0076] In Equation (4) and Equation (5), is the Sigmoid activation function, represents the fully connected layer, represents the eigenvalue of at position It means to select the maximum value in the dimension of the entire spatial position to obtain the global channel saliency. It means to select the maximum value among all channels, ignoring redundant information channels to highlight the characteristics of the target droplet size and edge position.

[0077] Step 3.2.3: Use Equation (6) to obtain the potential features including droplet size and edge position :

[0078] (6)

[0079] In Equation (6), is the Hadamard product.

[0080] Step 3.3: The classification module uses Equation (7) to obtain the judgment result of droplet detection in ;

[0081] (7)

[0082] In Equation (7), are respectively the weights and biases of the classifier, means at position and the potential feature value of channel t.

[0083] Step 4: If , it means that the droplets in the target field of view meet the requirements;

[0084] If , it means that the droplets in the target field of view do not meet the requirements; otherwise, it means that there are no droplets in the target field of view, where and represent 2 thresholds. In this embodiment, and are respectively taken as 0.5 and 0.1.

[0085] In this embodiment, an electronic device includes a memory and a processor. The memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0086] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is run by a processor, it executes the steps of the above method.

Claims

1. A droplet detection method for a high-throughput spotting instrument based on a residual network and a diffusion model, characterized in that, Including the following steps: Step 1: After obtaining a series of droplet images formed by ejecting droplets at different positions on the detection surface of a high-throughput spotter and performing normalization processing, a droplet image set is obtained , where represents the droplet image at the th position, is the total number of images; Step 2: Use forward noise addition and backward denoising to process to obtain a target purified image set ; where is the target purified image at the th position; Step 3: Build an image classification network based on ResNet-18, which successively includes: a backbone network module, an attention mechanism module CBAM, and a classification module, and perform processing on to obtain the judgment result of droplet detection in ; Step 4. If , it means that the droplets in the target field of view meet the requirements; If , it means that the droplets in the target field of view of do not meet the requirements; otherwise, it means that there are no droplets in the target field of view of , where and represent two thresholds.

2. The droplet detection method of a high-throughput spotting instrument based on a residual network and a diffusion model according to claim 1, wherein, The said step 2 includes: Step 2.1: Use Equation (1) to perform step-by-step positive noise addition to obtain the th droplet image at the t-th time step , thereby obtaining the th droplet image set at the total time step : (1) In Equation (1), represents the noise coefficient at the t-th time step, and , represents the diffusion function at the t-th time step, and , represents the diffusion intensity at the initial time step, represents the total number of time steps, is the shape parameter, represents the noise coefficient at the t-th step that follows a standard normal distribution; Step 2.2: Use Equation (2) to perform step-by-step reverse denoising on to obtain the th purified droplet image at the t-th time step , thereby obtaining the th set of purified droplet images under the total time steps : (2) In formula (2), is the noise recovery coefficient at the t-th step that follows a Gaussian distribution; Step 2.

3. Construct the reconstruction error loss of the th image using Equation (3), and perform minimization to obtain the th set of optimal purified droplet images at the total time step : (3) In formula (3), represents the expectation of the noise variable , is the square of the two-norm; Step 2.4: From at the initial time step, select the th optimal purified droplet image as the target purified image at the th position, thereby obtaining a set of target purified images . .

3. The droplet detection method of a high-throughput spotting instrument based on a residual network and a diffusion model according to claim 2, wherein, The said step 3 includes: Step 3.1, Input it into the backbone network module, and after being processed by a convolutional layer, a pooling layer, and a residual block in sequence, a target droplet high-dimensional feature map is obtained , where H, W, and C respectively represent the height, width, and number of channels; Step 3.

2. The attention mechanism module CBAM includes a channel attention unit and a spatial attention unit, which are used to extract the size and edge position part of the target droplet in the high-dimensional feature map of the target droplet, so as to convert it into corresponding latent features: in the target droplet, so as to convert it into corresponding latent features : Step 3.3, the classification module obtains the judgment result of droplet detection in ; (7) In Equation (7), are the weights and biases of the classifier, represents at position and the latent feature value on channel t.

4. The droplet detection method of a high-throughput spotting instrument based on a residual network and a diffusion model according to claim 3, wherein, The said step 3.2 includes: Step 3.2.1: The channel attention unit calculates the channel attention map of using Equation (4). : (4) In formula (4), represents a fully connected layer, is the Sigmoid activation function, represents the eigenvalue at position and channel t, represents selecting the maximum value over the dimensions of the entire spatial position; Step 3.2.2: The spatial attention unit calculates the spatial attention map of using Equation (5). : (5) In formula (4), represents selecting the maximum value among all channels; Step 3.2.

3. Obtain potential features including droplet size and edge position using Equation (6). : (6) In formula (6), is the Hadamard product.

5. An electronic device, comprising a memory and a processor, characterized in that, The memory is used for storing a program that supports the processor to execute any one of the high-throughput spotting instrument droplet recognition methods as claimed in claims 1-4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it executes the steps of any one of the high-throughput spotting instrument droplet recognition methods as claimed in claims 1-4.

Images