Automatic recognition method for array brain images

Abstract

The present invention relates to the technical field of image data processing, and particularly relates to an automatic recognition method for array brain images. The above-mentioned automatic recognition method includes the following steps: constructing a segmentation data set, constructing a segmentation model, training the segmentation model, obtaining the segmentation results of mouse brain images, and establishing an identification algorithm for array mouse brain cropping frames. The segmentation model provided by the present invention differentiates the filtered images under guided filters of different scales, better extracts the edge information of the array mouse brain, and constructs this module in combination with a convolutional neural network. Under variable parameters, the ability of this module to extract edge features is further enhanced; this module is combined with the U-Net network framework to achieve end-to-end segmentation of multiple mouse brains in the array image, and the identification algorithm for array mouse brain cropping frames can greatly improve the recognition accuracy of the cropping frame position.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and particularly relates to an automatic recognition method for array brain images. Background Art

[0002] The composition of the brain is highly complex, with numerous cell types and numbers. Different types of nerve cells have distinct morphologies and functions. In order to directly and clearly visualize the internal brain structure and better understand the corresponding brain functions and their mechanisms, Professor Luo Qingming's team has spent more than a decade developing the MOST (Micro-optical Sectioning Tomography) series of imaging technologies. In 2023, a method called array fluorescence microscopic optical sectioning tomography (array-fMOST) was described. This method includes the embedding and staining of array mouse brains, large-scale imaging, and data analysis after imaging.

[0003] The imaging mode of array-fMOST greatly improves the data throughput for obtaining the original mouse brain imaging dataset. It can perform complete coronal plane imaging on 32 mouse brain specimens simultaneously within 15 days at a three-dimensional voxel resolution of 0.65×0.65×3μm 3 and collect slice images for further analysis and calculation. To make the data meet the requirements of subsequent data analysis (such as soma counting, neuron reconstruction, etc.), preprocessing operations such as cross-sectional stitching, stripe artifact removal, and mouse brain region cropping need to be performed on the original strip data obtained from imaging. However, the scale of a set of original imaging strip data of array mouse brains obtained at this resolution can reach 220TB. Data preprocessing requires a large amount of computing resources and time, which poses a great limitation to brain science research. Therefore, how to improve the data preprocessing efficiency has become an important issue.

[0004] Since array-fMOST is a newly developed imaging technology and there is no targeted preprocessing method yet, in order to further process the original imaging strip data of the array mouse brain to obtain sets of mouse brain datasets with good quality, independence, and integrity, the R & D team of this imaging technology has designed a preprocessing scheme in engineering. This processing scheme is divided into five parts: First, redundant splicing is performed on the original imaging strip data to obtain a complete reconstructed image of the array data section, and then operations such as downsampling and z-projection are performed on it to obtain multi-level resolution images and save them; Second, after importing the array image into ImageJ, the position parameters of each mouse brain are repeatedly compared and recorded by visual observation of the human eye; Third, the spliced array image is read in according to the layer number sequence and the position of a single mouse brain is specified for cropping to obtain the whole brain data of a single set of mouse brains; Fourth, the moving median filtering method is used to filter out the stripe artifacts in the mouse brain imaging dataset obtained in the previous step; Fifth, the processed whole brain coronal mesoscopic imaging dataset of the mouse brain is re-sliced to obtain images from the cross-sectional and sagittal perspectives.

[0005] In the above processing scheme, obtaining the positions of each mouse brain in the entire set of array images by visual observation of the human eye will consume a lot of time and energy of the staff, which is not conducive to the integration of the entire preprocessing step and the automation of the preprocessing process, reducing the efficiency of the entire preprocessing scheme.

[0006] At the same time, the array mouse brain images also have the following major characteristics:

[0007] 1) Due to the differences in dozens of mouse brain samples embedded together (for example, differences in the sample death time: as the sample death time increases, the cell membrane permeability increases, and the PI dye can more easily cross the membrane and bind to the DNA molecule, resulting in an increase in fluorescence intensity or gray value), the gray intensity differences of each mouse brain in a single array image are large;

[0008] 2) When imaging with an optical microscope, light scattering occurs during the process of light passing through the sample, which causes some light to be scattered in non-original directions, affecting the imaging quality. Especially for deep samples, due to the longer light propagation distance, the scattering phenomenon in the sample is more significant, and the resulting imaging is often darker and more blurred, which makes there be an average gray difference between the layers of the entire set of array data;

[0009] 3) For different sets of array data, since the parameters of the imaging system are not constant (for example, different Gaussian excitation beams may be used, and there may also be differences in the sample embedding method), the average gray of the mouse brains in different sets of array data also has large differences.

[0010] Due to the above characteristics of the array mouse brain images, the array mouse brain data has a certain degree of complexity. Therefore, it is difficult to set an appropriate gray-scale threshold to define the mouse brain samples, noise, and background, making it difficult to manually set appropriate features for traditional algorithms that require strict manual features (such as the selection of binarization thresholds in traditional algorithms), resulting in a very low average recognition accuracy for mouse brains. Summary of the Invention

[0011] The object of the present invention is to overcome the defects of the prior art and propose an automatic recognition method for array brain images, which can effectively improve the recognition accuracy of the position of the mouse brain.

[0012] To achieve the above object, the present invention adopts the following specific technical solutions:

[0013] The automatic recognition method for array brain images provided by the present invention includes the following steps:

[0014] S1. Construct a segmentation data set and use the segmentation data set to train a segmentation model;

[0015] S2. Construct a segmentation model and perform segmentation processing on the mouse brain image;

[0016] S3. Train the segmentation model and input the segmentation data set into the segmentation model for training;

[0017] S4. Obtain the segmentation result of the mouse brain image and use the trained segmentation model to segment the mouse brain image to obtain the segmentation result of the mouse brain image;

[0018] S5. Establish an identification algorithm for the cropping frame of the array mouse brain, detect the segmentation result of the mouse brain image, and identify the position of the mouse brain.

[0019] Further, in step S1, constructing the segmentation data set includes the following two steps:

[0020] The first step is to select mouse brain images as the training set, validation set, and test set respectively, where the ratio of the training set to the validation set is 4:1;

[0021] The second step is to perform data annotation on the mouse brain images, mark the positions of the mouse brains in the mouse brain images, and generate corresponding binary label maps.

[0022] Further, in step S2, constructing the segmentation model includes the following steps:

[0023] S21. Construct a denoising module for denoising the mouse brain image. The denoising module is designed based on a deep difference guided filter;

[0024] S22. Construct an encoding module and a decoding module for segmenting the denoised mouse brain image and outputting the segmentation result of the mouse brain image. The encoding module and the decoding module are designed based on U-Net.

[0025] Further, in step S21, the specific construction process of the denoising module is as follows:

[0026] Use a 1×1 convolution to compress the image I input with d channels 0 into d / 4 channels; then reduce the image resolution to half of the original through downsampling operation to obtain the feature map I, so as to reduce the number of parameters of the module; after that, pass the feature map I through depth-guided filters DGF-i with different sizes in sequence to obtain the guided images F 1 、F 2 and F 3 at different scales, and then perform pairwise differences on the guided images at different scales to obtain multi-scale detail features D1, D2, and D3; use a concat layer combined with 1×1 convolution to fuse the obtained multi-scale detail layers, upsample the fused feature map to the original resolution through interpolation, and fuse it with the original input image in the form of a skip connection; finally, obtain the denoising module after batch normalization processing and PReLU non-linear activation function.

[0027] Further, in step S21, the specific process of the denoising module for denoising the mouse brain image is as follows:

[0028] S211. Compress and downsample the mouse brain image to obtain a feature map to reduce the number of parameters of the denoising module;

[0029] S212. Pass the feature map through depth-guided filters with different sizes in sequence to obtain guided images at different scales, and then perform pairwise differences on the guided images at different scales to obtain multi-scale detail features;

[0030] S213. Fuse the multi-scale detail features, upsample the fused feature map to the original resolution through interpolation, and fuse it with the original input mouse brain image in the form of a skip connection. After batch normalization processing and PReLU non-linear activation function, denoising and detail enhancement of the mouse brain image are realized.

[0031] Further, in step S21, the working principle of the depth difference guided filter is as follows:

[0032] The depth difference guided filter is designed by combining the traditional guided filter principle with the convolution operation in deep learning. The key assumption of the guided filter is the local linear model between the guided image I and the filtered output image q, and q is the linear transformation of I in the window w k centered on pixel k:

[0033]

[0034] The guiding image I, the image to be guided P, the image I×I, and I×P are respectively passed through the mean filter box_filter to obtain mean I , mean P , corr I and corr IP , and then the variance of I and the covariance of I and P are calculated:

[0035]

[0036]

[0037] a k and b k are passed through the mean filter box_filter to obtain mean ak and mean bk ;

[0038] Combining the above principle in the convolutional neural network, the above mean filter box_filter operation is replaced by a convolutional operation, and the input image is used as both the guiding image and the image to be guided to obtain the deep guiding filter DGF-i, where i represents the i-th time in the differential iteration process of the guiding filter in the depth difference guiding filtering operation, and the scale of DGF is correspondingly adjusted. The specific operation is to change the regularization parameter ε of the filter and the convolution kernel size of the box_filter: the regularization parameter ε i =τ i 2 , where τ i+1 =τ i / 2 (τ 0 =0.8), the convolution kernel radius r i =2r i-1 +1 (r 0 =1).

[0039] Furthermore, in step S22, the encoding module includes a first convolutional layer and a max pooling layer. The first convolutional layer is used to extract the features of the image, and the max pooling layer is used to reduce the resolution of the image and retain the features; the decoding module includes an upsampling layer and a second convolutional layer. The upsampling layer is used to restore the feature map in the encoder to the original resolution, and the second convolutional layer is used to convert the feature map into a pixel-level segmentation result.

[0040] Furthermore, in step S3, training the segmentation model includes the following steps:

[0041] S31. Input the training set and validation set data into the segmentation model for training;

[0042] S32. Input the test set into the segmentation model for testing and output the segmentation result as a binary image.

[0043] Further, in step S31, inputting the training set and validation set data into the segmentation model for training is specifically as follows:

[0044] Set batch_size to 6, set the number of training epochs to 300, set the initial learning rate to 0.001, adopt Adaptive Moment Estimation to adaptively adjust the learning rate during training; select CrossEntropy loss as the loss function, and update the weight parameters through the backpropagation algorithm;

[0045] The calculation formula of Cross Entropy loss is:

[0046]

[0047] where y i represents the true value of the training sample, the positive class is 1, the negative class is 0, and p i represents the probability that sample i is predicted as the positive class;

[0048] Print the current result once per training epoch and save the model with the minimum current loss function until the number of training epochs reaches 300 to complete the training.

[0049] Further, in step S5, detecting and identifying the position of the mouse brain in the mouse brain image segmentation result includes the following steps:

[0050] S51. Filter the noise in the mouse brain image segmentation result and output a binary image;

[0051] S52. Detect the position of the mouse brain in the binary image after filtering the noise and identify the position of the mouse brain.

[0052] Further, in step S51, filtering the noise in the mouse brain image segmentation result and outputting a binary image includes the following steps:

[0053] S511. Obtain the address index of the binary image sequence of the mouse brain image segmentation result and the maximum value of the number of connected components in a single binary image;

[0054] S512. Read the image corresponding to the address index, erode the image with a structuring element, and calculate the connected components in the image;

[0055] S513. Remove the connected components in the image that are less than the preset value of the maximum value of the number of connected components, and record the total number of remaining connected components in the image;

[0056] S514. If the total number of remaining connected components is greater than the maximum number of connected components, save the binary image after noise filtering.

[0057] Further, in step S52, detecting the position of the mouse brain in the binary image after noise filtering includes the following steps:

[0058] S521. Read the binary image after noise filtering, calculate the minimum bounding rectangle for each connected component therein, and store the parameters in a list.

[0059] S522. If the rectangle positions stored in the list intersect with the rectangle position of the binary image after noise filtering, merge the two rectangles and update the list parameters according to the merged result.

[0060] S523. Filter out the outliers and duplicate clipping position parameters in the list.

[0061] S524. Draw the specific positions of the clipping frames on the original array of mouse brain image sequences according to the clipping parameters in the list and save the images.

[0062] The present invention can achieve the following technical effects:

[0063] 1. The present invention provides a method for segmenting an array of mouse brains based on a convolutional neural network. By differentiating the filtered images under guided filters of different scales, the edge information of the array of mouse brains is better extracted, and a module is constructed in combination with a convolutional neural network. Under variable parameters, the ability of the module to extract edge features is further enhanced. Combining this module with the U-Net network framework, end-to-end segmentation of multiple mouse brains in the array image is realized, and the Dice index reaches 0.92.

[0064] 2. The present invention provides an algorithm for identifying clipping frames based on a binary image sequence. According to the continuous section imaging mode of the array-fMOST system and combined with the non-overlapping characteristics of the mouse brains in the array, an algorithm for identifying clipping frames of the mouse brains in the array is designed based on the binary result after semantic segmentation, and the recognition accuracy rate of the clipping frame position can reach more than 98%. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a schematic diagram of extracting image details by multi-scale iterative differential guided filtering according to an embodiment of the present invention.

[0066] Figure 2 is a flowchart of an automatic recognition method for an array of brain images according to an embodiment of the present invention.

[0067] Figure 3 is a schematic structural diagram of a segmentation model according to an embodiment of the present invention.

[0068] Figure 4 It is a schematic structural diagram of a depth difference guided filter provided according to an embodiment of the present invention.

[0069] Figure 5 It is a partial operation schematic diagram of a guided filter provided according to an embodiment of the present invention.

[0070] Figure 6 It is a partial test result schematic diagram of a segmentation model provided according to an embodiment of the present invention. Detailed implementation manners

[0071] In the following, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same modules are denoted by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, their detailed descriptions will not be repeated.

[0072] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention.

[0073] An embodiment of the present invention provides an automatic recognition method for array brain images, including constructing a segmentation model based on a convolutional neural network, and according to the continuous slice imaging mode of the array-fMOST system, combining the non-overlapping characteristics between the array mouse brains, and designing an array mouse brain cropping frame recognition algorithm based on traditional image processing algorithms based on the binary results after semantic segmentation to obtain the specific positions of each mouse brain.

[0074] Convolutional neural network (CNN) is a commonly used deep learning architecture. CNN can extract local information in images and integrate it into overall information for classification or segmentation. It can handle complex image scenarios such as mixed noise, low contrast, low resolution, etc. U-net is a commonly used image segmentation network, a fully convolutional network (FCN) based on convolutional neural network (CNN). Its structure is designed to be symmetric, which can effectively capture context information in images and achieve accurate image segmentation. The U-net network is mainly divided into two parts: the encoder and the decoder. Among them, the encoder is mainly composed of convolutional layers, pooling layers, and skip connections, which are used to extract feature information from the input image. At the same time, the resolution of the feature map is continuously reduced, and finally it is converted into a low-dimensional feature vector. The decoder is composed of transposed convolutional layers, skip connections, and convolutional layers, which are used to recover the high-resolution feature map from the low-dimensional feature vector and finally generate the segmentation result. It is one of the most widely used image segmentation models at present. Guided Filter is a local linear model based on images proposed by Kaiming He et al. in 2010 for image filtering and enhancement. It can reduce image blur and noise by guiding the information of the image and keep the edge information from being blurred. It is commonly used in tasks such as image denoising, image enhancement, and image smoothing. In 2022, Yu Zhang et al. proposed that iteratively differentiating the filtered images obtained by guided filters at different scales can effectively obtain the details of the image. We verified the characteristics on array mouse brain images. Figure 1 Shows the extraction of image details by multi-scale iterative differential guided filtering.

[0075] First, iteratively guided filtering is performed on the array mouse brain image by setting different window sizes and regularization parameters to generate smoothed images at multiple scales. The specific formula is:

[0076] F i = f g (F i-1 , F i-1 , r i , ε i );

[0077] Among them, 1 ≤ i ≤ n, f g represents the guided filter, F i represents the guided filtering result of the i-th scale. In particular, F 0 is the original input image, r i represents the radius of the local window in the i-th scale, ε i is the regularization parameter of the filter, n is the number of scales for image decomposition, r i = 2r i-1 + 1 (r 0= 1), ε i = τ i 2 , where τ i+1 = τ i / 2(τ 0 = 0.8).

[0078] Based on this, the present invention designs a Deep Differential GuideFilter (DDGF) convolution module. By combining the multi-scale iterative differential guide filtering method with deep learning, it can learn richer features under variable parameters, effectively enhancing the details of the array mouse brain images while removing the background noise in the images. Further, by combining this module with the U-Net network and continuously tuning the network structure through experiments, a lightweight and high-precision array mouse brain segmentation model is designed. Finally, according to the continuous slice imaging mode of the array-fMOST system and the non-overlapping characteristics of the array mouse brains, an array mouse brain cropping frame recognition algorithm is designed based on the traditional image processing algorithm relying on the binary result after semantic segmentation.

[0079] Figure 2 shows the flow of the automatic recognition method for array brain images provided by the embodiments of the present invention. As Figure 2 shown, the array mouse brain images produced by the array-fMOST are input into the network and a segmentation model is constructed through training. After the input images are segmented by the segmentation model, data cleaning is performed on the segmentation results. Then, according to the continuous slice imaging mode of the array-fMOST system and the non-overlapping characteristics of the array mouse brains, an array mouse brain cropping frame recognition algorithm is designed based on the traditional image processing algorithm relying on the binary result after semantic segmentation.

[0080] The segmentation model in the array mouse brain segmentation method mainly includes three modules, namely, a denoising module, an encoding module, and a decoding module. Figure 3 shows the structure of the segmentation model. As Figure 3 shown, for the input image, first, the denoising module enhances its details while suppressing the background noise generated by the array-fMOST system imaging, greatly improving the image quality. The basic design idea of the encoding and decoding module is consistent with the U-Net network. A shortcut connection is added in the double convolution part, that is, the input of the convolution is directly connected to the output, adding a linear connection, and then passing through the PReLU activation function.

[0081] The automatic recognition method for array brain images provided by the embodiments of the present invention specifically includes the following steps:

[0082] S1. Construct a segmentation data set.

[0083] A segmentation dataset was constructed using three sets of array mouse brain images (IDs: 221000, 220374, and 221405) produced by the Suzhou Institute of Brain Spatial Information, Huazhong University of Science and Technology. Since the xy-resolution of the actually produced array mouse brain images is 0.65 μm, a single image can reach a size of 16000×80000 pixels and vary in size. Therefore, the original images were cropped, downsampled, etc. using MATLAB, and finally, all the array mouse brain images obtained were 512×512 pixels in size. According to the above processing method, 150 images obtained from 221405 were used as the test set, and 1200 images processed from 221000 and 220374 were divided into the training set and the validation set (training set: validation set = 4:1). The processed array mouse brain images were data-annotated using Amira software, that is, the position of the mouse brain in the image was annotated to generate the corresponding binary label map.

[0084] S2. Build a segmentation model.

[0085] The segmentation model includes a denoising module, an encoding module, and a decoding module. The denoising module realizes the functions of image denoising and detail enhancement by designing a deep differential guided filter. The design ideas of the encoding module and the decoding module follow the U-Net scheme. The encoding module contains a series of convolutional layers and max-pooling layers. The convolutional layers are used to extract the features of the image, while the max-pooling layers are used to reduce the resolution of the image while retaining the most significant features. This part can be regarded as a conventional convolutional neural network. The decoding module also contains a series of upsampling layers and convolutional layers. The upsampling layers are used to restore the feature maps in the encoder to the original resolution, and the convolutional layers are used to convert the feature maps into pixel-level segmentation results.

[0086] S21. Build a denoising module;

[0087] Figure 4 The structure of the deep differential guided filter is shown, as Figure 4 shown. First, a 1×1 convolution is used to compress the d-channel input image I 0 into d / 4 channels, and then the image resolution is reduced to half of the original through downsampling operations to obtain the feature map I, which can effectively reduce the number of parameters of the module. Then, the feature map I is successively passed through deep guided filters DGF-i of different sizes to obtain the guided images F 1 , F 2 and F 3, pairwise differences are then performed to obtain multi-scale detail features D1, D2, and D3. The obtained multi-scale detail layers are fused using a concat layer combined with a 1×1 convolution. The feature map after detail fusion is upsampled to the original resolution by interpolation and fused with the original input image in the form of a skip connection. Finally, after batch normalization and the PReLU non-linear activation function, the fitting ability of the module can be effectively improved, realizing the functions of image denoising and detail enhancement.

[0088] Among them, the DGF-i structure represents depth-guided filters of different scales. The key assumption of the guided filter is the local linear model between the guidance image I and the filtered output image q, where q is the linear transformation of I in the window w centered on pixel k k :

[0089]

[0090] Figure 5 Shows the solution process of the parameters mean ak and mean bk in the formula, as shown in Figure 5 , the guidance image I, the image to be guided P, the image I×I, and I×P are respectively passed through mean filtering box_filter (replaced by convolution operation here) to obtain mean I , mean P , corr I and corr IP , and then calculate the variance of I and the covariance of I and P:

[0091]

[0092]

[0093] a k and b k are passed through mean filtering box_filter to obtain mean ak and mean bk , and the guided filter result is obtained according to the calculation method of formula (1). DGF-i is a depth-guided filter constructed according to the above steps with the input image as both the guidance image and the image to be guided. Here, i represents the i-th time in the differential iteration process, corresponding to adjusting the scale of DGF. The specific operation is to change the regularization parameter ε of the filter and the size of the convolution kernel in box_filter: the regularization parameter τ i+1 =τ i / 2 (τ 0 =0.8), and the convolution kernel radius r i =2r i-1 +1 (r 0= 1).

[0094] S22. Construct an encoding module and a decoding module;

[0095] In this design, through experimental testing, the number of channels of the encoding and decoding module is reduced to 1 / 4 of the original U-net scheme, and only three upsampling and downsampling operations are performed. While maintaining the segmentation accuracy, the number of network parameters is greatly reduced. In the encoding module, each convolutional layer consists of two consecutive 3x3 convolutional kernels and a PReLU activation function. There is a 2x2 max pooling layer behind each convolutional layer to reduce the resolution of the image. In the decoding module, an upsampling block contains a nearest upsampling layer, a concatenation function, and a double convolutional layer, using BatchNorm2d and PReLU activation functions. The network first performs two convolutions to change the number of channels, and then performs multiple upsampling, concatenation, and double convolution operations to restore to the original image size.

[0096] Following the U-Net scheme, skip connections are added between the encoder and the decoder to combine the high-level feature maps in the encoder with the low-level feature maps in the decoder, thereby helping to improve the segmentation performance and reduce over-smoothing. The entire network uses the PReLU activation function, except for the last layer activation using Sigmoid to scale the output to an appropriate range to calculate the cross-entropy loss.

[0097] S3. Input the segmentation dataset into the segmentation model for training.

[0098] Input the segmentation dataset constructed in step S1 into the segmentation model constructed in step S2 to train the segmentation model. The specific process is as follows:

[0099] S31. Input the training set and validation set data into the segmentation model for training.

[0100] Input the prepared training set and validation set of mouse brains in arrays into the constructed segmentation model for training. Set batch_size to 6, the number of training epochs to 300, and the initial learning rate to 0.001. Use Adam (Adaptive Moment Estimation) to adaptively adjust the learning rate during training; select Cross Entropy loss as the loss function, and update the weight parameters through the backpropagation algorithm.

[0101] The calculation formula of CE loss is:

[0102]

[0103] where y i represents the true value of the training sample, the positive class is 1, the negative class is 0, pi It represents the probability that sample i is predicted as the positive class.

[0104] Print the current result once per training epoch, and save the model with the minimum current loss function until the number of training epochs is set to 300 to complete the training.

[0105] S32: Input the test set into the segmentation model for testing, and output the segmentation result as a binary image.

[0106] Input the prepared test set of 150 array mouse brain images into the trained model for testing, and the segmentation results corresponding to the 150 array mouse brain sample images are obtained and presented in the form of binary images.

[0107] Compare the segmentation results of the segmentation model with those of the U-Net and ESP-Net models. Train and learn the U-Net and ESP-Net models using similar training strategies and the same training data, and finally evaluate the results by calculating the Dice and Jaccard similarity coefficients.

[0108] When calculating the similarity between two sets using the Dice coefficient, the formula is as follows:

[0109]

[0110] Among them, |x| represents the number of elements in set x, |y| represents the number of elements in set y, and intersection(x,y) represents the intersection of the two sets.

[0111] The Jaccard coefficient only concerns the issue of whether the features shared by individuals are consistent, and is generally used to detect the similarity and difference between finite sample sets. The formula is as follows:

[0112]

[0113] Among them, intersection(x,y) represents the intersection of x and y, and union(x,y) represents the union of x and y.

[0114] The larger the Dice and Jaccard coefficients, the higher the similarity between the two samples.

[0115] The average similarity coefficients obtained by segmenting with U-Net, ESP-Net, and DDGF-UNet (ours) through MATLAB are shown in Table 1, which shows the number of parameters of different models and the calculated average Jaccard coefficient and Dice coefficient.

[0116] Table 1

[0117] Params Mean Jaccard Mean Dice U-Net 31037698 0.8125 0.8953 ESP-Net 345563 0.7504 0.8544 Segmentation model 483459 0.8601 0.9233

[0118] As can be seen from Table 1, the segmentation model provided in this embodiment has a higher segmentation performance for array mouse brain images than the segmentation results of the ESP-Net and U-Net models while maintaining a similar number of parameters to the lightweight network ESP-Net.

[0119] S4. Obtain the mouse brain image segmentation result, and segment the mouse brain image through the trained segmentation model to obtain the mouse brain image segmentation result.

[0120] S5. Establish an algorithm for identifying the cropping frames of array mouse brains.

[0121] According to the segmentation result of the mouse brain in the obtained array image, further based on the continuous section imaging mode of the array-fMOST system and combined with the non-overlapping characteristics of array mouse brains, relying on the binary result after semantic segmentation, establish an algorithm for identifying the cropping frames of array mouse brains, and filter out the noise in the mouse brain image segmentation result and output a binary image, including the following steps:

[0122] S51. Filter out the noise in the segmentation result of the array mouse brain image.

[0123] The specific process of filtering out the noise in the segmentation result of the array mouse brain image is as follows:

[0124] S511. Obtain the address index of the binary image sequence of the mouse brain image segmentation result and the maximum value of the number of connected components in a single binary image;

[0125] S512. Read the image corresponding to the address index, erode the image with a 5×5 structuring element, and calculate the connected components in the image;

[0126] S513. Remove the connected components in the image that are less than 1 / 5 of the maximum value of the number of connected components, and record the total number of remaining connected components in the image;

[0127] S514. If the total number of remaining connected components is greater than the maximum value of the number of connected components, save the binary image after noise filtering.

[0128] S52. Detect the position of the mouse brain in the binary image after noise filtering.

[0129] The specific process of detecting the position of the mouse brain in the binary image after noise filtering is as follows:

[0130] S521. Read the binary image after noise filtering, calculate the minimum bounding rectangle for each connected component in it, and store the parameters in a list;

[0131] S522. If the rectangular positions stored in the list intersect with the rectangular positions of the binary image after noise filtering, merge the two rectangles and update the list parameters according to the merged result;

[0132] S523. Filter out the outliers and duplicate shear position parameters in the list;

[0133] S524. Draw the specific positions of the shear boxes on the original array of mouse brain image sequences according to the shear parameters in the list and save the images.

[0134] The complete scheme of the automatic recognition algorithm for the orientation of the array mouse brain proposed in this design was tested with 150 test sets of array mouse brain images made in the training data stage of the segmentation model. Figure 6 Some test results of the segmentation model are shown, such as Figure 6 As shown, there are 28 mouse brains in the figure, and the boxes are the shear boxes identified by our algorithm. The recognition accuracy of this algorithm for each mouse brain in the array mouse brain images can reach more than 98%.

[0135] In the description of this specification, the descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without conflict, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0136] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0137] The above specific implementation manners of the present invention do not constitute a limitation on the protection scope of the present invention. Any other corresponding changes and deformations made according to the technical concept of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. An automatic recognition method for array brain images, characterized in that, it includes the following steps: S1. Construct a segmentation dataset and use the segmentation dataset to train a segmentation model; S2. Construct a segmentation model and perform segmentation processing on mouse brain images; in step S2, constructing the segmentation model includes the following steps: S21. Construct a denoising module for denoising mouse brain images, and the denoising module is designed based on a depth difference guided filter; In step S21, the specific construction process of the denoising module is as follows: The image I with d channels as input is compressed to d / 4 channels by a 1×1 convolution 0 ; then the image resolution is reduced to half of the original through a downsampling operation to obtain the feature map I, so as to reduce the number of parameters of the module; afterwards, the feature map I passes through the depth-guided filters DGF-i with different sizes in sequence to obtain the guided images F at different scales 1 、F 2 and F 3 , and then the pairwise differences of the guided images at different scales are taken to obtain the multi-scale detail features D1, D2 and D3; a concat layer combined with a 1×1 convolution is used to fuse the obtained multi-scale detail layers, the detail-fused feature map is upsampled to the original resolution through interpolation, and is fused with the original input image in the form of a skip connection; finally, the denoising module is obtained through batch normalization processing and the PReLU non-linear activation function S22. Construct an encoding module and a decoding module for segmenting the denoised mouse brain images and outputting the segmentation results of the mouse brain images, and the encoding module and the decoding module are designed based on U-Net; S3. Train the segmentation model and input the segmentation dataset into the segmentation model for training; S4. Obtain the segmentation results of mouse brain images and use the trained segmentation model to segment mouse brain images to obtain the segmentation results of mouse brain images; S5. Establish an array mouse brain cropping frame recognition algorithm to detect and recognize the position of the mouse brain in the segmentation results of mouse brain images.

2. The automatic recognition method for array brain images according to claim 1, characterized in that, in step S1, constructing the segmentation dataset includes the following two steps: The first step is to select mouse brain images as the training set, validation set, and test set respectively, where the training set: validation set = 4:1; The second step is to perform data annotation on the mouse brain images, annotate the position of the mouse brain in the mouse brain images, and generate corresponding binary label maps.

3. The automatic recognition method for array brain images according to claim 1, characterized in that, in step S21, the specific process of the denoising module for denoising mouse brain images is as follows: S211. Perform compression and downsampling processing on the mouse brain images to obtain feature maps to reduce the number of parameters of the denoising module; S212. Pass the feature maps through depth-guided filters of different sizes in sequence to obtain guided images at different scales, and then perform pairwise differences on the guided images at different scales to obtain multi-scale detailed features; S213. Fuse the multi-scale detailed features, upsample the fused feature maps to the original resolution through interpolation, and fuse them with the original input mouse brain images in the form of skip connections, and after batch normalization processing and the PReLU non-linear activation function, realize denoising and detail enhancement of mouse brain images.

4. The automatic recognition method for array brain images according to claim 3, characterized in that, in step S21, the working principle of the depth difference guided filter is as follows: The depth difference guided filter is designed by combining the traditional guided filter principle with the convolution operation in deep learning. The key assumption of the guided filter is the local linear model between the guidance image I and the filtered output image q, where q is the linear transformation of I in the window w centered on the pixel k k in: ； The guiding image I, the image P to be guided, the images I×I and I×P are respectively passed through mean filtering box_filter to obtain mean I , mean P , corr I and corr IP , and then the variance of I and the covariance of I and P are calculated: ； ； Let a k and b k be filtered by the mean filter box_filter to obtain mean ak and mean bk ; Combining the implementation of the above principle in the convolutional neural network, replace the mean filter box_filter operation with a convolutional operation, and use the input image as both the guidance image and the image to be guided to obtain the depth guidance filter DGF-i, where i represents the i-th iteration of the guidance filter difference iteration process in the depth difference guidance filtering operation, and the scale of DGF is adjusted accordingly. The specific operation is to change the regularization parameter ε of the filter and the convolution kernel size of box_filter: the regularization parameter ε i =τ i 2 , where τ i+1 =τ i / 2 (τ 0 =0.8), the convolution kernel radius r i = 2r i-1 +1 (r 0 = 1).

5. The automatic recognition method for array brain images according to claim 1, characterized in that, in step S22, the encoding module includes a first convolutional layer and a max pooling layer. The first convolutional layer is used to extract the features of the image, and the max pooling layer is used to reduce the resolution of the image and retain the features; the decoding module includes an upsampling layer and a second convolutional layer. The upsampling layer is used to restore the feature map in the encoder to the original resolution, and the second convolutional layer is used to convert the feature map into a pixel-level segmentation result.

6. The automatic recognition method for array brain images according to claim 1, characterized in that, in step S3, training the segmentation model includes the following steps: S31. Input the training set and validation set data into the segmentation model for training; S32. Input the test set into the segmentation model for testing, and output the segmentation result as a binary image.

7. The automatic recognition method for array brain images according to claim 6, characterized in that, in step S31, inputting the training set and validation set data into the segmentation model for training is specifically as follows: Set batch_size to 6, set the number of training epochs to 300, set the initial learning rate to 0.001, adopt Adaptive Moment Estimation to adaptively adjust the learning rate during training; select CrossEntropy loss as the loss function, and update the weight parameters through the backpropagation algorithm; The calculation formula of Cross Entropy loss is: where y i represents the true value of the training sample, with the positive class being 1 and the negative class being 0, and p i represents the probability that sample i is predicted as the positive class; Print the current result every time an epoch is trained, and save the model with the minimum current loss function until the number of training epochs reaches 300 to complete the training.

8. The automatic recognition method for array brain images according to claim 1, characterized in that, in step S5, detecting the segmentation result of the mouse brain image and identifying the position of the mouse brain includes the following steps: S51. Filter the noise in the segmentation result of the mouse brain image and output a binary image; S52. Detect the position of the mouse brain in the binary image after filtering the noise to identify the position of the mouse brain.

9. The automatic recognition method for array brain images according to claim 8, characterized in that, in step S51, filtering the noise in the segmentation result of the mouse brain image and outputting a binary image includes the following steps: S511. Obtain the address index of the binary image sequence of the segmentation result of the mouse brain image and the maximum value of the number of connected components in a single image in the binary image; S512. Read the image corresponding to the address index, erode the image with a structuring element, and calculate the connected components in the image; S513. Remove the connected components in the image that are less than a preset value of the maximum value of the number of connected components, and record the total number of remaining connected components in the image; S514. If the total number of remaining connected components is greater than the maximum value of the number of connected components, save the binary image after noise filtering.

10. The automatic recognition method for array brain images according to claim 8, characterized in that, in step S52, detecting the position of the mouse brain in the binary image after filtering the noise includes the following steps: S521. Read the binary image after noise filtering, calculate the minimum bounding rectangle for each connected component therein, and store the parameters in a list; S522. If the rectangle positions stored in the list intersect with the rectangle position of the binary image after noise filtering, merge the two rectangles and update the list parameters according to the merged result; S523. Filter out the outliers and duplicate shear position parameters in the list; S524. Draw the specific position of the shear box on the original array mouse brain image sequence according to the shear parameters in the list and save the image.

Images