A method and system for generating focal length required for fused microscopic images

Abstract

The present invention relates to a method and system for generating a focal length required for a fused microscopic image, comprising a sample determination method, wherein original microscopic images collected at all positions at an initial focal length and corresponding position labels generate a sample set, which is constructed as a training data set and a test data set; a model training method, wherein the training data set and the test data set are input into a constructed neural network model for training; a method for generating a focal length required for a fused microscopic image, wherein the focal depth and number of layers required for microscopic image fusion at a certain position are output according to an original microscopic image collected at the initial focal length; a system for generating a focal length required for a fused microscopic image, comprising a microscopic image acquisition module and a focal depth and number of layers generation module, wherein the present invention can automatically generate the required focal depth and number of layers according to the position, thereby reducing the difficulty of a microscopic image fusion algorithm and improving the efficiency at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a method and system for generating a focal length required for a fused microscopic image. Background Art

[0002] Microscopic image fusion technology uses the complementary redundant information of different images collected by a microscope at different focal lengths to fuse and process them to obtain a clearer, more comprehensive and accurate microscopic image. Multi-focus image fusion technology can fuse multiple images with different focus areas in the same scene into a fully clear image, thereby effectively solving this problem and improving the information utilization of the image. Existing multi-focus fusion algorithms require users to provide specific focal depths and the number of layers required for fusion, and cannot automatically generate the required focal depths and number of layers. The algorithm is difficult, the operation is complex, and it is not flexible enough. Summary of the invention

[0003] The present invention aims to overcome at least one defect (shortcoming) of the above-mentioned prior art, provide a method and system for generating the focal length required for fused microscopic images, construct samples of a model for automatically generating the focal depth and number of layers required for fused microscopic images, and provide effective training data sets and test data sets for the subsequently constructed neural network model.

[0004] The technical solution adopted by the present invention is:

[0005] In a first aspect, a sample determination method for generating a model of a focal length required for fusion microscopic images is provided, comprising:

[0006] Preset the initial focal length, focal depth and number of layers, and set different acquisition parameter groups according to the initial focal length, focal depth and number of layers;

[0007] At the initial focal length, corresponding original microscopic images are collected according to different positions and different acquisition parameter groups, and several groups of original microscopic image sequence groups at different positions are constructed;

[0008] Performing microscopic image fusion on the original microscopic image sequences under the same acquisition parameter group in each group of original microscopic image sequence groups to obtain a number of fused images corresponding to each group;

[0009] Calculate the clarity of several fused images respectively;

[0010] According to the clarity calculation results, a fused image with the highest clarity after the microscopic image fusion is performed at each position is obtained, and the required focal depth and number of layers are determined according to the acquisition parameter group corresponding to the fused image with the highest clarity;

[0011] The focal depth and layer number corresponding to the fused image with the highest definition at each position and its corresponding position information are used to form a position label;

[0012] A sample set is generated by using the original microscopic images collected at all positions at the initial focal length and the corresponding position labels; and the sample set is divided into a training data set and a test data set.

[0013] By generating fused images with different acquisition parameter groups corresponding to different positions for clarity comparison, the fused image with the highest clarity value is selected, and the corresponding focal depth and number of layers and their corresponding position information are used to form position labels. In this way, training and test data sets for models with the required focal length for generating fused microscopic images are constructed, which is beneficial to the construction, training and use of subsequent models.

[0014] Furthermore, different acquisition parameter groups are set according to the initial focal length, focal depth and number of layers, specifically including: setting the initial focal length to Z0, the focal depth to h, and the number of layers to n, the acquisition parameter group includes [(Z0+h, Z0-h, n)], and changing h and n in the acquisition parameter group to form different acquisition parameter groups. Z0 is the distance at which a clear image can be acquired, h is the maximum defocus distance in the range, and n is the number of layers for microscopic image fusion. Acquiring the original microscopic image under the acquisition parameter group [(Z0-h, Z0+h, n] can make the data set more complete, so that the subsequent trained model can be more generalized.

[0015] Furthermore, at the initial focal length, corresponding original microscopic images are collected according to different positions and different acquisition parameter groups to construct several groups of original microscopic image sequences at different positions, specifically including:

[0016] A1, collect the original microscopic image P0 at the current collection position at the initial focal length Z0;

[0017] A2, according to the focal depth h and the number of layers n in different acquisition parameter groups, acquiring the original microscopic image sequence of the current acquisition position at different focal depths h and the number of layers n;

[0018] The acquisition position is changed, and steps A1 and A2 are executed cyclically to acquire original microscopic image sequence groups at different positions, and a total of several groups of original microscopic image sequence groups at different positions are acquired.

[0019] The original microscopic images with the same acquisition position and acquisition parameter group are constructed into an original microscopic image sequence, and the original microscopic image sequences with the same acquisition position but different acquisition parameter groups are constructed into an original microscopic image sequence group, which is beneficial to the subsequent microscopic image fusion work.

[0020] Further, the acquisition of the original microscopic image sequence at different focal depths h and layers n at the current acquisition position according to the different focal depths h and layers n in the acquisition parameter group specifically includes: acquiring n images P at the current acquisition position and at the focal length Z0+h+ {P 1+ , P 2+ ...P n+}; Collect n images P at the current acquisition position and at the focal length Z0-h - {P 1- , P 2- ...P n-}; Collect the original microscopic images P0, P + {P 1+ , P 2+ ...P n+} and P - {P 1- , P 2- ...P n-}Constructed as the original microscopic image sequence at the current acquisition position;

[0021] The 2n+1 original microscopic images with the same acquisition position and acquisition parameter group are constructed into an original microscopic image sequence, that is, the 2n+1 original microscopic images used for subsequent microscopic image fusion. The acquisition of 2n+1 original microscopic images can cover the images within the defocus range to the greatest extent, making the data richer and more complete.

[0022] Furthermore, the method of performing microscopic image fusion on the original microscopic image sequences under the same acquisition parameter group in each group of original microscopic image sequence groups to obtain a number of fused images corresponding to each group specifically includes: performing R, G, and B channel processing on each image in the original microscopic image sequence; calculating the weight matrix corresponding to each channel image in the original microscopic image sequence; performing weighted averaging on each channel image in the original microscopic image sequence according to the corresponding weight to obtain a fused image of each channel; and merging the fused images of each channel to obtain a final fused image.

[0023] Furthermore, the calculation of the weight matrix corresponding to each channel image in the original microscopic image sequence specifically includes: downsampling each channel image respectively; extracting the gradient features of each channel image after the downsampling process to obtain a gradient value; dilating the gradient value to diffuse the gradient effect; performing mean filtering on the gradient value after the dilation process; numerically scaling and exponentially calculating the gradient value after the mean filtering; and weighting the gradient value after the exponential calculation to obtain a weight matrix corresponding to each channel image.

[0024] Calculating the weight matrix is ​​the key to image fusion. Since real-time stitching needs to be taken into account during the microscopic image fusion process, the fusion efficiency must be improved. When calculating the weight matrix, the resolution reduction calculation will have a certain impact on the accuracy, and a balance needs to be found between the two. Therefore, the image needs to be downsampled. The advantage of using the expansion process is that it can diffuse the gradient effect, radiate the weights of the area with larger gradient values ​​to the neighborhood, and reduce the in-focus images that are missed due to insufficient gradient values. Since the clear images in the focal plane image do not appear as single pixels, they generally appear in blocks, and there is no mutation between the out-of-focus image and the clear image, but there is an intermediate buffer area. In the subsequent weight calculation process, due to the existence of the buffer area, the "out-of-group" phenomenon is avoided. In order to obtain a clear image, the image needs to be expanded; the mean filter is mainly for the smooth transition of the overall image; the numerical scaling and exponential calculation process is to solve the problem of which image the points in the target image take pixel values ​​from. In principle, the larger the gradient, the greater the weight. However, in actual application, the calculated gradient value is directly used as the weight, and the result is not ideal. After testing, it is found that in most cases the second largest gradient value often has side effects, resulting in the weakening of some textures. Therefore, the exponential calculation of the gradient is introduced to highlight the role of the first largest gradient value; considering that the exponential calculation sometimes causes the value to exceed the range, the gradient image is first numerically scaled when calculating the exponential value; and the weight allocation is to complete the incomplete matching ratio matrix.

[0025] Furthermore, the clarity calculation of the several fused images respectively includes: calculating a Bayer format image according to the final fused image; calculating a saturated image of the Bayer format image; obtaining candidate foreground points of the image according to the saturated image; calculating an effective foreground image according to the candidate foreground points; and calculating the clarity value of the image according to the effective foreground image.

[0026] The clarity calculation method is mainly used to calculate the clarity of the fused image in order to select the acquisition parameter group corresponding to the fused image with the highest clarity as the focal depth and number of layers required for microscopic image fusion at the acquisition position.

[0027] In a second aspect, a method for training a model for generating a focal length required for a fused microscopic image is provided, comprising: constructing a neural network model; training and testing the neural network model using the training data set and the test data set described in the first aspect to obtain a model for generating a focal length required for a fused microscopic image, wherein: an original microscopic image captured at a certain position at an initial focal length and a position label of the position are used as inputs to the neural network.

[0028] By generating fused images of different acquisition parameter groups corresponding to different positions for clarity comparison, the fused image with the highest clarity value is selected, and the corresponding acquisition parameter group is used as the acquisition parameter group required for microscopic image fusion at that position, that is, the focal depth and number of layers required for microscopic image fusion at that position are generated. The original microscopic images collected at all positions at the initial focal length and the position label of the position are used as the input of the neural network to train the neural network model, and a model for generating the focal length required for fused microscopic images can be obtained.

[0029] In a third aspect, a method for generating a focal length required for a fused microscopic image is provided, comprising: collecting an original microscopic image P1 at a position (X, Y) according to an initial focal length Z0; inputting the original microscopic image P1 into a model for generating a focal length required for a fused microscopic image as described in the second aspect; and generating a position tag corresponding to a position (X, Y) output by the model for generating the focal length required for the fused microscopic image, wherein the position tag includes a required focal depth h and a number of layers n.

[0030] The method for generating the focal length required for fused microscopic images uses a model for generating the focal length required for fused microscopic images, takes the original microscopic image collected at a certain position at the initial focal length as the input of the model, and takes the corresponding position label as the output, that is, outputs the focal depth and number of layers required for microscopic image fusion at that position. The required focal depth and number of layers are automatically generated according to the position, which reduces the difficulty of the microscopic image fusion algorithm and improves the efficiency.

[0031] In a fourth aspect, a system for generating the focal length required for fused microscopic images is provided, comprising: a microscopic image acquisition module for acquiring original microscopic images; a focal depth and layer number generation module for automatically generating the focal depth and layer number required for microscopic image fusion by generating a model of the focal length required for fused microscopic images, wherein the model for generating the focal length required for fused microscopic images is trained by the training method for generating the model of the focal length required for fused microscopic images described in the second aspect.

[0032] Optionally, the original microscopic image is collected from a stained slide cell specimen; the microscopic image collection device uses a digital microscope consisting of an optical system, an XYZ three-axis high-precision mobile platform, and an image collection camera, and the movable platform is conducive to changing the collection position.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] By constructing the original microscopic images collected at all positions at the initial focal length and the corresponding position labels to generate test data sets and training data sets for training the neural network, a model for the focal length required for generating fused microscopic images is obtained. The focal depth and number of layers required for the position can be automatically generated before the original microscopic images for microscopic image fusion are collected, which reduces the difficulty of the microscopic image fusion algorithm and improves its efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of the steps of the sample determination method according to Embodiment 1 of the present invention.

[0036] Figure 2 This is a flow chart of performing microscopic image fusion in Example 1 of the present invention, taking the number of layers n=1 as an example.

[0037] Figure 3 This is a structural diagram of the neural network model constructed in Example 2 of the present invention.

[0038] Figure 4 This is a flow chart of the method steps of Example 3 of the present invention.

[0039] Figure 5 This is a system schematic diagram of Example 4 of the present invention. DETAILED DESCRIPTION

[0040] The drawings of the present invention are only used for illustrative purposes and should not be construed as limiting the present invention. In order to better illustrate the following embodiments, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0041] Example 1

[0042] like Figure 1 As shown, this embodiment provides a sample determination method for generating a model of the focal length required for fused microscopic images, comprising:

[0043] S1, preset the initial focal length, focal depth and number of layers, and set different acquisition parameter groups according to the initial focal length, focal depth and number of layers;

[0044] S2. Under the initial focal length, corresponding original microscopic images are collected according to different positions and different acquisition parameter groups, and several groups of original microscopic image sequence groups at different positions are constructed;

[0045] S3, performing microscopic image fusion on the original microscopic image sequences under the same acquisition parameter group in each group of original microscopic image sequence groups, to obtain a number of fused images corresponding to each group;

[0046] S4, respectively calculating the clarity of a plurality of fused images;

[0047] S5. Obtain the fused image with the highest definition after the microscopic images are fused at each position according to the definition calculation result, and determine the required focal depth and number of layers according to the acquisition parameter group corresponding to the fused image with the highest definition;

[0048] S6, forming a position label with the focal depth and the number of layers corresponding to the fused image with the highest definition at each position and the corresponding position information;

[0049] S7, generating a sample set of original microscopic images collected at all positions at the initial focal length and corresponding position labels;

[0050] S8. Divide the sample set into a training data set and a test data set.

[0051] S1 of this embodiment sets different acquisition parameter groups according to the initial focal length, focal depth and number of layers, specifically including: setting the initial focal length to Z0, the focal depth to h, and the number of layers to n, the acquisition parameter group includes [(Z0+h, Z0-h, n)], and changing h and n in the acquisition parameter group to form different acquisition parameter groups.

[0052] In the specific implementation process, the number of layers n of this embodiment can be set to 1 layer, 3 layers, or 5 layers, and the focal depth h can be set to 1um, 2um, 3um, 4um, or 5um, so the total number of acquisition parameter groups is 15 groups.

[0053] S2 of this embodiment captures corresponding original microscopic images at different positions and according to different acquisition parameter groups at the initial focal length, and constructs a plurality of original microscopic image sequence groups at different positions, specifically including:

[0054] A1, collect the original microscopic image P0 at the current collection position at the initial focal length Z0;

[0055] A2, according to the focal depth h and the number of layers n in different acquisition parameter groups, acquiring the original microscopic image sequence of the current acquisition position at different focal depths h and the number of layers n;

[0056] A3, changing the acquisition position, and cyclically executing steps A1 and A2 to acquire original microscopic image sequence groups at different positions, and acquiring a total of several groups of original microscopic image sequence groups at different positions.

[0057] Since the total number of acquisition parameter groups in this embodiment is 15, there are 15 original microscopic image sequences acquired at the same position.

[0058] The method described in A2 of this embodiment, according to different focal depths h and number of layers n in the acquisition parameter group, acquiring the original microscopic image sequence at the current acquisition position at different focal depths h and number of layers n, specifically includes:

[0059] A201, collect n pictures P at the current collection position and at the focal length Z0+h + {P 1+ , P 2+ ...P n+}, if n=1, then collect 1 picture, if n=3, then collect 3 pictures, and so on;

[0060] A202, collect n pictures P at the current collection position and at the focal length Z0-h - {P 1- , P 2- ...P n-};

[0061] A203, collect the original microscopic images P0, P + {P 1+ , P 2+ ...P n+} and P - {P 1- , P 2- ...P n-} is constructed as a sequence of original microscopic images at the current acquisition position.

[0062] In this embodiment, each original microscopic image sequence corresponds to an acquisition position information and an acquisition parameter group. One acquisition position corresponds to one original microscopic image sequence under the same acquisition parameter group. Each original microscopic image sequence includes 2n+1 original microscopic images, the specific number of which is determined by the number of layers n. These 2n+1 original microscopic images are also the 2n+1 images used for subsequent image fusion. As mentioned above, when the acquisition parameter group includes 15 groups, one acquisition position corresponds to 15 acquisition parameter groups, and one acquisition position will correspond to 15 original microscopic image sequences, and 15 original microscopic image sequences constitute one original microscopic image sequence group.

[0063] In step A3, after modifying the acquisition position, steps A1 and A2 are executed again, and the original microscopic image sequence corresponding to the modified acquisition position is obtained. In the implementation process, different values ​​of the acquisition position can be set according to actual needs, and then steps A1 and A2 are cyclically executed through different acquisition positions to obtain the original microscopic image sequence groups corresponding to different positions, and a total of several groups of original microscopic image sequence groups at different positions are collected.

[0064] In this embodiment S3, the original microscopic image sequences under the same acquisition parameter group in each group of original microscopic image sequence groups are respectively subjected to microscopic image fusion to obtain a number of fused images corresponding to each group, specifically including:

[0065] S301, perform R, G, B channel processing on each image in the original microscopic image sequence; in specific implementation, the original microscopic image sequence refers to the original microscopic image sequence corresponding to each acquisition position under the same acquisition parameter group, then there are a total of 2n+1 original microscopic images, and each of the 2n+1 original microscopic images is processed by R, G, B channels to obtain each channel image corresponding to each of the 2n+1 original microscopic images; when there are 15 groups of acquisition parameter groups, the original microscopic image sequence corresponding to each acquisition parameter group is processed by channel.

[0066] S302, calculating the weight matrix corresponding to each channel image in the original microscopic image sequence;

[0067] S303, weighted average each image of each channel image in the original microscopic image sequence by corresponding weights, to obtain a fused image of each channel; specifically, the original microscopic image sequence includes 2n+1 original microscopic images, then there are 2n+1 channel images corresponding to the three channels R, G, and B, and the corresponding weights in the 2n+1 channel images of each channel are weighted averaged

[0068] S304, merging the fused images of each channel to obtain a final fused image. In step S303, each channel corresponds to a fused image, so the three fused images corresponding to the three channels are merged to obtain the final fused image.

[0069] like Figure 2 As shown, taking the number of layers n=1 as an example, 3 images in the original microscopic image sequence are fused. In this embodiment, the original microscopic image sequences under the same acquisition parameter group in each group of original microscopic image sequence groups are fused respectively, and 15 fused images can be obtained for each group.

[0070] The calculation of the weight matrix corresponding to each channel image in the original microscopic image sequence in S302 of this embodiment specifically includes:

[0071] S3021, downsampling the images of each channel respectively;

[0072] S3022, extracting the gradient features of each channel image after downsampling processing to obtain a gradient value;

[0073] S3023, dilating the gradient value using expansion processing to diffuse the gradient effect;

[0074] S3024, performing mean filtering on the gradient value after the expansion process;

[0075] S3025, performing numerical scaling and exponential calculation on the gradient value after mean filtering;

[0076] S3026. Perform weight assignment on the gradient values ​​after exponential calculation to obtain a weight matrix corresponding to each channel image.

[0077] In specific implementation, this embodiment can downsample by 4 times when downsampling each channel image separately; the gradient features of each channel image after downsampling are extracted using the laplacian function of opencv, and the gradient feature extraction is performed using the CV_32FC1 format, and the absolute value of the obtained gradient feature is taken as the final gradient feature; the numerical scaling is performed by multiplying the value of each point by 0.1; and the exponential calculation takes the R channel as an example, assuming that the gradient image obtained by the R channel is T, the corresponding exponential calculation is R_new=exp(T).

[0078] The clarity calculation of the plurality of fused images is performed separately as described in S4 of this embodiment, specifically including:

[0079] S401, calculating and obtaining a Bayer format image according to the final fused image;

[0080] S402, calculating a saturated image of the bayer format image;

[0081] S403, obtaining candidate foreground points of the image according to the saturated image;

[0082] S404, calculating a valid foreground image according to the candidate foreground points;

[0083] S405: Calculate the clarity value of the image according to the effective foreground image.

[0084] In this embodiment, the clarity of the 15 fused images obtained at each position is calculated, and the fused image with the highest clarity is selected as the best result of microscopic image fusion at that position, and the focal depth and number of layers in the corresponding position label are used as the focal depth and number of layers required for microscopic image fusion at that position. In this way, the original microscopic images collected at all positions at the initial focal length and the corresponding position labels are generated into a sample set as the training data set and test data set required for subsequent training of the neural network model.

[0085] Example 2

[0086] This embodiment provides a training method for generating a model of the focal length required for fusion microscopic images, including:

[0087] S9, build a neural network model;

[0088] S10. Use the training data set and the test data set described in Example 1 to train and test the neural network model to obtain a model for generating the focal length required for the fused microscopic image, wherein: the original microscopic image collected at a certain position at the initial focal length and the position label of the position are used as inputs to the neural network.

[0089] Since the neural network model constructed in this embodiment needs to predict the focal depth and the number of layers at the same time, the deep learning model used in this embodiment is a new Multilabel classification model.

[0090] The specific structure of the neural network model constructed in S9 of this embodiment can be as follows: Figure 3 The structure implementation shown in the figure assumes that the input size of the original microscopic image is 480×480×32. The first layer is a Conv2D convolution layer with input and output of (32,480,480). A ReLU activation layer is added with input and output of (32,480,480). Then a second Conv2D convolution layer is connected with input of (32,480,480) and output of (32,98,98). A ReLU activation is added with input and output of (32,98,98). Then a MaxPooling2D is connected with input of (32,98,98) and output of (32,49,49). A Dropout is connected. Layer, the input and output are both (32, 49, 49); then connect to the third Conv2d convolution layer, the input is (32, 49, 49), and the output is (64, 49, 49); then connect to a ReLU activation, the input and output are both (64, 49, 49); then connect to the fourth Conv2D convolution layer, the input is (64, 49, 49) and the output is (64, 47, 47); then connect to a ReLU activation, the input and output are both (64, 47, 47); Then connect a maximum pooling MaxPooling2D, with an input of (64,47,47) and an output of (64,23,23); then connect a Dropout layer, with both input and output of (64,23,23); then connect a Flatten layer, with an input of (64,23,23) and an output vector of size (33865); then connect a Dense layer, with an input of (33865) and an output vector of size (512); then connect a ReLU activation, with both input and output of (512) vectors; then connect a Dropout layer, with both input and output of (512) vectors; then connect a Dense layer, with an input of (512) vectors and an output vector of size (15); finally connect a Sigmoid activation layer, and the probability of each combination of focal depth h and number of layers n can be obtained.

[0091] The training method adopted by S10 in this embodiment can be: using the Tensorflow environment installed on the Windows 10 system to train the neural network. The specific method is: input the training data set generated by S1-S8 into the neural network model constructed by S9, complete the forward propagation of the network, and finally use Sigmoid to complete the classification output of the position label. During the training process, the initial learning rate can be set to 0.000001, and the learning rate can be gradually increased during the training process. Specifically, the training can be performed by changing the value of Warm_Up and the initial value of the learning rate to obtain a trained lightweight deep neural network model.

[0092] The test method adopted in S10 of this embodiment is to use the test data set generated in steps S1-S8 as the input of the trained lightweight deep neural network, and output the position labels corresponding to the test data set after passing through the model, and each position label corresponds to a different focal depth h and number of layers n. After the lightweight deep neural network model passes the test, a model of the focal length required for generating the fused microscopic image can be obtained.

[0093] Example 3

[0094] like Figure 4 As shown, this embodiment provides a method for generating a focal length required for a fused microscopic image, comprising:

[0095] S11, collecting the original microscopic image P1 at the position (X, Y) according to the initial focal length Z0;

[0096] S12, inputting the original microscopic image P1 into the model for generating the focal length required for the fused microscopic image as described in Example 2;

[0097] S13, generating a position tag corresponding to the model output position (X, Y) of the focal length required for the fused microscopic image, wherein the position tag includes the required focal depth h and the number of layers n.

[0098] The method for generating the focal length required for fused microscopic images provided in this embodiment uses the model for generating the focal length required for fused microscopic images trained in Example 2, takes the original microscopic image collected at a certain position at the initial focal length as the input of the model, and takes the corresponding position label as the output, that is, outputs the focal depth and number of layers required for microscopic image fusion at the position. The required focal depth and number of layers are automatically generated according to the position, which reduces the difficulty of the microscopic image fusion algorithm and improves the efficiency.

[0099] In specific applications, the focal depth h1 and the number of layers n1 required for the current acquisition position output by S13 can be applied to microscopic image fusion, specifically:

[0100] Collect n1 images P at the current acquisition position and at the focal length Z0+h1 + {P 1+ , P 2+ ...P n1+};

[0101] Collect n1 images P at the current acquisition position and at the focal length Z0-h1 - {P 1- , P 2- ...P n1-};

[0102] The collected original microscopic images P1, P + {P 1+ , P 2+ ...P n1+} and P - {P 1- , P 2- ...P n1-}Constructed as the original microscopic image sequence at the current acquisition position;

[0103] The 2n1+1 images in the original microscopic image sequence at the current acquisition position are fused to obtain a fused image at the current acquisition position.

[0104] Example 4

[0105] like Figure 5 As shown, this embodiment provides a system for generating a focal length required for a fused microscopic image, comprising:

[0106] Microscopic image acquisition module 101, used for acquiring original microscopic images;

[0107] The focal depth and layer number generation module 102 is used to automatically generate the focal depth and layer number required for microscopic image fusion by generating a model of the focal length required for fused microscopic images. The model for generating the focal length required for fused microscopic images is trained by the method described in Example 2.

[0108] The system for generating the focal length required for the fused microscopic image provided in this embodiment uses the method for generating the focal length required for the fused microscopic image provided in Example 3. The acquisition object of the original microscopic image in this embodiment is a stained glass slide cell specimen; the microscopic image acquisition device adopts a digital microscope composed of an optical system, an XYZ three-axis high-precision mobile platform and an image acquisition camera, and the movable platform is conducive to changing the acquisition position.

[0109] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A sample determination method for generating a model of the focal length required for fusion microscopic images, characterized in that: include: Preset the initial focal length, focal depth and number of layers, and set different acquisition parameter groups according to the initial focal length, focal depth and number of layers; At the initial focal length, corresponding original microscopic images are collected according to different positions and different acquisition parameter groups, and several groups of original microscopic image sequence groups at different positions are constructed; The original microscopic image sequences under the same acquisition parameter group in each group of original microscopic image sequence groups are respectively subjected to microscopic image fusion to obtain a number of fused images corresponding to each group, specifically including: Perform R, G, and B channel processing on each image in the original microscopic image sequence; Calculate the weight matrix corresponding to each channel image in the original microscopic image sequence, including: Downsample each channel image separately; Extract the gradient features of each channel image after downsampling processing to obtain the gradient value; The gradient value is treated with dilation to diffuse the gradient effect; Perform mean filtering on the gradient value after dilation processing; Numerical scaling and exponential calculation of the gradient value after mean filtering; Weights are assigned to the gradient values ​​after exponential calculation to obtain the weight matrix corresponding to each channel image; Each image of each channel in the original microscopic image sequence is weighted averaged by the corresponding weight value to obtain a fused image of each channel; Merge the fused images of each channel to obtain the final fused image; perform clarity calculation on several fused images respectively; According to the clarity calculation results, a fused image with the highest clarity after the microscopic image fusion is performed at each position is obtained, and the required focal depth and number of layers are determined according to the acquisition parameter group corresponding to the fused image with the highest clarity; The focal depth and layer number corresponding to the fused image with the highest definition at each position and its corresponding position information are used to form a position label; Generate a sample set by using the original microscopic images collected at all positions at the initial focal length and the corresponding position labels; The sample set is divided into training data set and test data set.

2. The sample determination method for generating a model of the focal length required for fusion microscopic images according to claim 1, characterized in that: Set different acquisition parameter groups according to the initial focal length, focal depth and number of layers, including: Assume that the initial focal length is Z0, the focal depth is h, and the number of layers is n. The acquisition parameter group includes [(Z0+h, Z0-h, n)]. Different acquisition parameter groups are formed by changing h and n in the acquisition parameter group.

3. The sample determination method for generating a model of the focal length required for fusion microscopic images according to claim 2, characterized in that: At the initial focal length, the corresponding original microscopic images are collected according to different positions and different acquisition parameter groups, and several groups of original microscopic image sequence groups at different positions are constructed, specifically including: A1, collect the original microscopic image P0 at the current collection position at the initial focal length Z0; A2, according to the focal depth h and the number of layers n in different acquisition parameter groups, acquiring the original microscopic image sequence of the current acquisition position at different focal depths h and the number of layers n; The acquisition position is changed, and steps A1 and A2 are executed cyclically to acquire original microscopic image sequence groups at different positions, and a total of several groups of original microscopic image sequence groups at different positions are acquired.

4. The sample determination method for generating a model of the focal length required for fusion microscopic images according to claim 3, characterized in that: According to the focal depth h and the number of layers n in different acquisition parameter groups, the original microscopic image sequence at the current acquisition position at different focal depths h and the number of layers n is acquired, specifically including: Collect n pictures P+{P1+, P2+...Pn+} at the current acquisition position and at the focal length Z0+h; Collect n pictures P-{P1-, P2-...Pn-} at the current acquisition position and at the focal length Z0-h; The collected original microscopic images P0, P+{P1+, P2+...Pn+} and P-{P1-, P2-...Pn-} are constructed as an original microscopic image sequence at the current collection position.

5. The sample determination method for generating a model of the focal length required for fused microscopic images according to claim 1, characterized in that: The clarity calculation of the plurality of fused images respectively includes: The Bayer format image is calculated based on the final fused image; Calculate the saturated image of the bayer format image; Obtain candidate foreground points of the image according to the saturated image; Calculate a valid foreground image based on candidate foreground points; The clarity value of the image is calculated based on the effective foreground image.

6. A training method for generating a model of focal length required for fusion microscopic images, characterized in that: include: Build a neural network model; The neural network model is trained and tested using the training data set and the test data set described in any one of claims 1 to 5 to obtain a model for generating the focal length required for the fused microscopic image, wherein: The original microscopic image collected at a certain position at the initial focal length and the position label of the position are used as the input of the neural network.

7. A method for generating a focal length required for a fused microscopic image, characterized in that: include: According to the initial focal length Z0, the original microscopic image P1 of the acquisition position (X, Y); Inputting the original microscopic image P1 into the model for generating the focal length required for the fused microscopic image as described in claim 6; Generate a position tag corresponding to the model output position (X, Y) of the focal length required for the fused microscopic image, wherein the position tag includes the required focal depth h and the number of layers n.

8. A system for generating a focal length required for fused microscopic images, characterized in that: include: A microscopic image acquisition module, used for acquiring original microscopic images; A focal depth and layer number generation module is used to automatically generate the focal depth and layer number required for microscopic image fusion by generating a model of the focal length required for fused microscopic images, wherein the model for generating the focal length required for fused microscopic images is trained by the method described in claim 6.

Images