A method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample

Abstract

The present invention provides a method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample. The method includes: acquiring a three-dimensional image data set; performing in-slice registration and splicing on image data blocks within the three-dimensional original slice data; sampling the image data blocks of the three-dimensional standard slice data to obtain a plurality of sub-image data blocks; registering adjacent three-dimensional standard slice data to obtain displacement parameters of paired sub-image data blocks; using the displacement parameters of the paired sub-image data blocks to obtain the upper surface deformation field of the three-dimensional standard slice data; based on the upper surface deformation field of the three-dimensional standard slice data, obtaining mutually matching upper and lower surface data between adjacent three-dimensional standard slice data, performing inter-slice surface image registration on the adjacent three-dimensional standard slice data to obtain an inter-slice registration two-dimensional deformation field; expanding the inter-slice registration two-dimensional deformation field into a three-dimensional deformation field, and applying the expanded three-dimensional deformation field to the three-dimensional image data set for three-dimensional data reconstruction.

Description

Technical Field

[0001] The present invention relates to the technical field of biological image processing, and particularly relates to a method for reconstructing three-dimensional microscopic images of three-dimensional biological samples, an electronic device, and a storage medium. Background Art

[0002] Due to the steady development of 3D microscopy technology in the past few decades, the acquisition of large datasets is no longer a limiting factor for the analysis of large specimens. The latest technologies such as two-photon microscopy have extended the ability of confocal microscopy to image through thicker tissues, and rapid acquisition can be achieved through spinning disk or slit-scanning confocal microscopes. In addition, the objective lens provides an almost distortion-free image across the entire field of view. Sections are obtained by sequentially moving the microscope stage to a new position (partially overlapping with the previous position) and performing high-resolution 3D local imaging of the sample. For example, the large-sample high-speed fluorescence microscopy imaging system (Blockface-VISoR) used in the present invention can complete whole-body imaging of three-dimensional large-scale biological samples (such as experimental mice) with a resolution of 1×1×2.5μm3 voxels. Compared with confocal microscopes and two-photon fluorescence microscopes, the imaging speed of Blockface-VISoR has increased by several hundred to thousands of times, making it a powerful tool for analyzing biological structures and functions. Blockface-VISoR scans three-dimensional large-scale biological samples along an "S" trajectory, and can obtain image data with a thickness of 600-700 microns at the top layer of the sample in a single scan; after a single scan is completed, the microtome cuts off a 400-micron-thick sample along the horizontal direction, and then performs the next scan. Due to the characteristics of whole-body imaging of three-dimensional large-scale biological samples, the image data of two adjacent sections have overlapping image regions, and the overlapping thickness is 200-300 microns. In order to obtain a panoramic view and high-resolution imaging of three-dimensional large-scale biological samples, a reconstruction algorithm for high-speed fluorescence microscopy is required to accurately stitch multiple image data. However, the imaging process of Blockface-VISoR will bring multiple problems: for example, the cutting of the sample by the microtome will cause non-rigid deformation of the sample surface; due to the displacement error in the vertical direction, the thickness of the overlapping region of each pair of adjacent sections will be different; due to the attenuation of the laser passing through the sample, the brightness within a single section of data is non-uniform, and the image data near the light source is brighter and clearer, and the inconsistent data brightness brings difficulties to the reconstruction.

[0003] In addition, existing tools either cannot be directly used for the reconstruction of whole-body data of three-dimensional large-scale biological samples by Blockface-VISoR, or lack a registration method for overlapping data between sections. Summary of the Invention

[0004] In view of the above problems, the present invention provides a method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample, an electronic device, and a storage medium, in order to at least solve one of the above problems.

[0005] According to a first aspect of the present invention, there is provided a method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample, including:

[0006] Using microscopic imaging technology to obtain a three-dimensional image dataset of a three-dimensional biological sample, wherein the three-dimensional image dataset of the three-dimensional biological sample includes a plurality of three-dimensional original slice data, and each three-dimensional original slice data includes a plurality of image data blocks;

[0007] Using a rigid registration algorithm based on normalized cross-correlation information to perform intra-slice registration and stitching on the image data blocks within each three-dimensional original slice data, obtaining a plurality of three-dimensional standard slice data;

[0008] Using a sub-region sampling method to sample the image data blocks of each three-dimensional standard slice data, obtaining a plurality of sub-image data blocks from each three-dimensional standard slice data, and numbering the plurality of sub-image data blocks;

[0009] Using a displacement registration algorithm based on normalized cross-correlation information to register the sub-image data blocks with the same label in adjacent three-dimensional standard slice data, obtaining the displacement parameters of the paired sub-image data blocks;

[0010] Using the displacement parameters of the paired sub-image data blocks, performing parameter fitting by an interpolation method to obtain the upper surface deformation field of each three-dimensional standard slice data;

[0011] Based on the upper surface deformation field of each three-dimensional standard slice data, obtaining the mutually matching upper and lower surface data between adjacent three-dimensional standard slice data, and using a non-rigid deformation method to perform inter-slice surface image registration on the adjacent three-dimensional standard slice data, obtaining an inter-slice registration two-dimensional deformation field;

[0012] Using a linear interpolation method to expand the inter-slice registration two-dimensional deformation field into a three-dimensional deformation field, and applying the expanded three-dimensional deformation field to the three-dimensional image dataset of the three-dimensional biological sample, thereby realizing the three-dimensional data reconstruction of the three-dimensional biological sample.

[0013] According to an embodiment of the present invention, the above-mentioned using microscopic imaging technology to obtain a three-dimensional image dataset of a three-dimensional biological sample includes:

[0014] Using a large-sample high-speed fluorescence microscopic imaging system to perform row-by-row continuous scanning on the three-dimensional biological sample according to a preset scanning mode, obtaining a three-dimensional image block with a preset sample thickness for each row;

[0015] After completing the entire planar scan of the three-dimensional biological sample, according to the preset slice thickness, use a slicing machine to perform sample slicing on each three-dimensional image block with a preset sample thickness per row;

[0016] Repeat the row-by-row continuous scanning operation and the sample slicing operation until the overall scan of the three-dimensional biological sample is completed, and all three-dimensional image data sets of the three-dimensional biological sample are obtained.

[0017] According to an embodiment of the present invention, the above-mentioned use of the rigid registration algorithm based on normalized cross-correlation information to perform in-slice registration and stitching on the image data blocks in each three-dimensional original slice data, and obtain multiple three-dimensional standard slice data, including:

[0018] For two adjacent image data blocks in the three-dimensional original slice data, take the first image data block as the fixed image data block, take the second image data block as the moving image data block, and initialize the parameter matrix of the rigid registration algorithm. Among them, the parameter matrix of the rigid registration algorithm includes the rotation center, the rotation matrix, and the displacement matrix;

[0019] Use the parameter matrix of the rigid registration algorithm to obtain the deformed moving image data block, and calculate the normalized cross-correlation information between the deformed moving image data block and the fixed image data block;

[0020] Use the adaptive gradient descent algorithm to update the parameter matrix of the rigid registration algorithm to minimize the normalized cross-correlation information, and based on the updated parameter matrix of the rigid registration algorithm, perform a stitching operation on the image data blocks in the three-dimensional original slice data to obtain three-dimensional standard slice data.

[0021] According to an embodiment of the present invention, the above-mentioned use of the sub-region sampling method to sample the image data blocks of each three-dimensional standard slice data, and obtain multiple sub-image data blocks from each three-dimensional standard slice data, including:

[0022] Obtain the slice thickness of each image data block in the three-dimensional standard slice data after the in-slice stitching operation, and calculate the thickness of the overlapping region of each image data block in the three-dimensional standard slice data according to the preset image data block thickness value;

[0023] According to the predefined sampling interval and sampling size, start sampling the image data blocks in the three-dimensional standard slice data at the predefined sampling interval from the position of the predefined initial reference image data block to obtain multiple sub-image data blocks with the predefined sampling size.

[0024] According to an embodiment of the present invention, the above-mentioned use of the displacement registration basis based on normalized cross-correlation information to register the sub-image data blocks with the same label in adjacent three-dimensional standard slice data, and obtain the displacement parameters of the paired sub-image data blocks, including:

[0025] In adjacent three-dimensional standard slice data, take the sub-image data block of the first three-dimensional standard slice data as the fixed sub-image data block, and take the sub-image data block in the second three-dimensional standard slice data that has the same label as the fixed sub-image data block as the moving sub-image data block;

[0026] Use the normalized cross-correlation information and the adaptive gradient descent algorithm to calculate the displacement parameters between the fixed sub-image data block and the moving sub-image data block.

[0027] According to an embodiment of the present invention, the above-mentioned obtaining the upper surface deformation field of each three-dimensional standard slice data by parameter fitting through an interpolation method using the displacement parameters of the paired sub-image data blocks includes:

[0028] Take the center point of the moving sub-image data block as the control point of the interpolation method, set the displacement parameters of the paired sub-image data blocks as the displacement values to be fitted by the interpolation method, and use the interpolation function used by the interpolation method to fit the upper surface deformation field of each three-dimensional standard slice data.

[0029] According to an embodiment of the present invention, the above-mentioned obtaining the mutually matching upper and lower surface data between adjacent three-dimensional standard slice data based on the upper surface deformation field of each three-dimensional standard slice data, and using the non-rigid deformation method to perform inter-slice surface image registration on the adjacent three-dimensional standard slice data, and obtaining the inter-slice registration two-dimensional deformation field includes:

[0030] Based on the upper surface deformation field of each three-dimensional standard slice data, obtain the two-dimensional upper surface image of the three-dimensional standard slice data, and take the bottom surface of the adjacent previous three-dimensional standard slice data as the two-dimensional lower surface image;

[0031] Use the non-rigid deformation method to perform inter-slice surface image registration on the two-dimensional upper surface image in the three-dimensional standard slice data and the two-dimensional lower surface image in the adjacent previous three-dimensional standard slice data to obtain the inter-slice registration two-dimensional deformation field.

[0032] According to an embodiment of the present invention, the above-mentioned using the linear interpolation method to expand the inter-slice registration two-dimensional deformation field into a three-dimensional deformation field, and applying the expanded three-dimensional deformation field to the three-dimensional image dataset of the three-dimensional biological sample, and further realizing the three-dimensional data reconstruction of the three-dimensional biological sample includes:

[0033] Use the linear interpolation method to expand the inter-slice registration two-dimensional deformation field in the vertical direction to obtain the three-dimensional deformation field of adjacent three-dimensional standard slice data;

[0034] Deform each three-dimensional standard slice data using a three-dimensional deformation field, and merge the three-dimensional standard slice data with inter-slice stitching, so that the cells and tissue structures on the upper and lower surfaces of adjacent three-dimensional standard slice data are continuous, thereby obtaining a three-dimensional image of the reconstructed three-dimensional biological sample.

[0035] According to a second aspect of the present invention, there is provided an electronic device, comprising:

[0036] One or more processors;

[0037] A storage device for storing one or more programs,

[0038] wherein, when the one or more programs are executed by the one or more processors, the one or more processors are caused to execute a method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample.

[0039] According to a third aspect of the present invention, there is provided a computer-readable storage medium having executable instructions stored thereon, which when executed by a processor cause the processor to execute a method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample.

[0040] The method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample provided by the present invention utilizes an interpolation method, a rigid transformation method, and a non-rigid deformation method to achieve high-precision image stitching; and during the image stitching process, the images of different slices can be accurately aligned, reducing the discontinuity problem between images, thereby providing high-quality three-dimensional images. At the same time, the present invention adopts displacement registration between adjacent slices, which can overcome the problem of inconsistent thickness of the overlapping area caused by displacement errors in the vertical direction, improving the continuity and consistency between different slices; then, by using interpolation and non-rigid deformation algorithms, the non-rigid deformation caused by sample cutting by a slicing machine can be effectively overcome, enabling the images between adjacent slices to be more accurately matched. The present invention successfully stitches and reconstructs the image data of adjacent slices, obtaining high-resolution three-dimensional data of a three-dimensional biological sample; the reconstruction algorithm for high-speed fluorescence microscopy can greatly improve the research efficiency, enabling those skilled in the art to obtain high-quality image data faster to deeply study biological processes and cell tissue structures, contributing to the research progress in the field of life sciences. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flowchart of a method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to an embodiment of the present invention;

[0042] Figure 2 is a flowchart of obtaining a three-dimensional image dataset of a three-dimensional biological sample according to an embodiment of the present invention;

[0043] Figure 3Schematic diagram of obtaining in-slice image data blocks by scanning a mouse sample with Blockface-VISoR according to an embodiment of the present invention;

[0044] Figure 4 Schematic diagram of the overlapping part between adjacent slice data according to an embodiment of the present invention;

[0045] Figure 5 Schematic diagram of the original image data block in the nth slice data according to an embodiment of the present invention;

[0046] Figure 6 Flow chart of obtaining multiple three-dimensional standard slice data according to an embodiment of the present invention;

[0047] Figure 7 Maximum projection diagram of the xoy plane of the nth slice data according to an embodiment of the present invention;

[0048] Figure 8 Schematic diagram of sub-region sampling according to an embodiment of the present invention;

[0049] Figure 9 Schematic diagram of the result of fitting control points according to an embodiment of the present invention;

[0050] Figure 10 Schematic diagram of two-dimensional interpolation result according to an embodiment of the present invention;

[0051] Figure 11 Three-dimensional surface rendering diagram generated by Matlab according to an embodiment of the present invention;

[0052] Figure 12 Schematic diagram of comparison between the image corrected by B-spline deformation and the image not corrected by the algorithm according to an embodiment of the present invention;

[0053] Figure 13 Schematic diagram of in-slice stitching work according to an embodiment of the present invention;

[0054] Figure 14 Whole-body data diagram of a whole-body mouse sample rendered by Imaris software according to an embodiment of the present invention;

[0055] Figure 15 Schematically shows a block diagram of an electronic device suitable for implementing a method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to an embodiment of the present invention. Detailed implementation manners

[0056] In order to make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to specific embodiments and the accompanying drawings.

[0057] In order to overcome various technical problems in the prior art and provide a three-dimensional image reconstruction algorithm capable of accurately splicing slice data, the present invention provides a method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample. By using the overlapping regions of adjacent slice data, the problems of non-rigid deformation and registration and splicing of the connection surfaces of adjacent slices are solved, facilitating better observation of the cells and / or tissue structures of three-dimensional biological samples by those skilled in the art.

[0058] It should be specifically noted that the source of the three-dimensional biological samples involved in the present invention complies with the provisions of relevant laws and regulations, and under the requirements of relevant laws and regulations, research, processing, and preservation of the above-mentioned three-dimensional biological samples are carried out.

[0059] Figure 1 is a flowchart of a method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to an embodiment of the present invention.

[0060] As Figure 1 shown, the method for reconstructing a three-dimensional microscopic image of the above-mentioned three-dimensional biological sample includes operations S110 to S170.

[0061] In operation S110, a three-dimensional image dataset of a three-dimensional biological sample is obtained by using microscopic imaging technology. Among them, the three-dimensional image dataset of the three-dimensional biological sample includes a plurality of three-dimensional original slice data, and each three-dimensional original slice data includes a plurality of image data blocks.

[0062] The above-mentioned three-dimensional biological samples generally refer to three-dimensional large-scale biological samples, such as laboratory mice, frogs, rabbits, etc., or tissues of animals, such as the brain tissues of laboratory monkeys, etc.

[0063] In operation S120, a rigid registration algorithm based on normalized cross-correlation information is used to perform intra-slice registration and splicing on the image data blocks within each three-dimensional original slice data, obtaining a plurality of three-dimensional standard slice data.

[0064] In operation S130, a sub-region sampling method is used to sample the image data blocks of each three-dimensional standard slice data, obtaining a plurality of sub-image data blocks from each three-dimensional standard slice data, and numbering the plurality of sub-image data blocks.

[0065] In operation S140, a displacement registration algorithm based on normalized cross-correlation information is used to register the sub-image data blocks with the same label in adjacent three-dimensional standard slice data, obtaining the displacement parameters of the paired sub-image data blocks.

[0066] In operation S150, using the displacement parameters of the paired sub-image data blocks, parameter fitting is performed by an interpolation method to obtain the upper surface deformation field of each three-dimensional standard slice data.

[0067] The above interpolation method is optionally a B-spline interpolation method.

[0068] In operation S160, based on the upper surface deformation field of each three-dimensional standard slice data, the upper and lower surface data that match each other between adjacent three-dimensional standard slice data are obtained, and the non-rigid deformation method is used to perform inter-slice surface image registration on the adjacent three-dimensional standard slice data to obtain an inter-slice registration two-dimensional deformation field.

[0069] The above non-rigid deformation method is optionally a B-Spline non-rigid deformation method.

[0070] In operation S170, the inter-slice registration two-dimensional deformation field is extended into a three-dimensional deformation field by using a linear interpolation method, and the extended three-dimensional deformation field is applied to the three-dimensional image dataset of the three-dimensional biological sample, thereby realizing the three-dimensional data reconstruction of the three-dimensional biological sample.

[0071] The method for reconstructing the three-dimensional microscopic image of the three-dimensional biological sample provided by the present invention uses an interpolation method, a rigid transformation method, and a non-rigid deformation method to achieve high-precision image stitching; and during the image stitching process, the images of different slices can be accurately aligned, reducing the discontinuity problem between images, thereby providing high-quality three-dimensional images. At the same time, the present invention adopts displacement registration between adjacent slices, which can overcome the problem of inconsistent thickness of the overlapping region caused by displacement errors in the vertical direction, which improves the continuity and consistency between different slices; then, by using interpolation and non-rigid deformation algorithms, the non-rigid deformation caused by the slicer cutting the sample can be effectively overcome, which enables the images between adjacent slices to be more accurately matched. The present invention successfully stitches and reconstructs the image data of adjacent slices to obtain high-resolution three-dimensional data of the three-dimensional biological sample; the reconstruction algorithm of high-speed fluorescence microscopy can greatly improve the research efficiency, enabling those skilled in the art to obtain high-quality image data faster to deeply study biological processes and cell tissue structures, which helps to promote the research progress in the field of life sciences.

[0072] Figure 2 It is a flowchart of obtaining a three-dimensional image dataset of a three-dimensional biological sample according to an embodiment of the present invention.

[0073] As Figure 2 shown, the above method for obtaining a three-dimensional image dataset of a three-dimensional biological sample by using a microscopy imaging technique includes operations S210 to S230.

[0074] In operation S210, the three-dimensional biological sample is scanned row by row continuously according to a preset scanning mode by using a large-sample high-speed fluorescence microscopy system to obtain three-dimensional image blocks with a preset sample thickness for each row.

[0075] After completing the entire planar scan of the three-dimensional biological sample in operation S220, according to a preset slice thickness, a slicing machine is used to slice the three-dimensional image blocks with a preset sample thickness for each row.

[0076] In operation S230, the operations of performing successive line-by-line scans and sample slicing are repeated until the overall scan of the three-dimensional biological sample is completed, obtaining all three-dimensional image data sets of the three-dimensional biological sample.

[0077] Taking a mouse as an example, the specific process of obtaining the three-dimensional image data set of the whole-body mouse using the Blockface-VISoR imaging technology is as follows: Blockface-VISoR scans the mouse sample successively line by line in an "S" shape. After scanning each line, a three-dimensional image block of that line is obtained, and the sample thickness included in each image block is 600 - 700 micrometers. After the entire planar scan is completed, a sample slice with a thickness of 400 micrometers is cut off using a slicing machine. Then, the next round of scanning continues until the mouse sample is scanned completely. The sliced samples of each mouse body will contain multiple image blocks.

[0078] To better illustrate the technical solutions involved in the above operations S210 - S230, the following combines specific embodiments and the attached Figure 3 ~Attached Figure 5 to further elaborate on the above operations S210 - S230 in detail.

[0079] Figure 3 is a schematic diagram of obtaining in-slice image data blocks by Blockface-VISoR scanning a mouse sample according to an embodiment of the present invention.

[0080] Figure 4 is a schematic diagram of the inter-slice overlapping part between adjacent slice data according to an embodiment of the present invention.

[0081] Figure 5 is a schematic diagram of the original image data block in the nth slice data according to an embodiment of the present invention.

[0082] The scanning movement trajectory of Blockface-VISoR is as Figure 3 shown, respectively used for moving in the x-axis and y-axis directions, and the y-step is set to include an overlapping area of approximately 10% between adjacent image blocks. The sample moves along the x-axis, and Blockface-VISoR scans continuously (as Figure 3The mouse sample (labeled with thy-1 virus as shown) is scanned line by line to obtain a three-dimensional image block for each line. The thickness of the sample contained in each image block is 600 - 700 microns. Then, it moves along the y-axis and continues to scan the next line. After the entire plane scan is completed, a sample slice with a thickness of 400 microns is cut off using a slicer. Then, the next round of scanning continues until the mouse sample is scanned completely. It should be noted that: (1) There is an overlapping area of 200 - 300 microns between adjacent slices ( Figure 4 ); (2) Each slice sample of the mouse body contains multiple image blocks. The resolution of each image block is (1 × 1 × 2.5 μm3), the pixel value of the image is 12 bits, and the storage format is BigTiff.

[0083] A single original image is shown as Figure 5 shown. The left side is the cross-sectional image in the z direction, and the right side is the xoy horizontal plane. There is also an overlapping area of 200 - 300 microns for the image data of adjacent slices.

[0084] Figure 6 is a flowchart for obtaining multiple three-dimensional standard slice data according to an embodiment of the present invention.

[0085] As Figure 6 shown, the above-mentioned rigid registration algorithm based on normalized cross-correlation information is used to perform intra-slice registration and splicing on the image data blocks within each three-dimensional original slice data, and obtaining multiple three-dimensional standard slice data includes operations S610 - S630.

[0086] In operation S610, for two adjacent image data blocks within the three-dimensional original slice data, the first image data block is used as the fixed image data block, the second image data block is used as the moving image data block, and the parameter matrix of the rigid registration algorithm is initialized. Among them, the parameter matrix of the rigid registration algorithm includes the rotation center, rotation matrix, and displacement matrix.

[0087] The above rotation center can be any point in the moving image.

[0088] For two adjacent image data blocks within the above three-dimensional original slice, generally, the order of the first and the second is determined from left to right, from front to back, and from top to bottom.

[0089] In operation S620, the deformed moving image data block is obtained using the parameter matrix of the rigid registration algorithm, and the normalized cross-correlation information between the deformed moving image data block and the fixed image data block is calculated.

[0090] In operation S630, the parameter matrix of the rigid registration algorithm is updated using the adaptive gradient descent algorithm to minimize the normalized cross-correlation information, and based on the updated parameter matrix of the rigid registration algorithm, the image data blocks in the three-dimensional original slice data are stitched to obtain the three-dimensional standard slice data.

[0091] Taking a mouse sample, an algorithm for rigid registration based on normalized cross-correlation information (NCC) is used to perform in-slice registration and stitching on the image data blocks (Image Stack) within a single three-dimensional original slice data, and multiple image stacks are stitched into a single slice three-dimensional data block. Among them, the NCC is shown in formula (1):

[0092] (1),

[0093] Wherein, represents the fixed image, represents the moving image (the moving image is deformed to align with the fixed image ), represents the domain of the fixed image , and The gray-scale means of are respectively represented as , .

[0094] Among them, the rigid registration algorithm is shown in formula (2):

[0095] (2),

[0096] Wherein, the matrix R is a rotation matrix (i.e., an orthogonal matrix and a proper matrix), c is the rotation center, and t is the translation matrix.

[0097] Figure 7 is the maximum value projection diagram of the xoy plane of the nth slice data according to the embodiment of the present invention.

[0098] Taking the nth slice of the mouse sample in the embodiment as an example, multiple image stacks in the nth slice are stitched into a single slice three-dimensional data block. Figure 7 is the nth slice data after in-slice stitching, and it is the maximum value projection image in the z direction.

[0099] According to an embodiment of the present invention, the above-mentioned sampling of image data blocks for each three-dimensional standard slice data using the sub-region sampling method, obtaining a plurality of sub-image data blocks from each three-dimensional standard slice data includes: obtaining the slice thickness of each image data block in the three-dimensional standard slice data after in-slice stitching operation, and calculating the thickness of the overlapping region of each image data block in the three-dimensional standard slice data according to a preset image data block thickness value; sampling the image data blocks in the three-dimensional standard slice data at a predefined sampling interval from the position of a predefined initial reference image data block according to a predefined sampling interval and sampling size, to obtain a plurality of sub-image data blocks with a predefined sampling size.

[0100] For a single slice data, a method of subregion sampling is used to obtain a certain number of data blocks. Specifically: First, calculate the thickness of the overlapping region , where represents the slice thickness after in-slice stitching, represents a thickness of 400 microns, which is a fixed value. Then, define the sampling interval and the size of the sampled data blocks. Starting from the position of the initial reference data block, move on the slice data at the defined sampling interval. Whenever moving to a new position, extract a sub-region of the corresponding size from the slice data.

[0101] Figure 8 is a schematic diagram of sub-region sampling according to an embodiment of the present invention.

[0102] The following further describes in detail the process of obtaining the above-mentioned plurality of sub-image data blocks through specific embodiments in combination with the attached Figure 8 drawings.

[0103] As Figure 8 shown, for a single slice data, a method of subregion sampling is used to obtain smaller data blocks. Starting from the position of the initial reference data block, move on the slice data at the defined sampling interval. Whenever moving to a new position, extract a sub-region of the corresponding size from the slice data. In the embodiment, the interval between data blocks is 900 pixels, the image sizes of the data blocks in the x and y directions are both 100 pixels, and the thickness of the data blocks in the z direction is the thickness of the overlapping region , where represents the slice thickness after in-slice stitching, represents a thickness of 400 microns, which is a fixed value.

[0104] According to an embodiment of the present invention, for the above displacement registration algorithm using normalized cross - correlation information, registering sub - image data blocks with the same label in adjacent three - dimensional standard slice data to obtain displacement parameters of paired sub - image data blocks includes: in adjacent three - dimensional standard slice data, taking the sub - image data block of the first three - dimensional standard slice data as the fixed sub - image data block, and taking the sub - image data block in the second three - dimensional standard slice data that has the same label as the fixed sub - image data block as the moving sub - image data block; using normalized cross - correlation information and an adaptive gradient descent algorithm to calculate the displacement parameters between the fixed sub - image data block and the moving sub - image data block.

[0105] Registering paired - label sub - image data blocks between adjacent three - dimensional standard slice data can obtain displacement parameters of each pair of sub - image data blocks. Specifically, for each pair of sub - region data blocks (subregion, that is, sub - image data blocks) to be registered between adjacent three - dimensional standard slice data, select the first data block as the fixed image and the other block as the moving image. Similarly, use the NCC formula to calculate the loss function between them, but replace rigid registration with displacement registration. The displacement registration is shown in formula (3):

[0106] (3),

[0107] where, represents the displacement parameter, represents the fixed image, represents the transformed image. In the present invention, the parameters of are respectively

[0108] which represent displacements in the x, y, and z directions.

[0109] According to an embodiment of the present invention, for the above, using the displacement parameters of paired sub - image data blocks, through an interpolation method for parameter fitting to obtain the upper - surface deformation field of each three - dimensional standard slice data includes: taking the center point of the moving sub - image data block as the control point of the interpolation method, setting the displacement parameters of the paired sub - image data blocks as the displacement values to be fitted by the interpolation method, and using the interpolation function used in the interpolation method to fit the upper - surface deformation field of each three - dimensional standard slice data.

[0110] For the obtained displacement parameters, the B-spline interpolation method is used to fit the parameters to obtain the deformation field of the upper surface. Specifically, the center points of the above-mentioned sub-motion image data blocks are used as the control points for B-spline interpolation, and the displacement parameters obtained from the above calculations are set as the displacement values to be fitted by the B-spline interpolation function. Finally, a two-dimensional surface is fitted using the B-spline interpolation function. The B-spline interpolation function of order 3*3 is shown in Equation (4):

[0111] (4),

[0112] where is the value of the interpolation surface, representing the estimated value at position , represents the control points at, which are the data points to be fitted, and are the cubic B-spline basis functions, which are shown by the Boor-Cox recurrence formula as Equation (5):

[0113] (5),

[0114] where t is the knot vector. Using the deformation field of the upper surface, the upper and lower surface image data for adjacent slice matching are obtained, and then the non-rigid deformation algorithm of B-spline is used to register the upper and lower surface image data.

[0115] Figure 9 is a schematic diagram of the result of fitting the control points according to an embodiment of the present invention.

[0116] Figure 10 is a schematic diagram of the two-dimensional interpolation result according to an embodiment of the present invention.

[0117] Figure 11 is a three-dimensional surface rendering diagram generated by Matlab according to an embodiment of the present invention.

[0118] The following further elaborates on the above process of obtaining the deformation field of the upper surface of each three-dimensional standard slice data through specific embodiments in combination with the attached Figures 9 - 11 drawings.

[0119] The center points of the sub-region data blocks of the above-mentioned motion image data blocks are used as the control points ( Figure 9 is the result of fitting the control points), and a two-dimensional deformation surface is fitted using a B-spline surface of order 3*3, Figure 10 is the two-dimensional interpolation result, and the color depth represents the displacement distance after interpolation of each pixel point, Figure 11 is a three-dimensional surface rendering diagram drawn using Matlab.

[0120] According to an embodiment of the present invention, based on the upper surface deformation field of each three-dimensional standard slice data, the upper and lower surface data that match each other between adjacent three-dimensional standard slice data are obtained, and a non-rigid deformation method is used to perform inter-slice surface image registration on adjacent three-dimensional standard slice data, and the inter-slice registration two-dimensional deformation field is obtained, including: based on the upper surface deformation field of each three-dimensional standard slice data, the two-dimensional upper surface image of the three-dimensional standard slice data is obtained, and the bottom surface of the adjacent previous three-dimensional standard slice data is used as the two-dimensional lower surface image; a non-rigid deformation method is used to perform inter-slice surface image registration on the two-dimensional upper surface image in the three-dimensional standard slice data and the two-dimensional lower surface image in the adjacent previous three-dimensional standard slice data, and the inter-slice registration two-dimensional deformation field is obtained..

[0121] For each pair of adjacent three-dimensional standard slice data, there is non-rigid deformation. To solve the deformation problem, first, the deformation field obtained by the above operations is used to obtain the upper and lower surface image data that match the adjacent slices. Then, for the image data of the upper and lower surfaces, image coordinate grid points are generated. These grid points are used to determine positions on the image. After that, using the B-spline deformation algorithm, the deformed upper and lower surface image data are registered, and their positions after deformation are calculated. Finally, applying the displacement information in the deformation field, each pixel point in the image is mapped to the position after non-rigid deformation, thereby solving the non-rigid deformation problem. The B-spline deformation algorithm is shown in formula (6):

[0122] (6),

[0123] wherein, represents the data value of the control point, represents the cubic multi-dimensional B-spline polynomial, represents the B-spline coefficient vector, represents the spacing of the B-spline control points, represents the set of all control points within the compact support of the B-spline at x. The control point represents the regular grid points covering the fixed image.

[0124] Figure 12 is a schematic diagram comparing the image corrected by the B-spline deformation according to the embodiment of the present invention with the image not corrected by the algorithm.

[0125] The following further describes in detail the process of obtaining the inter-slice registration two-dimensional deformation field through specific embodiments and in combination with the attached Figure 12 drawings.

[0126] To solve the non-rigid deformation between the nth slice and the (n + 1)th slice, it is first necessary to use the deformation field obtained from the previous operations to obtain the image data of the upper and lower surfaces of the slices. Then, using the B-spline deformation algorithm, the deformed upper and lower surface image data are registered to calculate their positions after deformation. Finally, the displacement information in the deformation field is applied to map each pixel point in the image to the position after non-rigid deformation, thereby solving the problem of non-rigid deformation. As Figure 12 shown, comparing the image corrected by the B-spline deformation with the image not corrected by the algorithm, the connection points of the blood vessels can be well docked after the transformation.

[0127] According to an embodiment of the present invention, the above method of using the linear interpolation method to expand the inter-slice registration two-dimensional deformation field into a three-dimensional deformation field and applying the expanded three-dimensional deformation field to the three-dimensional image dataset of the three-dimensional biological sample, and further realizing the three-dimensional data reconstruction of the three-dimensional biological sample includes: using the linear interpolation method to expand the inter-slice registration two-dimensional deformation field in the vertical direction to obtain the three-dimensional deformation field of adjacent three-dimensional standard slice data; using the three-dimensional deformation field to deform each three-dimensional standard slice data, and combining the three-dimensional standard slice data with inter-slice splicing, so that the cells and tissue structures on the upper and lower surfaces of adjacent three-dimensional standard slice data are continuous, thereby obtaining the three-dimensional image of the three-dimensional biological sample after reconstruction.

[0128] Taking the mouse sample as an example, the two-dimensional deformation field between slices is expanded into a three-dimensional deformation field. Since the whole-body dataset of the mouse is three-dimensional, for the deformation field of each adjacent slice, it is necessary to expand it in the vertical direction to adapt to the complete three-dimensional image. The linear interpolation method is used to realize the expansion of the dimension of the deformation field, so that the whole-body dataset is continuous and consistent in space. The formula of the specific linear interpolation method is shown in formula (7):

[0129] (7),

[0130] where, represents the two-dimensional deformation field displacement vector of the upper surface of a single slice, represents the two-dimensional deformation field displacement vector of the lower surface of the same slice, , , z represents the position in the vertical direction in the thickness of 0 - 400 microns, and the deformation field displacement vector at this position is .

[0131] Figure 13 is a schematic diagram of the inter-slice splicing work according to an embodiment of the present invention.

[0132] Figure 14 is the whole-body data map of the whole-body mouse sample rendered using Imaris software according to an embodiment of the present invention.

[0133] The following will further elaborate on the three-dimensional data reconstruction process of the above-mentioned three-dimensional biological sample through specific embodiments in combination with the appended Figure 13 and the appended Figure 14 drawings.

[0134] For the deformation fields of each adjacent slice, it is necessary to expand them in the vertical direction to adapt to the complete three-dimensional image. The linear interpolation method is used to achieve the expansion of the deformation field dimension, making the whole-body data set continuous and consistent in space. The interpolated deformation field is applied to the standard image data of a single slice to obtain the three-dimensional image after inter-slice registration. Finally, all the three-dimensional image data after inter-slice registration are merged into the complete mouse whole-body data, as Figure 13 shown in the inter-slice stitching work and Figure 14 the whole-body mouse whole-body data map rendered using Imaris software as shown.

[0135] Figure 15 FIG. schematically shows a block diagram of an electronic device suitable for implementing a method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to an embodiment of the present invention.

[0136] As Figure 15 shown, the electronic device 1500 according to an embodiment of the present invention includes a processor 1501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1502 or a program loaded from a storage section 1508 into a random access memory (RAM) 1503. The processor 1501 can include, for example, a general microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (such as an application-specific integrated circuit (ASIC)), etc. The processor 1501 can also include on-board memory for caching purposes. The processor 1501 can include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0137] In the RAM 1503, various programs and data required for the operation of the electronic device 1500 are stored. The processor 1501, the ROM 1502, and the RAM 1503 are connected to each other through a bus 1504. The processor 1501 performs various operations of the method flow according to an embodiment of the present invention by executing the programs in the ROM 1502 and / or the RAM 1503. It should be noted that the program can also be stored in one or more memories other than the ROM 1502 and the RAM 1503. The processor 1501 can also perform various operations of the method flow according to an embodiment of the present invention by executing the programs stored in one or more memories.

[0138] According to an embodiment of the present invention, the electronic device 1500 may further include an input / output (I / O) interface 1505, and the input / output (I / O) interface 1505 is also connected to the bus 1504. The electronic device 1500 may further include one or more of the following components connected to the I / O interface 1505: an input portion 1506 including a keyboard, a mouse, etc.; an output portion 1507 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage portion 1508 including a hard disk, etc.; and a communication portion 1509 including a network interface card such as a LAN card, a modem, etc. The communication portion 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. A removable medium 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1510 as needed so that a computer program read from it can be installed into the storage portion 1508 as needed.

[0139] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the methods according to the embodiments of the present invention are implemented.

[0140] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1502 and / or the RAM 1503 described above and / or one or more memories other than the ROM 1502 and the RAM 1503.

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than that noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0142] In the above specific embodiments, the objectives, technical solutions, and beneficial effects of the present invention have been further described in detail. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample, characterized in that, Including: Obtaining a three-dimensional image dataset of the three-dimensional biological sample by using a microscopy imaging technique, wherein the three-dimensional image dataset of the three-dimensional biological sample includes a plurality of three-dimensional original slice data, and each of the three-dimensional original slice data includes a plurality of image data blocks; Performing intra-slice registration and stitching on the image data blocks within each of the three-dimensional original slice data by using a rigid registration algorithm based on normalized cross-correlation information to obtain a plurality of three-dimensional standard slice data; Sampling the image data blocks of each of the three-dimensional standard slice data by using a sub-region sampling method, obtaining a plurality of sub-image data blocks from each of the three-dimensional standard slice data, and numbering the plurality of sub-image data blocks; Performing registration on the sub-image data blocks with the same label in adjacent three-dimensional standard slice data by using a displacement registration algorithm based on normalized cross-correlation information to obtain displacement parameters of the paired sub-image data blocks; Performing parameter fitting by using an interpolation method by using the displacement parameters of the paired sub-image data blocks to obtain an upper surface deformation field of each of the three-dimensional standard slice data; Based on the upper surface deformation field of each of the three-dimensional standard slice data, obtaining upper and lower surface data that match each other between the adjacent three-dimensional standard slice data, and performing inter-slice surface image registration on the adjacent three-dimensional standard slice data by using a non-rigid deformation method to obtain an inter-slice registration two-dimensional deformation field; Expanding the inter-slice registration two-dimensional deformation field into a three-dimensional deformation field by using a linear interpolation method, and applying the expanded three-dimensional deformation field to the three-dimensional image dataset of the three-dimensional biological sample, thereby realizing three-dimensional data reconstruction of the three-dimensional biological sample.

2. The method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to claim 1, wherein Obtaining the three-dimensional image dataset of the three-dimensional biological sample by using a microscopy imaging technique includes: Performing row-by-row continuous scanning on the three-dimensional biological sample by using a large-sample high-speed fluorescence microscopy system according to a preset scanning mode to obtain three-dimensional image blocks each having a preset sample thickness per row; After completing the entire plane scanning of the three-dimensional biological sample, performing sample slicing on the three-dimensional image blocks each having a preset sample thickness per row by using a slicing machine according to a preset slicing thickness; Repeating the row-by-row continuous scanning operation and the sample slicing operation until the overall scanning of the three-dimensional biological sample is completed to obtain the entire three-dimensional image dataset of the three-dimensional biological sample.

3. The method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to claim 1, characterized in that, Performing intra-slice registration and stitching on the image data blocks within each of the three-dimensional original slice data by using a rigid registration algorithm based on normalized cross-correlation information to obtain a plurality of three-dimensional standard slice data includes: For two adjacent image data blocks within the three-dimensional original slice data, taking the first image data block as a fixed image data block, taking the second image data block as a moving image data block, and initializing a parameter matrix of the rigid registration algorithm, wherein the parameter matrix of the rigid registration algorithm includes a rotation center, a rotation matrix, and a displacement matrix; Obtaining a deformed moving image data block by using the parameter matrix of the rigid registration algorithm, and calculating the normalized cross-correlation information between the deformed moving image data block and the fixed image data block; Update the parameter matrix of the rigid registration algorithm using an adaptive gradient descent algorithm to minimize the normalized cross-correlation information, and based on the updated parameter matrix of the rigid registration algorithm, perform a stitching operation on the image data blocks in the three-dimensional original slice data to obtain the three-dimensional standard slice data.

4. The method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to claim 1, wherein, Use the sub-region sampling method to sample the image data blocks of each of the three-dimensional standard slice data, and obtain multiple sub-image data blocks from each of the three-dimensional standard slice data, including: Obtain the slice thickness of each image data block in the three-dimensional standard slice data after in-slice stitching operation, and calculate the thickness of the overlapping region of each image data block in the three-dimensional standard slice data according to a preset image data block thickness value; According to a predefined sampling interval and sampling size, start from the position of a predefined initial reference image data block and sample the image data blocks in the three-dimensional standard slice data at the predefined sampling interval to obtain multiple sub-image data blocks with the predefined sampling size.

5. The method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to claim 1, wherein Use a displacement registration algorithm based on normalized cross-correlation information to register the sub-image data blocks with the same label in adjacent three-dimensional standard slice data, and obtain the displacement parameters of the paired sub-image data blocks, including: In the adjacent three-dimensional standard slice data, use the sub-image data block of the first three-dimensional standard slice data as the fixed sub-image data block, and use the sub-image data block in the second three-dimensional standard slice data that has the same label as the fixed sub-image data block as the moving sub-image data block; Use the normalized cross-correlation information and the adaptive gradient descent algorithm to calculate the displacement parameters between the fixed sub-image data block and the moving sub-image data block.

6. The method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to claim 5, wherein Use the displacement parameters of the paired sub-image data blocks to perform parameter fitting through an interpolation method to obtain the upper surface deformation field of each of the three-dimensional standard slice data, including: Use the center point of the moving sub-image data block as the control point of the interpolation method, set the displacement parameters of the paired sub-image data blocks as the displacement values to be fitted by the interpolation method, and use the interpolation function used by the interpolation method to fit the upper surface deformation field of each of the three-dimensional standard slice data.

7. The method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to claim 1, characterized in that, Based on the upper surface deformation field of each of the three-dimensional standard slice data, obtain the mutually matching upper and lower surface data between the adjacent three-dimensional standard slice data, and use a non-rigid deformation method to perform inter-slice surface image registration on the adjacent three-dimensional standard slice data to obtain an inter-slice registration two-dimensional deformation field, including: Based on the upper surface deformation field of each of the three-dimensional standard slice data, obtain the two-dimensional upper surface image of the three-dimensional standard slice data, and use the bottom surface of the previous adjacent three-dimensional standard slice data as the two-dimensional lower surface image; Use the non-rigid deformation method to perform inter-slice surface image registration on the two-dimensional upper surface image in the three-dimensional standard slice data and the two-dimensional lower surface image in the previous adjacent three-dimensional standard slice data to obtain an inter-slice registration two-dimensional deformation field.

8. The method for reconstructing a three-dimensional microscopic image of a three-dimensional biological sample according to claim 1, wherein, The method for reconstructing the three-dimensional microscopic image of the three-dimensional biological sample includes: expanding the inter-slice registration two-dimensional deformation field into a three-dimensional deformation field by using a linear interpolation method, and applying the obtained three-dimensional deformation field to the three-dimensional image data set of the three-dimensional biological sample, so as to realize the three-dimensional data reconstruction of the three-dimensional biological sample. Expand the inter-slice registration two-dimensional deformation field in the vertical direction by using a linear interpolation method to obtain the three-dimensional deformation field of the adjacent three-dimensional standard slice data. Deform each piece of the three-dimensional standard slice data by using the three-dimensional deformation field, and merge the three-dimensional standard slice data with inter-slice splicing, so that the cells and tissue structures on the upper and lower surfaces of the adjacent three-dimensional standard slice data are continuous, and then obtain the three-dimensional image of the reconstructed three-dimensional biological sample.

9. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the method for reconstructing the three-dimensional microscopic image of the three-dimensional biological sample according to any one of claims 1 to 8.

10. A computer-readable storage medium, on which executable instructions are stored, and when the instructions are executed by a processor, the processor is caused to execute the method for reconstructing the three-dimensional microscopic image of the three-dimensional biological sample according to any one of claims 1 to 8.

Images