A three-dimensional brain atlas template construction method

Abstract

The application provides a three-dimensional brain atlas template construction method, which comprises the following steps: step one, deformation field acquisition: a sequence of two-dimensional images is selected, and for any two adjacent two-dimensional images, the former is registered to the latter, and a registration process generates a deformation field; step two, intermediate deformation field generation: for the deformation field generated by the registration of any two adjacent two-dimensional images, an intermediate deformation field is calculated by integrating the deformation path according to time; and step three, interpolation image generation: for any two adjacent two-dimensional images, a to-be-interpolated image is calculated according to the intermediate deformation field. The three-dimensional brain atlas template construction method, storage medium and electronic equipment provided by the application avoid problems such as fragment holes, gap overlaps and the like that may be caused by a traditional three-dimensional reconstruction idea.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for constructing a three-dimensional brain map template. Background Technology

[0002] 3D reconstruction based on 2D image sequences is a routine task in the field of image processing. Its purpose is to reconstruct the 3D morphological information of the object recorded in the image sequence when axial information is scarce, that is, when the axial resolution of the image sequence is much lower than the horizontal resolution of its individual images.

[0003] Brain atlas template construction is also a type of 3D reconstruction task. A brain atlas template typically refers to a 3D image annotated with several brain structures, where voxels belonging to the same anatomical structure are labeled with the same grayscale value. Compared to natural images containing real brain structural information acquired by instruments such as optical microscopes and magnetic resonance imaging (MRI), brain atlas templates containing only labeled information are more suitable as standard references for manual comparison or image registration calculations. Therefore, they can be used to assist users in locating different types of brain information, such as neural circuits and vascular distribution, in 3D space.

[0004] The construction of brain atlas templates typically follows these steps: First, several two-dimensional brain slices are obtained from the same brain tissue sample and imaged under a microscope. Then, the acquired natural images are aligned axially to construct a dataset of natural images to be labeled. Due to limitations in imaging technology, the horizontal resolution of the natural image dataset is often one to two orders of magnitude higher than its axial resolution. Next, experts with neuroanatomical knowledge manually define the boundaries of various anatomical structures on the brain slice images and fill the pixels within the boundaries of each anatomical structure with different grayscale values ​​to obtain labeled images. Since the labeled images are derived from natural images, their spatial resolution is also much higher than the axial spatial distance. Therefore, axial interpolation is performed on the labeled images to calculate several "intermediate images," ensuring that the spatial resolution of the labeled image dataset is comparable to its horizontal resolution. The dataset obtained through interpolation is the three-dimensional brain atlas template.

[0005] In recent years, new imaging technologies, such as "microscopic optical section tomography," have been able to acquire mammalian brain image datasets with both axial and horizontal resolutions on the order of 1 micrometer. Taking the mouse brain as an example, a single dataset obtained using current technology can contain over 10,000 brain slices with horizontal resolutions at the micrometer level, and the axial spacing between adjacent slices is also 1 micrometer. However, due to the large number of anatomical structures contained in a single brain slice, and the difficulty in automatically identifying and extracting their boundaries, manual annotation by anatomical experts is still required. Furthermore, acquiring expert experience requires years of neuroanatomical training, which is difficult to scale and batch, limiting the number of images that experts can manually annotate. Again, using the mouse brain as an example, to construct a 3D atlas template from the image dataset acquired by the new imaging technology, approximately one brain slice needs to be manually annotated for every 100 images to keep the workload within a reasonable range. Even then, the axial resolution of the resulting labeled image sequence is still two orders of magnitude lower than the horizontal resolution. Therefore, how to construct a 3D brain atlas template based on a limited number of 2D annotated images remains a pressing task.

[0006] The current mainstream processing approaches can be broadly divided into two categories. The first category treats it as a special case of the regular image interpolation problem, including classic nearest neighbor interpolation, linear interpolation, and bicubic interpolation. The core idea is to search for the nearest pixels in a two-dimensional image sequence for any point in space, and then obtain the gray value that should be filled at that point by sorting these pixels by distance or by weighted averaging of their gray values. This type of method does not treat the labeled brain anatomical structures as a whole; it processes isolated voxel points. Therefore, the final interpolation result inevitably produces holes and fragmentation, making it impossible for the reconstruction result to meet the requirement that the boundaries of each anatomical structure in the three-dimensional brain atlas template must be closed and continuous.

[0007] The second approach involves first extracting feature points from the outer boundary of the labeled structure in a 2D image sequence to construct 3D point cloud data. Then, triangular facets are constructed from the point cloud to reconstruct the 3D morphology of the labeled object. Since this approach treats the point cloud as a single object, it effectively utilizes the overall 3D morphological features of the labeled structure. Furthermore, by combining this with filtering algorithms for outliers, the 3D morphology can be further optimized, resulting in a smoother outer surface of the reconstructed object. However, this method is still unsuitable for 3D reconstruction of brain atlas templates. This is because adjacent anatomical structures in a 3D brain atlas are seamlessly connected; in other words, a location in space belongs to only one specific structure within the brain atlas template. There is no situation where a point simultaneously belongs to two different anatomical structures or does not belong to any structure, otherwise it would violate the realities of biological tissue. Methods based on 3D point clouds only consider the 3D morphological information of each labeled object itself, neglecting the spatial relationships between objects. Therefore, the reconstructed object surfaces either have gaps or overlaps, failing to meet the requirements for 3D reconstruction of brain atlas templates.

[0008] In summary, since the boundaries of each anatomical structure in the brain atlas template must be continuous and closed, and adjacent anatomical structures must meet the requirement of "seamless connection", the conventional approaches currently used for three-dimensional reconstruction of two-dimensional image sequences are difficult to apply. Summary of the Invention

[0009] Based on the aforementioned deficiencies in the existing technology, this invention provides a method for constructing a three-dimensional brain map template, which can avoid the fragmentation, voids, gaps, or overlaps caused by traditional three-dimensional reconstruction methods.

[0010] To achieve the above objectives, the present invention provides a method for constructing a three-dimensional brain map template, comprising:

[0011] Step 1, Deformation field acquisition: Select a sequence of two-dimensional images. For any two adjacent two-dimensional images, register the previous image to the next image. The registration process generates a deformation field.

[0012] Step 2, intermediate deformation field generation: For any two adjacent two-dimensional images registered and generated deformation fields, the intermediate deformation field is calculated by integrating the deformation path over time.

[0013] Step 3, Interpolation Image Generation: For any two adjacent two-dimensional images, calculate the image to be interpolated based on the intermediate deformation field.

[0014] In one embodiment, in step one, the sequence of two-dimensional images includes N two-dimensional images, denoted as S = {s1, s2, ... s...} i,…s N}, any image s in this sequence i For each i = 1, 2, ..., N, the number of pixels and resolution are the same in both the length and width directions, and the pixel resolution is denoted as hμm / pixel; in the axial direction, any two adjacent two-dimensional images s in the sequence... i and s i+1 The distance between them is denoted as v. i μm, v i >h.

[0015] In one embodiment, the two-dimensional image is a two-dimensional marked image.

[0016] In one embodiment, any two adjacent two-dimensional images s in the sequence i and s i+1 The distances between them may be the same or different.

[0017] In one embodiment, step one includes: for any two adjacent two-dimensional images s i and s i+1 Using a differential homeomorphic nonlinear deformation image registration algorithm, s i Registration to s i+1 The above registration process generates the deformation field, denoted as D. i .

[0018] In one embodiment, the deformation field D i Satisfying Similarity(D) i (F i ),F i+1 )≥Thresh,F i and F i+1 For any two adjacent two-dimensional images s i and s i+1 The image features are a set of features, which are feature points or feature regions marked with the same gray value, and the number of features is unlimited. Similarity is the similarity evaluation corresponding to the image features, and Thresh is a preset similarity threshold.

[0019] In one embodiment, step two includes:

[0020] 1) Calculate the number of images to be interpolated: m i =floor(v i / h)-1,m i For any two adjacent two-dimensional images s in a sequence S of two-dimensional images i and s i+1 The number of images that need to be interpolated, where floor represents the floor operation;

[0021] 2) Calculate the intermediate deformation field: For time point T k The intermediate deformation field under T k To divide the integration time into m equal parts i +1 segment, the end time of each segment.

[0022] In one embodiment, step three includes:

[0023] 1) Generating intermediate images: Let s be any two adjacent two-dimensional images in the sequence S of two-dimensional images. i and s i+1 Between them, the images to be interpolated are arranged in order as C i ={c1,c2,…c k ,…c mi}, for any image c to be interpolated k ,have This generates all the images to be interpolated;

[0024] 2) Construct a 3D image stack: Combine the sequence S of 2D images with all the images C to be interpolated. i Arranged in spatial order, they form a three-dimensional image stack R, which serves as the template for the constructed three-dimensional brain atlas.

[0025] The core idea of ​​this invention is to treat the "axial interpolation problem of two-dimensional image sequences" as a registration problem, applying the time integral characteristics of the registration deformation field to the field of image interpolation. Based on the registration, a deformation field is generated along the time integral, and intermediate deformation fields at various intermediate time points are obtained. Interpolation is then performed to obtain intermediate images, making the axial resolution comparable to the horizontal resolution, thus enabling the construction of a three-dimensional brain atlas template. The Jacobian matrix of the deformation field is non-negative everywhere, meaning that folding and tearing will not occur, and therefore the interpolated intermediate images will not contain fragments or holes. Simultaneously, the deformation field is generated based on the entire image, and is smooth and continuous throughout the entire image domain. Therefore, the adjacency relationships between different brain anatomical structures can still be maintained in the interpolated intermediate images, avoiding errors such as gaps and overlaps. These characteristics can be summarized as "shape preservation" of the labeled objects, avoiding problems such as fragmentation, holes, and overlapping gaps that may occur in traditional three-dimensional reconstruction methods. Attached Figure Description

[0026] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of the invention in any way. Furthermore, the shapes and proportions of the components in the drawings are merely illustrative to aid in understanding the invention and are not intended to specifically limit the shapes and proportions of the components. Those skilled in the art, guided by the teachings of this invention, can select various possible shapes and proportions to implement the invention according to specific circumstances. In the drawings:

[0027] Figure 1 This is a flowchart illustrating a method for constructing a three-dimensional brain map template according to the first embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0029] Please see Figure 1 As shown, the first embodiment of the present invention provides a method for constructing a three-dimensional brain map template, comprising:

[0030] Step 1 S1, Deformation field acquisition: Select a sequence of two-dimensional images. For any two adjacent two-dimensional images, register the previous image to the next image. The registration process generates a deformation field.

[0031] Step 2 S2, intermediate deformation field generation: For the deformation fields generated by registration of any two adjacent two-dimensional images, the intermediate deformation field is calculated by integrating the deformation path over time.

[0032] Step 3 S3, Interpolation Image Generation: For any two adjacent two-dimensional images, calculate the image to be interpolated based on the intermediate deformation field.

[0033] In one embodiment, in step one, the sequence of two-dimensional images includes N two-dimensional images, denoted as S = {s1, s2, ... s...} i ,…s N}, any image s in this sequence i For each i = 1, 2, ..., N, the number of pixels and resolution are the same in both the length and width directions, and the pixel resolution is denoted as hμm / pixel; in the axial direction, any two adjacent two-dimensional images s in the sequence... i and s i+1 The distance between them is denoted as v. i μm, v i >h.

[0034] In one embodiment, the two-dimensional image is a two-dimensional labeled image, that is, the three-dimensional brain atlas template construction method of the present invention is used to perform three-dimensional reconstruction of a two-dimensional labeled image after the anatomical structures have been labeled. Of course, the three-dimensional brain atlas template construction method of the present invention can also be used to perform three-dimensional reconstruction of two-dimensional images before the anatomical structures have been labeled, as well as for other similar tasks besides three-dimensional brain atlas template construction, as long as it is applicable to the three-dimensional reconstruction of two-dimensional images.

[0035] In one embodiment, any two adjacent two-dimensional images s in the sequence i and s i+1 The distances between them can be the same or different. The following uses any two adjacent two-dimensional images s... i and s i+1 Taking the different distances between them as an example, for any two adjacent two-dimensional images s i and s i+1 When the distances between them are the same, those skilled in the art can easily conceive of adapting and adjusting the construction method accordingly.

[0036] In one embodiment, step one includes: for any two adjacent two-dimensional images s i and s i+1 Using a differential homeomorphic nonlinear deformation image registration algorithm, s i Registration to s i+1 The above registration process generates the deformation field, denoted as D. i The differential homeomorphic nonlinear deformation image registration algorithm is a two-dimensional image registration algorithm constructed based on the differential homeomorphic nonlinear deformation model, and can be used in the three-dimensional brain atlas template construction method of the present invention. Furthermore, not all differential homeomorphic nonlinear deformation image registration algorithms are suitable for the three-dimensional brain atlas template construction method of the present invention.

[0037] In one embodiment, the differential homeomorphic nonlinear deformation image registration algorithm should meet the following requirements: the deformation field D i Satisfying Similarity(D) i (F i ),F i+1 )≥Thresh,F i and F i+1 For any two adjacent two-dimensional images s i and s i+1 The image features are a set of features, which are feature points or feature regions marked with the same gray value, and the number of features is unlimited. Similarity is the similarity evaluation corresponding to the image features, and Thresh is a preset similarity threshold.

[0038] In one embodiment, step two includes:

[0039] 1) Calculate the number of images to be interpolated: m i =floor(v i / h)-1,m i For any two adjacent two-dimensional images s in a sequence S of two-dimensional images i and s i+1 The number of images to be interpolated, where floor represents the floor function; for example, floor(2.7) = 2. In any two adjacent 2D images s... i and s i+1 Insert m between i By generating one image, the resolution in the horizontal direction (length and width) of the image can be made consistent with the resolution in the axial direction.

[0040] 2) Calculate the intermediate deformation field: For time point T k The intermediate deformation field under T k To divide the integration time into m equal parts i +1 segment, the end time point of each segment. Because under the differential homeomorphic nonlinear deformation model, image s i Registration to s i+1 The generated deformation field D i , is image s i The deformation path integral is taken along time t, where t∈[0,1]. Based on this definition, the deformation field at any time t can be denoteed. Where M t Let be the image shape variable at any time t. Also, we have... Divide the integration time into m equal parts. i +1 segment, then the end time of each segment Where k = 1, 2, ..., m i The corresponding intermediate state deformation field Therefore, time point T is obtained. k The intermediate deformation field under M. t If the difference with t is not significant, it can be approximated as...

[0041] In one embodiment, step three includes:

[0042] 1) Generating intermediate images: Let s be any two adjacent two-dimensional images in the sequence S of two-dimensional images. i and s i+1 Between them, the images to be interpolated are arranged in order as C i ={c1,c2,…c k ,…c mi}, for any image c to be interpolatedk ,have This generates all the images to be interpolated;

[0043] 2) Construct a 3D image stack: Combine the sequence S of 2D images with all the images C to be interpolated. i The images are arranged in spatial order to form a three-dimensional image stack R, which serves as the template for the constructed three-dimensional brain atlas. This completes the construction of the three-dimensional brain atlas template.

[0044] The following example illustrates the process of performing deformation operations on a two-dimensional image using a deformation field. This example is specifically for processing two-dimensional images; the process can be similarly applied to images larger than two dimensions. Those skilled in the art, based on this document and common knowledge in the field, can understand the complete process of generating an intermediate image based on an intermediate deformation field. The example process includes:

[0045] 1) Let the image be s and the deformation field be D. Since in image processing, "image" is generally considered a "digital image," s can be considered an image containing 'a' pixels horizontally and 'b' pixels vertically, where 'a' and 'b' can be any positive integers. Similarly, the deformation field D can be considered an image containing 'm' pixels horizontally and 'n' pixels vertically. The difference is that each pixel in D is a 2D vector, and each component of the vector is a real number representing how far the input image s should move horizontally and vertically from the position marked by that pixel in the deformation field.

[0046] 2) Based on the above conventions, the calculation process of D(s) is as follows: First, consider the simplest case. When the image resolution (pixel size) of s and D is the same, the pixels of the two can be corresponded one-to-one in the spatial coordinate system. Therefore, for image s, traverse from 1 to a along the horizontal direction and from 1 to b along the vertical direction. For each pixel p in the image, according to the coordinates (x, y) of pixel p, find the corresponding pixel p' in D. Take the offset d recorded by the pixel value. Apply the horizontal component d(1) and the vertical component d(2) of the offset to the coordinates of pixel p respectively to obtain the new coordinates (x+d(1), y+d(2)) that pixel p should move to under the deformation field D. Since the new coordinates are usually not integers, the new coordinates are usually rounded when outputting D(s). In addition, in some cases, the coordinate range m×n of D may not completely cover the coordinate range a×b of image s. For the part not covered by D, the default offset is 0.

[0047] 3) Building upon 2), consider a more complex scenario: the image resolutions (pixel sizes) of images s and D are different. In this case, calculations need to be performed using physical coordinates. Assuming the image resolution of s is rμm / pixel, when traversing any pixel p in s, its pixel coordinates (x, y) are converted to physical coordinates (x×r, y×r). Based on the physical coordinates of p, the offset at that position in D is found. If this coordinate happens to be the same as the coordinate of a pixel in D, then the steps in 2) are followed. If this coordinate does not fall on a pixel in D, then the four nearest pixels in D to that coordinate are searched, and then a weighted average is calculated based on the distances of the four pixels to the current coordinate to obtain the offset at the current coordinate.

[0048] The second embodiment of the present invention provides a storage medium on which a computer program is stored. When the computer program is read and executed by a processor, it performs the three-dimensional brain map template construction method as described above.

[0049] A second embodiment of the present invention provides an electronic device, including: a processor, a memory, and a communication bus, wherein the processor communicates with the memory via the communication bus to execute the three-dimensional brain map template construction method as described above.

[0050] The core idea of ​​the three-dimensional brain atlas template construction method, storage medium, and electronic device described in this invention is to treat the "axial interpolation problem of two-dimensional image sequences" as a registration problem. The time integral characteristics of the registration deformation field are applied to the field of image interpolation. Based on the registration, a deformation field is generated along the time integral, and intermediate deformation fields at various intermediate time points are obtained. Interpolation is then used to obtain intermediate images, making the axial resolution comparable to the horizontal resolution, thus realizing the construction of a three-dimensional brain atlas template. The Jacobian matrix of the deformation field is non-negative everywhere, meaning that folding and tearing will not occur. Therefore, the interpolated intermediate images will not have fragments or holes. Simultaneously, the deformation field is generated based on the entire image, and the deformation field is smooth and continuous throughout the entire image domain. Therefore, the adjacency relationships between different brain anatomical structures can still be maintained in the interpolated intermediate images, avoiding errors such as gaps and overlaps. These characteristics can be summarized as "shape preservation" of the labeled objects, avoiding problems such as fragmentation, holes, and overlapping gaps that may occur in traditional three-dimensional reconstruction methods.

[0051] It should be understood that the above description is for illustrative purposes and not for limitation. Many embodiments and applications beyond the provided examples will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of this teaching should not be determined by reference to the above description, but rather by reference to the foregoing claims and the full scope of their equivalents. For purposes of completeness, all articles and references, including patent applications and publications, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the foregoing claims is not intended as a waiver of that subject matter, nor should it be construed as an indication that the applicant has not considered that subject matter as part of the disclosed application subject matter.

Claims

1. A method for constructing a three-dimensional brain atlas template, characterized in that, include: Step 1, Deformation Field Acquisition: Select a sequence of two-dimensional images. For any two adjacent two-dimensional images, register the previous image to the next image. The registration process generates the deformation field D. i ; Step 2, intermediate deformation field generation: For any two adjacent two-dimensional images registered and generated deformation fields, the intermediate deformation field is calculated by integrating the deformation path over time. Step 3, Interpolation Image Generation: For any two adjacent two-dimensional images, calculate the image to be interpolated based on the intermediate deformation field; Step two includes: 1) Calculate the number of images to be interpolated: m i =floor(v i / h)-1,m i For any two adjacent two-dimensional images s in a sequence S of two-dimensional images i and s i+1 The number of images that need to be interpolated, where floor represents the floor operation; 2) Calculate the intermediate deformation field: For time point T k The intermediate deformation field under T k To divide the integration time into m equal parts i +1 segment, the end time of each segment; Step three includes: 1) Generating intermediate images: Let s be any two adjacent two-dimensional images in the sequence S of two-dimensional images. i and s i+1 Between these, the images to be interpolated are arranged in order as C i ={c1,c2,…c k ,…c mi }, for any image c to be interpolated k ,have This generates all the images to be interpolated; 2) Construct a 3D image stack: Combine the sequence S of 2D images with all the images C to be interpolated. i Arranged in spatial order into a three-dimensional image stack R, the three-dimensional image stack R is the template for the constructed three-dimensional brain atlas; In step one, the sequence of two-dimensional images includes N two-dimensional images, denoted as S = {s1, s2, ... s...} i ,…s N }, any image s in this sequence i For each i = 1, 2, ..., N, the number of pixels and resolution are the same in both the length and width directions, and the pixel resolution is denoted as hμm / pixel; in the axial direction, any two adjacent two-dimensional images s in the sequence... i and s i+1 The distance between them is denoted as v. i μm, v i >h.

2. The method for constructing a three-dimensional brain atlas template as described in claim 1, characterized in that, The two-dimensional image is a two-dimensional labeled image.

3. The method for constructing a three-dimensional brain map template as described in claim 1, characterized in that, Any two adjacent two-dimensional images s in the sequence i and s i+1 The distances between them may be the same or different.

4. The method for constructing a three-dimensional brain atlas template as described in claim 1, characterized in that, Step one includes: for any two adjacent two-dimensional images s i and s i+1 Using a differential homeomorphic nonlinear deformation image registration algorithm, s i Registration to s i+1 The above registration process generates the deformation field, denoted as D. i .

5. The method for constructing a three-dimensional brain atlas template as described in claim 4, characterized in that, The deformation field D i Satisfying Similarity(D) i (F i ),F i+1 )≥Thresh,F i and F i+1 For any two adjacent two-dimensional images s i and s i+1 The image features are a set of features, which are feature points or feature regions marked with the same gray value, and the number of features is unlimited. Similarity is the similarity evaluation corresponding to the image features, and Thresh is a preset similarity threshold.

Images