A cross-modality mouse brain region cascade registration method and system
By employing a cross-modal mouse brain region cascade registration method and utilizing non-rigid and non-linear registration techniques for multi-channel images, the spatial correlation problem between mouse brain in vivo fluorescence microscopy data and standard brain atlases was solved. This enabled accurate anatomical localization of mouse brain in vivo fluorescence microscopy data, supporting the precise correspondence between neural circuit activity patterns and anatomical brain region functional attributes, as well as the standardized construction of brain functional atlases.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG HEHU TECH CO LTD
- Filing Date
- 2026-06-15
- Publication Date
- 2026-07-14
AI Technical Summary
Existing technologies cannot effectively and stably correlate data with standard brain maps in mouse brain in vivo fluorescence microscopy, resulting in the loss of data isolation and cross-individual reproducibility, which limits a deeper understanding of structure-function relationships.
A cross-modal mouse brain region cascade registration method was adopted. By acquiring multi-channel images of mice before and after craniotomy, non-rigid registration and cross-modal nonlinear registration were performed. Feature matching was carried out using the venous features on the meningeal surface to achieve cascade registration of images and transfer the Bregma spatial reference frame to the field of view of in vivo fluorescence microscopy.
It enables accurate anatomical localization of mouse brain in vivo fluorescence microscopy imaging data without the need for endogenous signal imaging devices or dedicated optical equipment, supporting the precise correspondence between subsequent neural circuit activity patterns and the functional attributes of anatomical brain regions, as well as the standardized construction of brain functional atlases.
Smart Images
Figure CN122391316A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of microscopic imaging technology, and more specifically to a cross-modal mouse brain region cascade registration method. Background Technology
[0002] Currently, in vivo fluorescence microscopy is one of the important methods for observing nerve cell function. However, due to the obstruction of the skull, in vivo fluorescence microscopy of the mouse brain usually requires the use of skull window technology. Constructing a transparent skull window allows for clear observation of mouse brain nerve cells and vascular branch structures, and enables long-term dynamic observation, thereby providing real-time understanding of structural changes in nerve cell and tissue morphology, and further studying the correlation between these changes and disease or behavioral changes.
[0003] However, obtaining only in vivo fluorescence microscopy data of the mouse brain leads to the loss of data isolation and cross-individual reproducibility. Due to the lack of anatomical background information, this limits a deeper understanding of structure-function relationships. Therefore, it is necessary to use image registration to map the in vivo mouse brain imaging data to the same standard mouse brain template, namely the Allen standard brain atlas, and then perform corresponding analogy and visualization analysis.
[0004] The anterior fontanelle (Bregma) and posterior fontanelle (Lambda) of the mouse skull are anatomical landmarks with well-defined coordinates. These sites allow for precise registration of mouse brain in vivo fluorescence microscopy data to the CCFv3 brain atlas, providing accurate anatomical localization of nerve cells and tissue structures. However, during the construction of the skull window, the skull is partially or completely removed, making it impossible to directly obtain sites with well-defined anatomical coordinates such as Bregma or Lambda from the mouse brain in vivo fluorescence microscopy data. Currently, spatial mapping between mouse brain in vivo fluorescence microscopy data and the CCFv3 brain atlas primarily relies on intrinsic imaging methods. However, intrinsic signals suffer from inherent limitations such as limited spatial resolution, low signal-to-noise ratio, and the need for dedicated intrinsic signal acquisition devices. Literature reports that the resolution of intrinsic imaging is typically 100 micrometers, while current mouse in vivo fluorescence microscopy equipment typically achieves resolutions ranging from hundreds of nanometers to several micrometers.
[0005] Therefore, how to efficiently and stably correlate mouse brain fluorescence microscopy data with standard brain atlases without relying on endogenous imaging is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] In view of the above problems, the present invention proposes a cross-modal mouse brain region cascade registration method and system to overcome or at least partially solve the above problems.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] In a first aspect, embodiments of the present invention provide a cross-modal mouse brain region cascade registration method, comprising the following steps: S1: Obtain the three-channel RGB image of the mouse before craniotomy as the first image, the three-channel RGB image after craniotomy as the second image, and the single-channel grayscale image of the brain region after craniotomy as the third image. S2: Perform non-rigid registration on the first image and the second image, calculate the first spatial transformation relationship from the first image to the second image based on the control point set of the first image and the second image, and align the coordinate system of the first image to the coordinate system of the second image; S3: Perform cross-modal nonlinear registration on the second image and the third image, extract the morphological features of the meningeal surface veins from the second image and the third image respectively, perform cross-modal feature matching based on the extracted vascular features, and construct a second spatial transformation relationship from the second image to the third image based on the matching results; S4: Concatenate the first spatial transformation relationship with the second spatial transformation relationship, and sequentially transfer the coordinates of the skull anatomical landmarks to be mapped in the first image to the third image to complete the cascaded registration of the mouse brain regions across modalities.
[0009] Further, in step S2, the non-rigid registration specifically includes: The first and second images are downsampled and then displayed, and multiple sets of corresponding control points are collected interactively. The coordinates of the acquired control points are restored to the original image resolution space; The homography transformation matrix from the first image to the second image is solved using the Random Sampling Consensus (RANSAC) algorithm, which serves as the first spatial transformation relationship.
[0010] Furthermore, in step S3, before extracting the morphological features of the veins on the meningeal surface, a preprocessing step is included for the second image and the third image. This preprocessing includes: Region of interest extraction and scale uniformity; Highlight area detection and repair; Enhanced vascular contrast and extraction of vascular structures.
[0011] Furthermore, in step S3, the highlight area detection and repair includes: Highlight regions are extracted by threshold segmentation to generate a binary mask; The binary mask generated in the highlight area is morphologically expanded to eliminate small gaps and completely cover the highlight area. The TELEA algorithm, based on the fast traversal method, is adopted.
[0012] Furthermore, the vascular contrast enhancement and vascular structure extraction include: Local contrast enhancement of the image is achieved by using contrast-limited adaptive histogram equalization. The Frangi filtering algorithm is used to perform tubular structure enhancement on the image after local contrast enhancement. The Frangi filtering result is binarized to obtain the vascular skeleton mask.
[0013] Furthermore, in step S1, when performing cross-modal feature matching, multi-scale LoFTR feature matching is used, specifically including: Multi-scale scaling is performed on the blood vessel masks corresponding to the second image and the third image to obtain multi-scale image pairs; The image pairs at each scale are input into the LoFTR model, and the matching point pairs and confidence scores are output. Filter out matching pairs with confidence scores below a preset confidence threshold to obtain valid matching pairs; The coordinates of the matching points in the effective matching point pairs are restored to the original scale to obtain the restored matching point set.
[0014] Furthermore, after obtaining the restored matching point set, when constructing the second spatial transformation relationship from the second image to the third image based on the matching results, a piecewise affine transformation based on Delaunay triangulation is adopted, specifically including: Perform Delaunay triangulation on the restored matching point set to generate a non-overlapping triangular mesh topology covering the entire restored matching point set; Based on the one-to-one correspondence of matching point pairs in the restored matching point set, the vertex indices of the source triangle are mapped to the target points in the restored matching point set to generate the target triangle. The area of the source triangle is calculated by vector cross product, and source triangles with areas close to 0 are filtered out to obtain the effective computation set. For each set of valid triangular faces within the valid computation set, calculate the affine transformation matrix from the source triangular face to the target triangular face; For each valid triangular facet corresponding to the image region, perform an affine transformation and fuse it using a mask to obtain the registered image.
[0015] Furthermore, after obtaining the registered image, the transformation relationship from the second image to the third image is obtained:
[0016] in, The original size coordinates of the point to be mapped in the second image I2. The coordinates of the top-left corner of the region of interest selected by I2. Let be the coordinates of the point to be mapped within the region of interest in I2. The scale factor for the region of interest of I2. These are the scaled coordinates. The coordinates are after piecewise affine transformation. The coordinates of the top-left corner of the region of interest selected in the third image I3. These are the mapped coordinates on I3.
[0017] Secondly, embodiments of the present invention provide a cross-modal mouse brain region cascade registration system, comprising: The image acquisition module is used to acquire a three-channel RGB image of the mouse before craniotomy as the first image, a three-channel RGB image after craniotomy as the second image, and a single-channel grayscale image of the brain region after craniotomy as the third image. The first registration module is used to perform non-rigid registration between the first image and the second image, calculate the first spatial transformation relationship from the first image to the second image based on the control point set of the first image and the second image, and align the coordinate system of the first image to the coordinate system of the second image. The second registration module is used to perform cross-modal nonlinear registration between the second image and the third image, extract the morphological features of the meningeal surface veins from the second image and the third image respectively, perform cross-modal feature matching based on the extracted vascular features, and construct a second spatial transformation relationship from the second image to the third image based on the matching results. The cascaded mapping module is used to cascade the first spatial transformation relationship with the second spatial transformation relationship, and sequentially transfer the coordinates of the skull anatomical landmarks to be mapped in the first image to the third image, thereby completing the cascaded registration of the mouse brain regions across modalities.
[0018] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a cross-modal mouse brain region cascade registration method, which has the following beneficial effects: This invention achieves efficient transfer of the Bregma spatial reference frame from the skull surface to the field of view in mouse in vivo fluorescence microscopy through a two-stage cascaded registration strategy. Without requiring additional intrinsic signal imaging devices or dedicated optical equipment, this invention can accurately locate the anatomical structure of mouse brain in vivo fluorescence microscopy data, providing crucial technical support for understanding the precise correspondence between specific neural circuit activity patterns and the functional attributes of their respective anatomical brain regions, as well as for the standardized construction of brain functional atlases. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0020] Figure 1 This is a flowchart of the cross-modal mouse brain region cascade registration method provided in this embodiment of the invention; Figure 2 This is a schematic diagram of the registration mapping points of the brain region after craniotomy provided in an embodiment of the present invention; Figure 3 This is a semi-transparent overlay schematic diagram showing the registration results between post-craniotomy images and brain region images captured by a head-mounted camera, provided in an embodiment of the present invention. Figure 4 This is a schematic diagram of brain region localization for a mouse brain neuron calcium signal activity image provided in an embodiment of the present invention; Figure 5 This is a structural diagram of the cross-modal mouse brain region cascade registration system provided in an embodiment of the present invention. Detailed Implementation
[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] This invention discloses a cross-modal mouse brain region cascade registration method, such as... Figure 1 As shown, it includes the following steps: S1: Obtain the three-channel RGB image of the mouse before craniotomy as the first image, the three-channel RGB image after craniotomy as the second image, and the single-channel grayscale image of the brain region after craniotomy as the third image. S2: Perform non-rigid registration on the first image and the second image, calculate the first spatial transformation relationship from the first image to the second image based on the control point set of the first image and the second image, and align the coordinate system of the first image to the coordinate system of the second image; S3: Perform cross-modal nonlinear registration on the second image and the third image, extract the morphological features of the meningeal surface veins from the second image and the third image respectively, perform cross-modal feature matching based on the extracted vascular features, and construct a second spatial transformation relationship from the second image to the third image based on the matching results; S4: Concatenate the first spatial transformation relationship with the second spatial transformation relationship, and sequentially transfer the coordinates of the skull anatomical landmarks to be mapped in the first image to the third image to complete the cascaded registration of the mouse brain regions across modalities.
[0023] This invention utilizes a two-stage cascaded registration strategy to effectively transfer the Bregma spatial reference frame of the anterior fontanelle from the skull surface to the field of view of in vivo fluorescence microscopy imaging in mice. Without requiring additional intrinsic signal imaging devices or dedicated optical equipment, this invention can achieve accurate anatomical localization of in vivo fluorescence microscopy imaging data of the mouse brain. This provides crucial technical support for understanding the precise correspondence between specific neural circuit activity patterns and the functional attributes of their respective anatomical brain regions, as well as for the standardized construction of brain functional atlases.
[0024] The method proposed in this invention is primarily aimed at in vivo fluorescence microscopy imaging in mice. It employs a cross-modal data cascade scheme and a two-level registration algorithm to effectively transfer the Bregma space reference frame from the skull surface to the in vivo fluorescence microscopy imaging site in mice. Compared to the extraction of local tissue features in the human brain, the sulci and gyri of the mouse brain are not prominent, thus the method targets significant and widely distributed vascular features.
[0025] The following is a detailed description of each of the above steps: In step S1, image acquisition is performed first; Three-channel RGB images of the mouse before craniotomy were captured using a stereomicroscope and designated as the first image, denoted as I1. In this embodiment, a resolution of 360 nm was used. 5888 Image 10368, using the same stereomicroscope camera and maintaining the same imaging parameters, was captured as the second image (I2) after craniotomy to expose the pia mater surface. A single-channel grayscale image of the mouse brain region after craniotomy was captured using a head-mounted microscope and designated as the third image (I3). The resolution was 1. 648 1152.
[0026] The first image is an initial bright-field reflectance image containing the anatomical landmark Bregma of the skull. The second and third images clearly show the morphological features of the veins on the surface of the meninges. This provides a technical basis for the standardized construction of brain functional atlases.
[0027] In step S2, the first-level registration is performed; The first-level registration involves pre- and post-craniotomy images, i.e., a non-rigid registration of the first and second images. Based on the control point sets acquired from the first and second images, the RANSAC algorithm is used to solve the homography transformation matrix, aligning the coordinate system of the first image to the coordinate system of the second image. The specific process is as follows: First, read the first and second images, and then use a downsampling strategy. D Display the original image at extremely high resolution and collect the corresponding control points through mouse interaction; Establish a coordinate scaling model to restore the interactive coordinates to the original resolution space, for click coordinates on the downsampled image. Its true coordinates in the original high-resolution image Determined by the following formula:
[0028] This step ensures that the accuracy of subsequent geometric transformation calculations is not affected by the display resolution.
[0029] Based on the collected control point set, a homography transformation matrix is constructed. This invention uses the RANSAC random sampling consensus algorithm to solve the matrix, eliminating mismatched points that may be generated by manual point selection. Homography transformation describes linear mapping relationships in homogeneous coordinates:
[0030] Where s is the scale factor, H 1→2 It is a 3×3 homography matrix; by calculating this matrix, the geometric transformation relationship of brain tissue before and after craniotomy under fluoroscopic projection can be approximately described, and the coordinate system of the first image can be initially aligned to the coordinate system of the second image.
[0031] In step S3, the second-level registration is performed; The second-level registration is a cross-modal nonlinear registration between the second and third images. This step aims to address changes in the microscopic vascular morphology of brain tissue after craniotomy, such as vascular tortuosity and local stretching, as well as modal differences between macroscopic images and head-mounted microscope images. The specific process is as follows: The first step is preprocessing, which includes extracting the Region of Interest (ROI) and scaling. Since the second image is a macroscopic color image, it needs to be converted to grayscale before being registered with the third image. The conversion process is as follows:
[0032] in, This represents the converted grayscale image, where R, G, and B represent the values of the red, green, and blue channels in the original I2, respectively.
[0033] The ROI brain regions were manually selected on the converted grayscale image and the third image to quickly remove tissues outside the mouse brain and scene interference, thereby enhancing the robustness of cross-modal registration.
[0034] To ensure that subsequent registration is performed at a uniform scale, the selected ROI region needs to be cropped and scaled:
[0035] in, They are respectively The width and height of the ROI These are the width and height of I3ROI, respectively. The image is scaled up. For the original The ROI image cropped from the top.
[0036] Highlighted and reflective areas are prone to appear during brain region imaging, leading to the loss of vascular features. Repairing the highlighted areas can reduce image grayscale abnormalities and false features caused by highlights, providing a complete feature basis for subsequent feature matching. Traditional pixel-by-pixel filling is inefficient. This step uses the TELEA algorithm based on the fast traversal method to achieve efficient adaptive filling.
[0037] Highlight regions are extracted using threshold segmentation to generate a binary mask.
[0038] in, This is the empirical threshold for the highlight threshold.
[0039] Morphological expansion of the highlight mask eliminates small gaps and ensures complete coverage of the highlight area:
[0040] in, For morphological expansion operations, K is a 3x3 all-1 expansion kernel. .
[0041] This invention employs the TELEA algorithm based on the fast traversal method, the core of which is to calculate the pixel values of non-highlight regions by distance-weighted averaging.
[0042] in, The neighborhood centered at (x,y) For the weight function, , for and The Euclidean distance.
[0043] Blood vessel features in brain region images have low contrast, and direct matching can easily lead to the loss of key features. This invention enhances contrast through CLAHE and extracts blood vessel texture through Frangi filtering, highlighting the vascular skeleton and improving the accuracy of feature matching.
[0044] Limiting Contrast Adaptive Histogram Equalization (CLAHE) is used to enhance local contrast and avoid noise amplification caused by global equalization.
[0045] in, To limit the contrast threshold, This is the size of the grid.
[0046] Frangi filtering is an enhancement filter for tubular structures, and it is extremely suitable for enhancing vascular features. The core formula is:
[0047] in, Let λ1 be the eigenvalue of the Hessian matrix at pixel (x,y) (λ1≥λ2). S is the flattening, S is the eigenvalue magnitude, and α and β are the filtering parameters, which are 0.5 in this example.
[0048] Binarize the Frangi filter result to obtain the vascular skeleton mask:
[0049] in, The threshold for skeleton binarization is set to 15 in this embodiment.
[0050] After obtaining two images Next, mask feature matching is performed. Single-scale feature matching is prone to losing details or introducing noise, while multi-scale matching can take into account both global and local features. This invention uses LoFTR (Local Feature Transformer) to achieve high-precision multi-scale feature matching.
[0051] Multi-scale scaling of the vascular mask:
[0052] Where s is the scaling factor, It is a scale set.
[0053] Each pair under the three scaling factors Input the LoFTR output of the matching point pairs and their confidence scores, and filter out low-confidence (conf>0.3) matching points:
[0054] in, Let be the set of matching points for the left and right images at scale s, and conf(p) be the confidence score of the matching points. The confidence threshold. This is the set of filtered matching points.
[0055] Restore the coordinates of multi-scale matching points to their original scale:
[0056] in, The coordinates of the matching point at scale s. These are the original scale coordinates.
[0057] Piecewise affine transformation based on Delaunay triangulation: obtaining the matching point set Subsequently, based on the piecewise affine transformation of Delaunay triangulation, the image is divided into multiple independent triangular sub-regions. Each sub-region undergoes an affine transformation independently to accurately adapt to local deformations in the brain region, achieving high-precision registration that takes into account both global and local factors.
[0058] right Perform Delaunay triangulation to generate a non-overlapping triangular mesh topology covering the entire set of matching points:
[0059] in, T The triangulation result consists of M triangular patches, each storing 3 vertices. The index set {i,j,k} in the dataset.
[0060] Based on the one-to-one correspondence between matching point pairs, the vertex indices of the source triangle are directly mapped to the target matching point set p2, generating a target triangle that strictly corresponds to the source triangle:
[0061] in, Let m be the m-th source triangle, which is composed of three vertices with indices {i,j,k} from the source matching point set. The m-th target triangle is composed of three vertices with the same index in the target matching point set, ensuring strict correspondence between the vertices of the source and target triangles. M is the total number of triangles generated by Delaunay triangulation.
[0062] Calculate the area of the source triangle using vector cross product, and filter out degenerate triangles with areas approaching 0 (collinear / approximately collinear vertices):
[0063] in, Source triangular facet The coordinates of the three vertices, Set a minimum area threshold for the pixel area of the source triangle. (in pixels), only non-degenerate triangle facet pairs that meet the area requirements are retained to form an effective computation set:
[0064] in, This is the set of valid triangular facet pairs after filtering; only facets within this set will be included in subsequent affine transformation calculations.
[0065] right For each pair of valid triangular facets within the target facet, estimate the affine transformation matrix from the source facet to the target facet:
[0066] Wherein, the affine transformation matrix It is a 2×3 linear transformation matrix, in the form of:
[0067] The linear mapping relationship from the source patch to the target patch is satisfied:
[0068] in, The pixel coordinates within the source patch. These are the pixel coordinates within the target area.
[0069] For each valid triangular facet corresponding to an image region, an affine transformation is performed and the regions are fused using a mask to avoid pixel conflicts between different triangular regions.
[0070] Where W and H are the width and height of the target image, warpAffine is the affine transformation function, and finally, the facet-by-facet mask is used to fill the image, ensuring that each pixel is only assigned the value by the transformation result of its own triangle, without conflict or overlap.
[0071] Specify point mapping to map any point in the second image to the coordinate system of the third image:
[0072] in, The original size coordinates of the point to be mapped in the second image I2. The coordinates of the top-left corner of the region of interest selected by I2. Let be the coordinates of the point to be mapped within the region of interest in I2. The scale factor for the region of interest of I2. These are the scaled coordinates. The coordinates are after piecewise affine transformation. The coordinates of the top-left corner of the region of interest selected in the third image I3. These are the mapped coordinates on I3. Figure 2 This is a schematic diagram of the registration and mapping points of brain regions after craniotomy and imaging with a head-mounted camera. The red dots are the verification mapping points after registration. Figure 3 A schematic diagram showing a semi-transparent overlay of the registration result.
[0073] In step S4, the first spatial transformation relationship obtained in step S2 is concatenated with the second spatial transformation relationship obtained in step S3, which allows the coordinates of any point in the first image to be transferred to the third image. Figure 4 This is a schematic diagram of brain region localization in the final image of mouse brain neuronal calcium signal activity. The blue dots represent Bregma points.
[0074] This invention establishes a rapid cross-modal cascade registration method. This method eliminates the dependence on endogenous imaging and can accurately locate brain regions in images of neuronal calcium signal activity in the mouse brain using only conventional imaging conditions. This reduces the experimental hardware threshold and implementation cost, and can be widely deployed and applied in conventional biomedical laboratories and preclinical research scenarios that do not have dedicated endogenous imaging equipment.
[0075] Based on the same inventive concept, embodiments of the present invention also provide a cross-modal mouse brain region cascade registration system, such as... Figure 5 As shown, it includes: an image acquisition module, used to acquire a three-channel RGB image of the mouse before craniotomy as the first image, a three-channel RGB image of the mouse after craniotomy as the second image, and a single-channel grayscale image of the brain region after craniotomy as the third image. The first registration module is used to perform non-rigid registration between the first image and the second image, calculate the first spatial transformation relationship from the first image to the second image based on the control point set of the first image and the second image, and align the coordinate system of the first image to the coordinate system of the second image. The second registration module is used to perform cross-modal nonlinear registration between the second image and the third image, extract the morphological features of the meningeal surface veins from the second image and the third image respectively, perform cross-modal feature matching based on the extracted vascular features, and construct a second spatial transformation relationship from the second image to the third image based on the matching results. The cascaded mapping module is used to cascade the first spatial transformation relationship with the second spatial transformation relationship, and sequentially transfer the coordinates of the skull anatomical landmarks to be mapped in the first image to the third image, thereby completing the cascaded registration of the mouse brain regions across modalities.
[0076] Since the principle behind the problem solved by this system is similar to that of the aforementioned cross-modal mouse brain region cascade registration method, the implementation of this system can refer to the implementation of the aforementioned method, and the repetitive parts will not be repeated.
[0077] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A cross-modal mouse brain region cascade registration method, characterized in that, Includes the following steps: S1: Obtain the three-channel RGB image of the mouse before craniotomy as the first image, the three-channel RGB image after craniotomy as the second image, and the single-channel grayscale image of the brain region after craniotomy as the third image. S2: Perform non-rigid registration on the first image and the second image, calculate the first spatial transformation relationship from the first image to the second image based on the control point set of the first image and the second image, and align the coordinate system of the first image to the coordinate system of the second image; S3: Perform cross-modal nonlinear registration on the second image and the third image, extract the morphological features of the meningeal surface veins from the second image and the third image respectively, perform cross-modal feature matching based on the extracted vascular features, and construct a second spatial transformation relationship from the second image to the third image based on the matching results; S4: Concatenate the first spatial transformation relationship with the second spatial transformation relationship, and sequentially transfer the coordinates of the skull anatomical landmarks to be mapped in the first image to the third image to complete the cascaded registration of the mouse brain regions across modalities.
2. The cross-modal mouse brain region cascade registration method as described in claim 1, characterized in that, In step S2, the non-rigid registration specifically includes: The first and second images are downsampled and then displayed, and multiple sets of corresponding control points are collected interactively. The coordinates of the acquired control points are restored to the original image resolution space; The homography transformation matrix from the first image to the second image is solved using the Random Sampling Consensus (RANSAC) algorithm, which serves as the first spatial transformation relationship.
3. The cross-modal mouse brain region cascade registration method as described in claim 1, characterized in that, In step S3, before extracting the morphological features of the veins on the meningeal surface, a preprocessing step is included for the second image and the third image. This preprocessing includes: Region of interest extraction and scale uniformity; Highlight area detection and repair; Enhanced vascular contrast and extraction of vascular structures.
4. The cross-modal mouse brain region cascade registration method as described in claim 3, characterized in that, In step S3, the highlight area detection and repair includes: Highlight regions are extracted by threshold segmentation to generate a binary mask; The binary mask generated in the highlight area is morphologically expanded to eliminate small gaps and completely cover the highlight area. The TELEA algorithm, based on the fast traversal method, is adopted.
5. The cross-modal mouse brain region cascade registration method as described in claim 3, characterized in that, The blood vessel contrast enhancement and blood vessel structure extraction include: Local contrast enhancement of the image is achieved by using contrast-limited adaptive histogram equalization. The Frangi filtering algorithm is used to perform tubular structure enhancement on the image after local contrast enhancement. The Frangi filtering result is binarized to obtain the vascular skeleton mask.
6. The cross-modal mouse brain region cascade registration method as described in claim 1, characterized in that, In step S3, when performing cross-modal feature matching, multi-scale LoFTR feature matching is used, specifically including: Multi-scale scaling is performed on the blood vessel masks corresponding to the second image and the third image to obtain multi-scale image pairs; The image pairs at each scale are input into the LoFTR model, and the matching point pairs and confidence scores are output. Filter out matching pairs with confidence scores below a preset confidence threshold to obtain valid matching pairs; The coordinates of the matching points in the effective matching point pairs are restored to the original scale to obtain the restored matching point set.
7. The cross-modal mouse brain region cascade registration method as described in claim 6, characterized in that, After obtaining the restored matching point set, when constructing the second spatial transformation relationship from the second image to the third image based on the matching results, a piecewise affine transformation based on Delaunay triangulation is adopted, specifically including: Perform Delaunay triangulation on the restored matching point set to generate a non-overlapping triangular mesh topology covering the entire restored matching point set; Based on the one-to-one correspondence of matching point pairs in the restored matching point set, the vertex indices of the source triangle are mapped to the target points in the restored matching point set to generate the target triangle. The area of the source triangle is calculated by vector cross product, and source triangles with areas close to 0 are filtered out to obtain the effective computation set. For each set of valid triangular faces within the valid computation set, calculate the affine transformation matrix from the source triangular face to the target triangular face; For each valid triangular facet corresponding to the image region, perform an affine transformation and fuse it using a mask to obtain the registered image.
8. The cross-modal mouse brain region cascade registration method as described in claim 7, characterized in that, After obtaining the registered image, the transformation relationship from the second image to the third image is obtained: in, The original size coordinates of the point to be mapped in the second image I2. The coordinates of the top-left corner of the region of interest selected by I2. Let be the coordinates of the point to be mapped within the region of interest in I2. The scale factor for the region of interest of I2. These are the scaled coordinates. The coordinates are after piecewise affine transformation. The coordinates of the top-left corner of the region of interest selected in the third image I3. These are the mapped coordinates on I3.
9. A cross-modal mouse brain region cascade registration system, characterized in that, include: The image acquisition module is used to acquire a three-channel RGB image of the mouse before craniotomy as the first image, a three-channel RGB image after craniotomy as the second image, and a single-channel grayscale image of the brain region after craniotomy as the third image. The first registration module is used to perform non-rigid registration between the first image and the second image, calculate the first spatial transformation relationship from the first image to the second image based on the control point set of the first image and the second image, and align the coordinate system of the first image to the coordinate system of the second image. The second registration module is used to perform cross-modal nonlinear registration between the second image and the third image, extract the morphological features of the meningeal surface veins from the second image and the third image respectively, perform cross-modal feature matching based on the extracted vascular features, and construct a second spatial transformation relationship from the second image to the third image based on the matching results. The cascaded mapping module is used to cascade the first spatial transformation relationship with the second spatial transformation relationship, and sequentially transfer the coordinates of the skull anatomical landmarks to be mapped in the first image to the third image, thereby completing the cascaded registration of the mouse brain regions across modalities.