A method and system for registration of spatial metabolic composition images with HE stained images

Abstract

This invention relates to the fields of biomedical imaging and spatial metabolomics, specifically to a method and system for registering spatial metabolomics images with HE-stained images. The method includes the following steps: acquiring two adjacent frozen sections of the same tissue sample; performing image preprocessing using the frozen sections; obtaining the extraction and matching results of key point features in the images; performing rigid image registration and obtaining a rigid registration result; performing non-rigid image registration and obtaining a non-rigid registration result; and obtaining the spatial location information of the spatial metabolic imaging using the non-rigid registration result. This invention, by combining advanced image processing technology and machine learning algorithms, automatically registers metabolic imaging images with HE-stained images, significantly improving registration accuracy and efficiency. It effectively solves the problems of large errors, low efficiency, and strong operational dependence in existing manual registration methods, meeting the needs of high-throughput, high-precision spatial metabolomics analysis and supporting large-scale data processing.

Description

Technical Field

[0001] This invention relates to the fields of biomedical imaging and spatial metabolomics, specifically to a method and system for registering spatial metabolomics images with HE staining images. Background Technology

[0002] Spatial metabolomics is an advanced mass spectrometry imaging technique that enables the detection and analysis of various endogenous metabolites and exogenous drug molecules in biological tissue samples. This technique provides crucial information about molecular structure, concentration, and precise spatial distribution within tissues. To accurately resolve the spatial distribution of these metabolites, precise registration of metabolic imaging data with HE-stained images of adjacent tissue sections is typically required.

[0003] In existing technologies, the registration process mainly relies on manual operation, where operators align metabolic imaging images with HE-stained images through rotation, scaling, translation, and flipping. However, this process has the following main problems: First, it depends on the operator's subjective judgment, making it difficult to achieve completely accurate alignment during rotation, scaling, and flipping, resulting in large spatial positioning errors; second, the registration process is time-consuming and highly dependent on the operator's experience and skills, leading to low work efficiency and poor repeatability of registration results between different operators; third, due to the operation and processing steps during slicing, differences may occur between slices, causing visual inconsistencies between metabolic imaging images and HE-stained images, further increasing the complexity of manual alignment, especially when highly precise positioning is required, as subtle differences can lead to significant analytical errors.

[0004] Currently, there is not enough research on the registration of spatial metabolic composition images with HE staining images, and there is no specific automated, accurate and efficient registration method. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for registering spatial metabolic composition images with HE staining images.

[0006] In a first aspect, the present invention provides a method for registering a spatial metabolomics image with an HE staining image, comprising the following steps: acquiring two adjacent frozen sections of the same tissue sample; performing image preprocessing on the two adjacent frozen sections and obtaining processing results, wherein the image preprocessing includes HE staining and spatial metabolomics sequencing; based on the processing results, obtaining the extraction and matching results of key point features of the image; performing rigid registration of the image based on the extraction and matching results and obtaining rigid registration results; performing non-rigid registration of the image based on the rigid registration results and obtaining non-rigid registration results, wherein the non-rigid registration of the image includes constructing a deformation field for adjusting image deformation; and obtaining spatial location information of the spatial metabolic imaging through the non-rigid registration results. This invention achieves high-precision registration of HE-stained images and spatial metabolic composition images by employing image preprocessing techniques on adjacent frozen sections. This overcomes the image information mismatch problem caused by slice intervals in traditional methods, significantly improving the accuracy and reliability of image registration. By introducing key point feature extraction and matching techniques, it effectively solves the difficulty of feature point matching caused by tissue sample heterogeneity, achieving efficient registration of complex tissue samples and overcoming the limitations of existing technologies in feature matching. By combining rigid and non-rigid registration strategies, it preserves the overall structural consistency of the image while finely adjusting local deformations, overcoming the shortcomings of single registration methods in handling complex deformations and achieving high-precision correction of tissue sample deformations. By constructing and applying a deformation field, it accurately obtains the spatial location information of spatial metabolic imaging, overcoming the ambiguity in spatial positioning of traditional methods.

[0007] Optionally, the step of performing image preprocessing and obtaining processing results using the two adjacent frozen sections includes: using digital pathological slide scanning technology to scan the spatial structure of the first section and obtain a spatial structure image; based on the spatial structure image, performing HE staining on the second section and obtaining a staining result; based on the staining result, using the digital pathological slide scanning technology to scan the spatial structure of the second section and obtain a reference image; based on the reference image, performing spatial metabolomics sequencing on the first section to obtain a spatial metabolic imaging image; and registering the spatial structure image and the spatial metabolic imaging image to obtain the specific spatial location of the spatial metabolic imaging data. This invention utilizes digital pathological slide scanning technology to acquire spatial structural images of a first slide and combines this with HE staining results of a second slide to generate a reference image. This achieves high-precision alignment of multimodal images of the same tissue sample, overcoming the image registration error problem caused by different slide processing orders in traditional methods, and significantly improving the positioning accuracy of spatial metabolic imaging data. By performing spatial metabolomics sequencing on the first slide and registering it based on the reference image, efficient fusion of metabolomics data and tissue structure is achieved, solving the problem of spatial information separation between metabolomics data and pathological images in existing technologies, and providing more accurate spatial positioning support for metabolomics research. By registering spatial structural images with spatial metabolic imaging images, precise mapping of the specific spatial location of metabolomics data in tissue samples is achieved, overcoming the ambiguity and limitations of traditional methods in spatial metabolomics data analysis. By integrating digital pathological slide scanning technology and spatial metabolomics sequencing technology, efficient acquisition and fusion of multidimensional information of the same tissue sample is achieved, overcoming the technical bottleneck of existing technologies in multimodal data integration.

[0008] Optionally, obtaining the extraction and matching results of image key point features based on the processing results includes: extracting image key points and descriptors based on the spatial structure image of the first slice and the reference image of the second slice to obtain extraction results. The key points include corner points, edges, and obvious texture areas in the image, and the descriptors include unique image features around the key points. Based on the extraction results, comparing the similarity between the spatial structure image and the reference image, filtering and matching the key points to obtain matching results. This invention extracts key points and descriptors from a spatial structure image based on a first slice and a reference image based on a second slice, achieving high-precision identification of corners, edges, and textured regions in complex tissue samples. This overcomes the reliance of traditional methods on single-type key points in feature extraction, significantly improving the comprehensiveness and robustness of feature extraction. By encoding unique image features around key points using descriptors, it achieves deep representation of local image features, solving the problem of low matching accuracy caused by insufficient feature description in existing technologies, and providing more reliable feature support for multimodal image registration. By comparing the similarity between the spatial structure image and the reference image and selecting matching key points, it achieves efficient matching of multimodal image features, overcoming the technical bottleneck of traditional methods in cross-modal image matching, and significantly improving the accuracy and stability of matching results. By combining a dual strategy of key point extraction and descriptor matching, it achieves high-precision alignment of fine structures in complex tissue samples, overcoming the excessive reliance of existing technologies on global features in image registration.

[0009] Optionally, the step of comparing the spatial structure image and the reference image based on the extraction results, filtering and matching the key points, and obtaining matching results includes: establishing a target matching algorithm model based on the extraction results; filtering and matching the key points of the spatial structure image and the reference image through the target matching algorithm model to obtain matching results. This invention, by establishing a target matching algorithm model, achieves efficient filtering and matching of key points in spatial structure images and reference images, breaking through the reliance on manual intervention in the matching process of traditional methods, and significantly improving the automation and efficiency of the matching process; by accurately matching key points through the target matching algorithm model, it solves the problem of mismatch caused by image noise or deformation in existing technologies, significantly improving the accuracy and reliability of matching results; by combining a dual optimization strategy of key point extraction and target matching algorithm models, it achieves high-precision alignment of multimodal image features, overcoming the technical bottleneck of traditional methods in cross-modal image matching; by using the target matching algorithm model to efficiently filter and match key points in complex tissue samples, it achieves high-precision registration of fine structures, breaking through the excessive reliance of existing technologies on global features in image registration.

[0010] Optionally, the target matching algorithm model satisfies the following expression:

[0011]

[0012] in, The transformation matrix is... For the first slice of spatial structure image The coordinates of the matching points For the second slice reference image The coordinates of the matching points The present invention utilizes a target matching algorithm model to achieve high-precision transformation matrix calculation for keypoint coordinates in spatial structure images and reference images. This overcomes the dependence of traditional methods on local optima in transformation matrix calculation, significantly improving the global optimality and stability of registration. By minimizing the error between matching point coordinates, efficient alignment of multimodal image features is achieved, solving the problem of low registration accuracy caused by image deformation or noise in existing technologies. By combining the optimization calculation of the transformation matrix with a dual strategy of keypoint matching, high-precision registration of fine structures in complex tissue samples is achieved, overcoming the excessive dependence of traditional methods on single-type features in image registration, and significantly improving the robustness and applicability of the registration results. By using the target matching algorithm model to globally optimize the matching point coordinates, efficient correction of image deformation is achieved, overcoming the technical bottleneck of existing technologies in non-rigid registration.

[0013] Optionally, the step of performing rigid image registration and obtaining rigid registration results based on the extraction and matching results includes: uniformly rotating and translating the position of each pixel point on the first slice spatial structure image based on the extraction and matching results to obtain rotation and translation results; and rigidly registering the first slice spatial structure image and the second slice reference image based on the rotation and translation results to obtain rigid registration results. This invention achieves high-precision alignment of the overall structure of an image by uniformly rotating and translating each pixel on the first slice spatial structure image. This overcomes the excessive reliance on local features in rigid registration of traditional methods, significantly improving the global consistency and stability of registration. By applying the rotation and translation results to rigid registration, efficient alignment of the first slice spatial structure image and the second slice reference image is achieved, solving the registration error problem caused by image rotation or translation in the prior art, and providing a more reliable spatial basis for multimodal image fusion. By combining the dual optimization strategy of rotation and translation and rigid registration, high-precision correction of the overall structure in complex tissue samples is achieved, overcoming the limitations of traditional methods in image registration for a single transformation type, and significantly improving the applicability and robustness of the registration results.

[0014] Optionally, the step of performing non-rigid registration of the image based on the rigid registration result and obtaining the non-rigid registration result includes: constructing a deformation field of the first slice spatial structure image based on the rigid registration result; establishing a target deformation field optimization algorithm model using the deformation field; adjusting the deformation of the first slice spatial structure image according to the target deformation field optimization algorithm model to determine the final deformation shape; and performing non-rigid registration of the first slice spatial structure image and the second slice reference image using the final deformation shape to obtain the non-rigid registration result. This invention achieves high-precision modeling of local image deformation by constructing a deformation field of the first slice spatial structure image, overcoming the oversimplification of global deformation in traditional methods in non-rigid registration, and significantly improving the local adaptability and accuracy of registration. By using a target deformation field optimization algorithm model to adjust the deformation of the first slice spatial structure image, it achieves efficient correction of fine structures in complex tissue samples, solving the problem of insufficient registration accuracy caused by complex local deformation in existing technologies, and providing more refined spatial information support for multimodal image fusion. Furthermore, by constructing a target deformation field optimization algorithm model, it achieves efficient optimization and adjustment of image deformation, overcoming the dependence of traditional methods on a single deformation model in non-rigid registration, and significantly improving the robustness and applicability of the registration results.

[0015] Optionally, the target deformation field optimization algorithm model satisfies the following expression:

[0016]

[0017] in, The deformation field of the spatial structure image of the first slice. For similarity measurement, For regularization terms, This represents the reference image for the second slice. This is a deformation image of the spatial structure of the first slice. These are the weighting coefficients. For the image at points along Displacement in the direction of For the image at points along The displacement in the direction. This invention achieves high-precision optimization of the deformation field of the first slice spatial structure image by utilizing a target deformation field optimization algorithm model. It breaks through the dependence of traditional methods on a single target in deformation optimization, significantly improving the global optimality and local adaptability of registration. By maximizing the similarity metric and introducing a regularization term, it achieves efficient constraint and adjustment of image deformation, solving the registration error problem caused by excessive or insufficient deformation in the prior art, and providing more accurate spatial information support for multimodal image fusion. By combining the dual optimization strategy of similarity metric and regularization term, it achieves high-precision registration of fine structures in complex tissue samples, overcoming the limitations of traditional methods on a single deformation model in non-rigid registration, and significantly improving the robustness and applicability of the registration results.

[0018] Optionally, obtaining the spatial location information of the spatial metabolic imaging data through the non-rigid registration result includes: obtaining the final deformation shape of the spatial structure image of the first slice through the non-rigid registration result; applying the deformation field corresponding to the final deformation shape to the spatial metabolic imaging data to obtain the registration result of the spatial metabolic imaging image of the first slice and the reference image of the second slice; and obtaining the spatial location information of the spatial metabolic imaging data through the registration result. This invention achieves high-precision registration of the spatial metabolic imaging image and the HE slice reference image by applying the deformation field to the spatial metabolic imaging data, breaking through the dependence of traditional methods on a single registration strategy in cross-modal image registration, and significantly improving the accuracy and reliability of the registration results; by obtaining the specific spatial location information of the spatial metabolic imaging, it solves the problem of inaccurate metabolomics data analysis caused by spatial information separation in the prior art, providing more accurate spatial positioning support for metabolomics research; by combining the dual optimization strategy of deformation field and registration result, it achieves efficient mapping of metabolomics data in complex tissue samples, overcoming the technical bottleneck of traditional methods in spatial metabolomics data analysis.

[0019] Secondly, the present invention provides a registration system for spatial metabolic composition images and HE staining images, comprising an input device, a processor, an output device, and a memory, wherein the input device, the processor, the output device, and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprising program instructions, the processor being configured to call the program instructions, and the system using the aforementioned registration method for spatial metabolic composition images and HE staining images. The system provided by this invention has a high degree of system integration and smooth information transmission between its components. By employing image preprocessing technology for adjacent frozen sections, combined with digital pathological section scanning and spatial metabolomics sequencing, it achieves high-precision registration between HE staining images and spatial metabolomics images. This overcomes the image information mismatch problem caused by different section intervals or processing orders in traditional methods, significantly improving the accuracy and reliability of multimodal image registration. By introducing key point feature extraction and matching technology, combined with target matching algorithm models and deformation field optimization algorithm models, it achieves efficient alignment of corners, edges, and texture regions in complex tissue samples, solving the problem of low registration accuracy caused by insufficient feature matching or complex deformation in existing technologies. By combining rigid and non-rigid registration strategies and utilizing deformation fields to perform high-precision registration of spatial metabolic imaging data with HE sections, it achieves accurate mapping of spatial location information of metabolomics data, overcoming the ambiguity and limitations of traditional methods in spatial metabolomics data analysis, and providing an important solution for precision medicine research. Attached Figure Description

[0020] Figure 1 This is a flowchart of the registration method between a spatial metabolic composition image and an HE staining image according to an embodiment of the present invention;

[0021] Figure 2 This is a slice spatial structure and metabolic imaging diagram according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of the registration of the first slice spatial structure image and the second slice reference image in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the registration system for spatial metabolic composition images and HE staining images according to an embodiment of the present invention;

[0024] Figure 5 This is a flowchart illustrating the operation of the registration system for spatial metabolic composition images and HE staining images according to an embodiment of the present invention. Detailed Implementation

[0025] Specific embodiments of the present invention will now be described in detail. It should be noted that the embodiments described herein are for illustrative purposes only and are not intended to limit the invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to practice the invention. In other instances, well-known circuits, software, or methods have not been specifically described to avoid obscuring the invention.

[0026] Throughout this specification, references to "an embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with that embodiment or example is included in at least one embodiment of the invention. Therefore, the phrases "in an embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily refer to the same embodiment or example. Furthermore, specific features, structures, or characteristics can be combined in one or more embodiments or examples in any suitable combination and / or sub-combination. Moreover, those skilled in the art will understand that the illustrations provided herein are for illustrative purposes and are not necessarily drawn to scale.

[0027] Please see Figure 1 The present invention provides a method for registering a spatial metabolic composition image with an HE staining image, the method comprising the following steps:

[0028] S1. Obtain two adjacent frozen sections of the same tissue sample.

[0029] In one embodiment, a suitable tissue sample is first selected to ensure that the sample is fresh and uncontaminated, and it is processed as soon as possible after collection to preserve the original metabolic state of the sample to the greatest extent.

[0030] Furthermore, the sample surface is washed with physiological saline to remove substances that may affect the quality of the slides, including blood and impurities.

[0031] Furthermore, the samples are trimmed to an appropriate size to facilitate subsequent embedding and sectioning. It is important to note that the sample size should not exceed a certain range, and the cross-section should be large enough to prevent tissue wrinkling or insufficient cell nuclei during sectioning.

[0032] Furthermore, the embedding solution and the equipment needed for sectioning are pre-cooled to prevent ice crystals from forming and damaging the tissue structure during the embedding process.

[0033] Next, place the target tissue in pre-cooled embedding solution, ensuring that the tissue is completely encapsulated by the embedding solution. During the embedding process, care should be taken to avoid generating air bubbles and to ensure that the tissue is correctly positioned in the embedding cassette for subsequent sectioning operations.

[0034] Next, the embedded tissue blocks are placed in appropriate cryostats for rapid freezing. The freezing time depends on the tissue type and size, and it is necessary to ensure that the tissue is completely solidified and turns white.

[0035] Furthermore, an advanced cryostat is used for slicing operations. The parameters of the cryostat, including slice thickness and slicing speed, are adjusted according to the tissue type and experimental requirements. The cryostat includes LEICA and CM1950 models.

[0036] Next, place the frozen tissue block on the microtome's sample stage, adjust the sectioning position, and begin sectioning. During the sectioning process, it is important to maintain the stability of the microtome and the consistency of the sectioning speed to ensure high-quality frozen sections.

[0037] Furthermore, during the sectioning process, two consecutive frozen sections are acquired and labeled as section 1 and section 2, respectively. These two sections should have the same tissue structure and metabolic characteristics to facilitate subsequent spatial metabolomics analysis.

[0038] This method, through a series of meticulous steps, ensures that tissue samples retain their original metabolic state during slicing, thus providing high-quality samples for spatial metabolomics analysis. Compared to existing techniques, this method offers significant advantages in metabolic state preservation, slice quality, and data consistency, providing more reliable data support for spatial metabolomics research.

[0039] S2. Using the two adjacent frozen sections, perform image preprocessing and obtain the processing results. The image preprocessing includes HE staining and spatial metabolomics sequencing.

[0040] S2 includes the following steps:

[0041] S21. Using digital pathological slide scanning technology, scan the spatial structure of the first slide to obtain a spatial structure image.

[0042] In one embodiment, the first slice is first placed on a multimodal digital pathology scanner.

[0043] Furthermore, configure scanner parameters, including scanning speed and resolution, to ensure image quality.

[0044] Furthermore, a scanning procedure is initiated, combining optical coherence tomography and secondary ion mass spectrometry imaging techniques to obtain information on the optical structure and chemical composition of the slices.

[0045] Furthermore, AI-driven autofocus and image enhancement algorithms are applied to optimize the quality of scanned images in real time.

[0046] Furthermore, the scanned spatial structure images are saved, ensuring they are clear, noise-free, and free of artifacts. The spatial structure images are as follows: Figure 2 As shown in 'a'.

[0047] This method achieves the acquisition and preservation of high-quality slide images through multimodal digital pathology scanning technology and AI-driven image optimization algorithms. Compared with existing technologies, this method has significant advantages in multimodal data acquisition, real-time image optimization, and image quality assurance. It can provide more comprehensive and accurate data support for research in spatial metabolomics and pathology, not only improving experimental efficiency but also enhancing data reliability and the accuracy of analysis results.

[0048] S22. Based on the spatial structure image, the second slice is stained with HE and the staining result is obtained.

[0049] In one embodiment, based on the spatial structure image, image preprocessing is performed, namely, HE staining of the second slice.

[0050] Specifically, the second slice is first placed on the intelligent staining robot system.

[0051] Furthermore, by combining the spatial structure image of the first slice, the target region is automatically identified and the staining path is planned. It should be noted that the spatial structure of the first and second slices is the same before preprocessing.

[0052] Furthermore, configure the HE staining parameters, including staining time and dye concentration.

[0053] Furthermore, an intelligent staining program is activated to precisely control the staining process and ensure uniform and consistent staining.

[0054] Further examine the staining results to ensure that the target area is clear and has high contrast.

[0055] Furthermore, the stained sections are preserved for subsequent scanning and analysis.

[0056] This method automates HE staining of the second slide using an intelligent staining robot system. Combined with the spatial structure image of the first slide, it enables precise staining path planning and parameter control, significantly improving staining efficiency and result consistency. Compared to existing technologies, this method offers the following advantages: Based on the spatial structure image of the first slide, the intelligent staining robot can automatically identify the target area and plan the staining path, avoiding the problems of uneven staining or missed target areas caused by human error in traditional manual staining; By configuring staining parameters and starting the intelligent staining program, the staining process is precisely controlled, ensuring staining uniformity and consistency, reducing the uncertainty of human operation; The intelligent staining system can quickly complete the staining process and ensure clear target areas and high contrast by checking the staining results, significantly improving staining efficiency and result reproducibility; Fourthly, because the first and second slides have the same spatial structure before preprocessing, the staining results are highly matched with spatial metabolomics data, ensuring data consistency and reliability for subsequent analysis. Therefore, this intelligent staining technology not only improves experimental efficiency but also enhances the accuracy and reproducibility of staining results, providing higher-quality data support for pathological research and spatial metabolomics analysis.

[0057] S23. Based on the staining results, the spatial structure of the second slice is scanned using the digital pathological slide scanning technology to obtain a reference image.

[0058] In one embodiment, the stained second slide is placed on a super-resolution digital pathology scanner, and the scanner parameters are configured to ensure that the scanning resolution reaches the 10nm level.

[0059] Furthermore, the scanning process is initiated, combining deep learning super-resolution reconstruction algorithms and adaptive optics technology to acquire high-resolution images of the slices, and the optical parameters are monitored and adjusted in real time to ensure high image fidelity, no distortion, and no artifacts.

[0060] Furthermore, the scanned reference image is saved, and the reference image is preprocessed as necessary, including denoising and contrast enhancement, to improve the accuracy of subsequent registration.

[0061] This method utilizes a super-resolution digital pathology scanner combined with a deep learning-based super-resolution reconstruction algorithm and adaptive optics technology to achieve high-resolution image acquisition of the second stained slide, significantly improving image quality and the accuracy of subsequent analysis. Compared to existing technologies, this method offers the following advantages: a scanning resolution reaching the 10nm level, far exceeding the resolution of traditional pathology scanners, enabling the capture of finer tissue structural details; the combination of deep learning algorithms and adaptive optics technology to monitor and adjust optical parameters in real time, ensuring high image fidelity, distortion-free images, and artifact-free images; and preprocessing of the reference image to denoise and enhance contrast, further improving image quality and providing a more accurate data foundation for subsequent registration and analysis, significantly enhancing the ability to restore image details and the reliability of experimental results.

[0062] S24. Based on the reference image, perform spatial metabolomics sequencing on the first slice to obtain a spatial metabolic imaging image.

[0063] In one embodiment, the first slice is first placed on a high-throughput spatial metabolomics sequencing platform, and the sequencing path is planned according to the target region in the reference image.

[0064] Furthermore, the parameters for mass spectrometry imaging and Raman spectroscopy imaging techniques are configured.

[0065] Furthermore, the sequencing process is initiated to detect metabolites in the target region with high sensitivity and high spatial resolution.

[0066] Furthermore, in situ labeling technology for metabolites is applied to locate and quantify specific metabolites.

[0067] Furthermore, the spatial metabolic imaging data obtained from sequencing are preserved, and the data are processed as necessary, including denoising and calibration, to improve the accuracy of subsequent analyses.

[0068] Furthermore, using the aforementioned spatial metabolic imaging data, spatial metabolic imaging images are generated, such as... Figure 2 As shown in b in the figure.

[0069] This method, combining a high-throughput spatial metabolomics sequencing platform with mass spectrometry and Raman spectroscopy, achieves high-sensitivity, high-spatial-resolution metabolite detection in target regions, significantly improving the accuracy and efficiency of metabolomics analysis. Compared to existing technologies, this method offers the following advantages: precise sequencing path planning based on reference images ensures comprehensive coverage of the target region; the combination of mass spectrometry and Raman spectroscopy enables high-sensitivity, high-resolution metabolite detection; in-situ metabolite labeling allows for precise localization and quantification of specific metabolites; denoising and calibration of sequencing data improves accuracy and reliability; and the resulting spatial metabolic imaging provides more intuitive and comprehensive data support for metabolomics research, significantly improving the resolution and efficiency of metabolite detection and offering higher-quality technical means for spatial metabolomics research.

[0070] S25. Register the spatial structure image and the spatial metabolic imaging image to obtain the specific spatial location of the spatial metabolic imaging data.

[0071] In one embodiment, spatial structure images and spatial metabolic imaging data are loaded first.

[0072] Furthermore, the parameters of the deep learning-based multimodal image registration algorithm are configured.

[0073] Furthermore, the registration process is initiated, and graph neural networks and generative adversarial networks are combined to monitor the registration process in real time, achieving high-precision, nonlinear registration.

[0074] Furthermore, the specific spatial location of the registered spatial metabolic imaging data is preserved, such as... Figure 2 As shown in c in the figure.

[0075] This embodiment achieves high-precision, nonlinear registration of spatial structural images and spatial metabolic imaging data by employing a deep learning-based multimodal image registration algorithm combined with graph neural networks and generative adversarial networks, significantly improving the accuracy and efficiency of data integration. Compared to existing technologies, this method has the following advantages: it utilizes deep learning algorithms to achieve complex nonlinear registration, overcoming the shortcomings of traditional linear registration methods in terms of accuracy and adaptability; it combines graph neural networks and generative adversarial networks to monitor the registration process in real time, ensuring the accuracy and stability of the registration results; and by preserving the specific spatial location information of the registered spatial metabolic imaging data, it provides a more reliable data foundation for subsequent analysis, significantly improving the accuracy and efficiency of multimodal data registration and providing higher-quality technical support for spatial metabolomics research.

[0076] S3. Based on the processing results, obtain the extraction and matching results of key point features in the image.

[0077] S3 includes the following steps:

[0078] S31. Based on the spatial structure image of the first slice and the reference image of the second slice, extract the key points and descriptors of the image to obtain the extraction results.

[0079] In one embodiment, keypoints and descriptors of an image are extracted by introducing a convolutional neural network and a feature pyramid network. The feature pyramid network can capture multi-scale features, making keypoint detection more robust. Specifically, a pre-trained deep learning model is used as the backbone network of the feature extractor to capture high-level semantic information in the image, i.e., keypoint information; a dedicated keypoint detection head is designed, utilizing an attention mechanism to enhance the accuracy of keypoint localization.

[0080] Furthermore, using deep metric learning techniques, a descriptor generation network is designed that can learn unique image features around keypoints and generate discriminative descriptor vectors.

[0081] Furthermore, local and global contextual information is introduced, and the expressive power of the descriptor is enhanced by fusing feature maps of different scales.

[0082] Furthermore, a self-supervised learning method is employed to generate pseudo-labels using unlabeled image data through image transformations, including rotation, scaling, and cropping, to train the keypoint detector and descriptor generator.

[0083] It should be noted that the key points include corners, edges, and obvious textured areas in the image, and the descriptor includes unique image features around the key points.

[0084] S32. Based on the extraction results, establish a target matching algorithm model, filter and match the key points of the spatial structure image and the reference image, and obtain the matching results.

[0085] In one embodiment, based on step S31, key points are filtered and paired by comparing the similarity of descriptors in the spatial structure image and the reference image, such as... Figure 3 As shown in 'a', during the matching process, a target matching algorithm model is established to identify a consistent set of matches and eliminate matches that do not conform to the model, thereby maximizing the alignment of the positions of the two sets of key points. The target matching algorithm model satisfies the following expression:

[0086]

[0087] in, For the first slice of spatial structure image The coordinates of a matching point, also known as a keypoint for matching. For the second slice reference image The coordinates of the matching points The number of matching points, Let be the transformation matrix.

[0088] Furthermore, the transformation matrix is ​​initially calculated using the aforementioned target matching algorithm model. .

[0089] Furthermore, The input is fed into a lightweight convolutional neural network that is pre-trained to dynamically adjust the parameters of the transformation matrix, including rotation angles and translations, based on the image content.

[0090] Furthermore, the network finds the optimal transformation matrix by minimizing the matching error between the first slice spatial structure image and the second slice reference image, thereby enabling accurate correspondence matching between images. The matching error is also called the transformation error.

[0091] It should be noted that this invention optimizes the transformation matrix through deep learning, which significantly improves the registration accuracy, especially in the presence of local deformation or noise.

[0092] This method combines a target matching algorithm model with a lightweight convolutional neural network to achieve high-precision keypoint matching and transformation matrix optimization between spatial structure images and reference images, significantly improving the accuracy and efficiency of image registration. Compared with existing technologies, this method has the following advantages: it filters and pairs keypoints through a target matching algorithm model, eliminating inconsistent matches and ensuring the accuracy of the initial alignment; it uses a pre-trained lightweight convolutional neural network to dynamically adjust the transformation matrix parameters, further optimizing the rotation angle and translation, and minimizing the matching error; combining the advantages of deep learning and traditional algorithms, it achieves high-precision, adaptive image registration, overcoming the limitations of traditional methods in complex nonlinear transformations, significantly improving the accuracy and robustness of multimodal image registration, and providing more reliable technical support for spatial metabolomics research.

[0093] S4. Based on the extraction and matching results, perform rigid image registration and obtain the rigid registration result.

[0094] S4 includes the following steps:

[0095] S41. Based on the extraction and matching results, rotate and translate each key point position on the first slice spatial structure image in a uniform manner to obtain the rotation and translation results.

[0096] In one embodiment, the optimal transformation matrix obtained from the target matching algorithm model is... When applied to the first slice spatial structure image, the rotation and translation results are obtained by uniformly rotating and translating each pixel position on the first slice spatial structure image.

[0097] This method provides an efficient, accurate, and automated image alignment approach through a unified rotation and translation transformation combined with global optimization of the optimal transformation matrix, demonstrating significant innovation and practicality.

[0098] S42. Based on the rotation and translation results, rigidly register the first slice spatial structure image and the second slice reference image to obtain the rigid registration result.

[0099] In one embodiment, based on the rotation and translation result, the first slice spatial structure image and the second slice reference image are rigidly registered to obtain a rigid registration result, such as... Figure 3 As shown in b in the figure.

[0100] Traditional rigid registration methods typically rely on local feature matching, iterative optimization, or manual adjustments. This new method, however, offers a novel solution through a unified rotation and translation result and a global optimization strategy. This approach not only simplifies the registration process but also improves registration accuracy and robustness.

[0101] S5. Based on the rigid registration result, perform non-rigid image registration and obtain a non-rigid registration result, wherein the non-rigid image registration includes constructing a deformation field for adjusting image deformation.

[0102] In one embodiment, based on the rigid registration result, a deformation field of the first slice spatial structure image is constructed, and the deformation field of the first slice spatial structure image is as follows:

[0103]

[0104] Furthermore, using the deformation field, an optimization algorithm model for the target deformation field is established; the optimization algorithm model for the target deformation field satisfies the following expression:

[0105]

[0106] in, The deformation field of the first slice spatial structure image determines the deformation result of the first slice spatial structure image. This is a similarity metric used to measure the similarity between the deformed spatial structure image of the first slice and the reference image of the second slice. This is a regularization term used to ensure the smoothness and physical feasibility of the deformation field. This represents the reference image for the second slice. This is a deformation image of the spatial structure of the first slice. These are the weighting coefficients. For the image at points along Displacement in the direction of For the image at points along The amount of displacement in the direction.

[0107] Furthermore, based on the target deformation field optimization algorithm model, the spatial structure image of the first slice is deformed and adjusted, and the deformation field most similar to the reference image of the second slice is selected, such as... Figure 3 As shown in c in the figure.

[0108] Furthermore, the final deformation morphology of the first slice spatial structure image is determined, such as... Figure 3 As shown in d.

[0109] Furthermore, through the final deformation shape, non-rigid registration of the first slice spatial structure image and the second slice reference image is achieved, and the result of the non-rigid registration is as follows: Figure 3 As shown in e.

[0110] Traditional methods typically rely on simple rigid transformations or local affine transformations, which struggle to handle complex non-rigid deformations. This new method, however, constructs a pixel-level deformation field and optimizes it using a target deformation field optimization algorithm model, achieving high-precision description and adjustment of complex deformations and overcoming the limitations of traditional registration techniques. By establishing a target deformation field optimization algorithm model and combining similarity metrics and regularization terms, the smoothness and physical feasibility of the deformation field are ensured, providing a novel mathematical framework for the registration problem and significantly improving registration accuracy and robustness. Furthermore, by automatically adjusting the deformation field and determining the final deformation morphology, the method automates non-rigid registration, reducing reliance on manual intervention and overcoming the limitations of traditional methods that require extensive manual adjustments and parameter optimization.

[0111] S6. Using the non-rigid registration results, obtain the spatial location information of spatial metabolic imaging.

[0112] In one embodiment, based on the non-rigid registration result, it is first checked whether the registration error during the iteration process has stably converged to a small threshold to ensure that the registration parameters, including the smoothness of the deformed model and the number of iterations, have been optimized to obtain the best registration effect.

[0113] Furthermore, based on the converged registration results, the deformation field from the first slice spatial structure image to the second slice reference image is calculated. The deformation field is represented as a spatial transformation function, which describes the mapping of each pixel from its original position to its optimal deformed position.

[0114] Traditional methods typically rely on fixed registration parameters, lacking dynamic monitoring and optimization of registration errors. This new method, however, ensures optimized registration parameters by checking whether the registration error stably converges to a small threshold during iteration, thus significantly improving registration accuracy and stability and overcoming the limitations of traditional fixed-parameter registration. Furthermore, by calculating the deformation field from the first slice of spatial structure image to the second slice of reference image, it achieves pixel-level spatial transformation mapping, overcoming the limitations of traditional methods based on coarse alignment and providing more precise technical support for spatial alignment of multimodal data. The deformation field is described as a spatial transformation function that accurately describes the mapping of each pixel from its original position to its optimal deformed position, providing a theoretical basis for quantitative analysis and subsequent applications of registration results, overturning the limitations of traditional methods that rely on empirical or qualitative analysis.

[0115] Furthermore, a deformation field is used to spatially transform the spatial metabolic imaging data, aligning it spatially with the HE slice (the second slice). This process involves applying the corresponding transformation in the deformation field to each pixel in the metabolic imaging data.

[0116] Furthermore, the registered metabolic imaging data and HE slices were visually examined to ensure anatomical consistency. This process used overlap and quantitative mutual information metrics to assess registration quality.

[0117] Furthermore, in the registered metabolic imaging data, based on the results of the deformation field application, the spatial location information of the region of interest is extracted. The spatial location information includes the coordinates, size, shape, and relative position of the region of interest to specific anatomical structures in the HE slice.

[0118] Furthermore, the extracted spatial location information is combined with the intensity information of metabolic imaging data for further analysis and interpretation to obtain the final integrated spatial location information. This process involves comparing metabolic differences between different regions of interest and assessing the relationship between metabolic changes and anatomical structures.

[0119] Traditional methods typically analyze metabolic imaging data and HE slice data independently, making it difficult to achieve spatial alignment and information integration between the two. This new method, however, revolutionizes the traditional paradigm of multimodal data analysis through spatial transformation and fusion analysis, providing a completely new technical approach for research in related fields. By achieving high-precision alignment and integration of metabolic imaging data with anatomical structures, it provides strong technical support for precision medicine, enabling more accurate localization of lesions, assessment of treatment effectiveness, and elucidation of disease mechanisms.

[0120] Furthermore, this method achieves high-precision spatial alignment between metabolic imaging data and HE slices by using a deformation field to spatially transform the data. It not only considers pixel-level spatial transformations but also ensures anatomical consistency through visual inspection and quantitative indicators such as overlap and mutual information, significantly improving the accuracy of multimodal data registration. In the registered metabolic imaging data, spatial location information of the region of interest is extracted, including coordinates, size, shape, and relative position to specific anatomical structures, enabling spatial localization and quantitative description of the metabolic data. By combining the extracted spatial location information with the intensity information of the metabolic imaging data, in-depth analysis of the relationship between metabolic changes and anatomical structures is achieved, providing a new approach for the integration of multimodal data.

[0121] Please see Figure 4 , Figure 4 This is a schematic diagram of the registration system for spatial metabolic composition images and HE staining images in an embodiment of the present invention. The system includes an input device, a processor, an output device, and a memory, which are interconnected. The memory stores a computer program, which includes program instructions. The processor is configured to call the program instructions. The system uses the aforementioned registration method for spatial metabolic composition images and HE staining images, and its operation flow is as follows: Figure 5 As shown.

[0122] In this embodiment, the input devices include a digital pathology slide scanner and a spatial metabolomics sequencer, which are used to convert the physical information of tissue samples into digital images and data, providing a high-quality input source for subsequent image processing and registration.

[0123] Specifically, the digital pathology slide scanner is used to scan frozen slides to obtain high-resolution spatial structure images and HE-stained images; the spatial metabolomics sequencer is used to collect metabolomics data from the slides and generate spatial metabolomics imaging maps.

[0124] The processor includes a high-performance computing unit and an image processing algorithm module, which are used to process and analyze the images and data provided by the input device to complete the registration of multimodal images and the accurate mapping of spatial information.

[0125] Specifically, the high-performance computing unit is responsible for performing complex mathematical operations and algorithm calculations, such as image key point extraction, feature matching, rigid registration, non-rigid registration, and deformation field optimization; the image processing algorithm module integrates target matching algorithm model and deformation field optimization algorithm model to realize the automation and optimization of image registration.

[0126] The output device includes a high-resolution display and a data export interface, used to present the registration results in an intuitive or digital form, providing users with actionable results.

[0127] Specifically, the high-resolution display is used to visualize the registration results, showing the registration effect between the spatial metabolic composition image and the HE staining image; the data export interface supports exporting the registration results in digital form, which is convenient for further analysis or integration with other systems.

[0128] The memory includes a cache and a large-capacity storage device to ensure efficient data access and long-term preservation, supporting rapid system operation and retrospective analysis of historical data.

[0129] Specifically, the cache is used to temporarily store intermediate data during the processing, such as image key points, matching results, and deformation fields; the large-capacity storage device is used to store the original image data, registration results, and algorithm parameters for a long period of time.

[0130] In summary, this invention achieves high-precision registration of HE-stained images and spatial metabolomics images by employing image preprocessing techniques on adjacent frozen sections. This overcomes the image information mismatch problem caused by slice intervals in traditional methods, significantly improving the accuracy and reliability of image registration. By introducing key point feature extraction and matching techniques, it effectively solves the difficulty of feature point matching caused by tissue sample heterogeneity, achieving efficient registration of complex tissue samples and overcoming the limitations of existing technologies in feature matching. By combining rigid and non-rigid registration strategies, it preserves the overall structural consistency of the image while finely adjusting local deformations, overcoming the shortcomings of single registration methods in handling complex deformations and achieving high-precision correction of tissue sample deformations. By constructing and applying a deformation field, it accurately obtains the spatial location information of spatial metabolic imaging, overcoming the ambiguity of spatial positioning in traditional methods, meeting the needs of high-throughput, high-precision spatial metabolomics analysis, providing support for large-scale data processing, and promoting technological progress in this field.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for registering a spatial metabolic composition image with an HE staining image, characterized in that, The method includes the following steps: Obtain two adjacent frozen sections of the same tissue sample; Using the two adjacent frozen sections, image preprocessing was performed to obtain the processing results. The image preprocessing included HE staining and spatial metabolomics sequencing. The process of preprocessing the image using the two adjacent frozen sections and obtaining the processing result includes: Using digital pathological slide scanning technology, the spatial structure of the first slide is scanned to obtain a spatial structure image; Based on the spatial structure image, the second slice was stained with HE and the staining result was obtained. Based on the staining results, the spatial structure of the second slice is scanned using the digital pathological slide scanning technology to obtain a reference image; Based on the reference image, spatial metabolomics sequencing is performed on the first slice to obtain a spatial metabolic imaging image; The spatial structure image and the spatial metabolic imaging image are registered to obtain the specific spatial location of the spatial metabolic imaging data; Based on the processing results, the extraction and matching results of key point features in the image are obtained; Based on the extraction and matching results, rigid image registration is performed to obtain the rigid registration result; Based on the rigid registration result, perform non-rigid registration of the image and obtain the non-rigid registration result; The spatial location information of the spatial metabolic imaging data is obtained through the non-rigid registration results.

2. The registration method for spatial metabolic composition images and HE staining images according to claim 1, characterized in that, The extraction and matching results of key point features in the image based on the processing results include: Based on the spatial structure image of the first slice and the reference image of the second slice, key points and descriptors are extracted to obtain the extraction results. The key points include corner points, edges and obvious texture areas in the image, and the descriptors include unique image features around the key points. Based on the extraction results, the similarity between the spatial structure image and the reference image is compared, and the key points are filtered and matched to obtain the matching results.

3. The registration method for spatial metabolic composition images and HE staining images according to claim 2, characterized in that, The step of comparing the spatial structure image and the reference image based on the extraction results, filtering and matching the key points, and obtaining matching results includes: Based on the extraction results, a target matching algorithm model is established; The target matching algorithm model is used to filter and match key points of the spatial structure image and the reference image to obtain matching results.

4. The registration method for spatial metabolic composition images and HE staining images according to claim 3, characterized in that, The target matching algorithm model satisfies the following expression: in, The transformation matrix is... For the first slice of spatial structure image The coordinates of the matching points For the second slice reference image The coordinates of the matching points This represents the number of matching points.

5. The registration method for spatial metabolic composition images and HE staining images according to claim 1, characterized in that, The step of performing rigid image registration based on the extraction and matching results and obtaining rigid registration results includes: Based on the extraction and matching results, the position of each pixel in the first slice spatial structure image is uniformly rotated and translated to obtain the rotation and translation result; Based on the rotation and translation results, the first slice spatial structure image and the second slice reference image are rigidly registered to obtain the rigid registration result.

6. The registration method for spatial metabolic composition images and HE staining images according to claim 1, characterized in that, The step of performing non-rigid image registration based on the rigid registration result and obtaining the non-rigid registration result includes: Based on the rigid registration results, the deformation field of the first slice spatial structure image is constructed; Using the aforementioned deformation field, an optimization algorithm model for the target deformation field is established; Based on the target deformation field optimization algorithm model, the first slice spatial structure image is deformed and adjusted to determine the final deformation shape; Using the final deformed shape, the first slice spatial structure image and the second slice reference image are non-rigidly registered to obtain a non-rigid registration result.

7. The registration method for spatial metabolic composition images and HE staining images according to claim 6, characterized in that, The target deformation field optimization algorithm model satisfies the following expression: in, The deformation field of the spatial structure image of the first slice. For similarity measurement, For regularization terms, This represents the reference image for the second slice. This is a deformation image of the spatial structure of the first slice. These are the weighting coefficients. For the image at points along Displacement in the direction of For the image at points along The amount of displacement in the direction.

8. The registration method for spatial metabolic composition images and HE staining images according to claim 1, characterized in that, The spatial location information of the spatial metabolic imaging data obtained through the non-rigid registration results includes: The final deformed shape of the spatial structure image of the first slice is obtained through the non-rigid registration result. The deformation field corresponding to the final deformed shape is applied to the spatial metabolic imaging data to obtain the registration result of the spatial metabolic imaging image of the first slice and the reference image of the second slice. The spatial location information of the spatial metabolic imaging data is obtained through the registration results.

9. A registration system for spatial metabolic composition images and HE staining images, said system using the registration method for spatial metabolic composition images and HE staining images according to any one of claims 1 to 8, characterized in that, The system includes an input device, a processor, an output device, and a memory, which are interconnected. The memory stores a computer program, which includes program instructions, and the processor is configured to invoke the program instructions.

Images