Image registration method, device and equipment based on spatial omics and in situ sequencing

Abstract

This application discloses an image registration method, apparatus, device, and product based on spatial omics and in situ sequencing. The method acquires bright-field histological images, fluorescence sequencing images, and spatial genomic metadata of a target specimen. It calculates a global translation parameter between the bright-field histological image and the fluorescence sequencing image, performs affine optimization based on the global translation parameter, the bright-field histological image, and the fluorescence sequencing image to obtain a first affine matrix between them. A first convergence index is calculated based on the first affine matrix, the bright-field histological image, and the fluorescence sequencing image. If the first convergence index is greater than or equal to a convergence threshold, the first affine matrix is ​​used as the final affine matrix; if the first convergence index is less than the convergence threshold, the final affine matrix is ​​calculated based on the local correlation between the bright-field histological image and the fluorescence sequencing image. The final affine matrix is ​​then used to register the spatial genomic metadata. This application aims to improve the efficiency of image registration.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to an image registration method, apparatus, and device based on spatial omics and in situ sequencing. Background Technology

[0002] As life sciences shift from single-cell analysis to spatial multi-omics, establishing a precise physical mapping between morphological and histological images and high-throughput genome sequencing coordinates has become a critical engineering bottleneck. The rapid development of spatial multi-omics technologies has made it possible to acquire whole-genome or whole-transcriptome data while preserving tissue spatial structure information; however, the integration and alignment of data from different modalities, platforms, and resolutions are increasingly prominent. Specifically, high-resolution morphological images obtained after staining and imaging tissue sections and subsequent sequencing data often originate from the same tissue. However, due to factors such as deformation during slide preparation, differences in section thickness, tissue folding, and shrinkage and expansion caused by chemical fixation, there are significant nonlinear differences between the two in the physical coordinate system, severely affecting the accuracy of downstream cell type annotation, spatial expression pattern recognition, and multimodal data fusion analysis.

[0003] Currently, registration is usually achieved by executing a single algorithm path, that is, a fixed set of processing steps are predetermined in the registration process. This means that registration depends on predefined registration steps, such as configuring a specific transformation model after a specific feature detector to achieve image registration. Since there is a strong coupling relationship between the predefined feature detector and the transformation model, once the selection of a certain step is not suitable for the characteristics of the current data, the entire registration method is difficult to adjust. Therefore, this registration method is not flexible enough and the efficiency of image registration is low. Summary of the Invention

[0004] The present invention aims to improve the efficiency of image registration.

[0005] According to the first aspect, one embodiment provides an image registration method based on spatial omics and in situ sequencing, comprising:

[0006] Acquire bright-field histological images, fluorescence sequencing images, and spatial genomic metadata of the target specimen;

[0007] Calculate the global translation parameter between the bright-field histological image and the fluorescence sequencing image, and perform affine optimization based on the global translation parameter, the bright-field histological image, and the fluorescence sequencing image to obtain a first affine matrix between the bright-field histological image and the fluorescence sequencing image; wherein, the global translation parameter is used to characterize the displacement relationship between the bright-field histological image and the fluorescence sequencing image;

[0008] A first convergence index is calculated based on the first affine matrix, the bright-field histological image, and the fluorescence sequencing image; wherein, the first convergence index is used to characterize the correlation between the fluorescence sequencing image transformed by the first affine matrix and the bright-field histological image;

[0009] If the first convergence index is greater than or equal to the preset convergence threshold, the first affine matrix is ​​used as the final affine matrix; if the first convergence index is less than the convergence threshold, the final affine matrix is ​​calculated based on the local correlation between the bright-field histology image and the fluorescence sequencing image.

[0010] The spatial genomic metadata is registered using the final affine matrix to map the spatial genomic metadata onto the bright-field histological image.

[0011] In one embodiment, calculating the final affine matrix based on the local correlation between the bright-field histological image and the fluorescence sequencing image includes:

[0012] The final affine matrix is ​​calculated using a feature-matching-based affine matrix calculation method and / or a phase-correlation-based affine matrix calculation method.

[0013] The final affine matrix is ​​calculated using a feature-matching-based affine matrix calculation method, including:

[0014] Multiple key feature points of the fluorescence sequencing image are obtained, and a preset clustering algorithm is used to identify the target feature point clusters corresponding to the multiple key feature points;

[0015] The homography matrix corresponding to the target feature point cluster is calculated based on a preset parameter estimation method, and the homography matrix is ​​used as the final affine matrix.

[0016] The final affine matrix is ​​calculated using a phase-dependent affine matrix calculation method, including:

[0017] The phase correlation coefficients of the bright-field histology image and the fluorescence sequencing image are obtained, and the final affine matrix is ​​constructed based on the phase correlation coefficients.

[0018] In one embodiment, acquiring multiple key feature points of the fluorescence sequencing image includes:

[0019] The fluorescence sequencing image is used to extract features using different types of feature detectors, and the feature points output by each feature detector are obtained.

[0020] The feature points output by each feature detector are traversed to construct a union, and the feature points in the union are used as multiple key feature points.

[0021] In one embodiment, the phase correlation coefficients of the bright-field histological image and the fluorescence sequencing image are obtained, and a final affine matrix is ​​constructed based on the phase correlation coefficients, including:

[0022] The bright-field histological image and the fluorescence sequencing image are subjected to a first downsampling process to obtain a first downsampled bright-field histological image and a first downsampled fluorescence sequencing image, respectively. The bright-field histological image and the fluorescence sequencing image are then subjected to a second downsampling process to obtain a second downsampled bright-field histological image and a second downsampled fluorescence sequencing image, respectively. The first downsampling process and the second downsampling process use a first resolution ratio and a second resolution ratio, respectively, and the first resolution ratio and the second resolution ratio are different.

[0023] A first phase correlation coefficient is calculated based on the first downsampled bright-field histological image and the first downsampled fluorescence sequencing image, and a second phase correlation coefficient is calculated based on the second downsampled bright-field histological image and the second downsampled fluorescence sequencing image.

[0024] The final affine matrix is ​​constructed based on the first phase correlation coefficient and the second phase correlation coefficient.

[0025] In one embodiment, registering the spatial genome metadata using the final affine matrix includes:

[0026] Dense corresponding blocks are extracted from the bright-field histological image and the fluorescence sequencing image, and the initial deformation field is calculated based on the dense corresponding blocks;

[0027] The final deformation field is calculated based on the initial deformation field and the final affine matrix. The final deformation field is then inverted and mapped onto the spatial genome metadata.

[0028] In one embodiment, the step of performing affine optimization based on the global translation parameters, the bright-field histological image, and the fluorescence sequencing image to obtain a first affine matrix between the bright-field histological image and the fluorescence sequencing image includes:

[0029] The fluorescence sequencing image is shifted using the global translation parameters to obtain a shifted fluorescence sequencing image.

[0030] Based on a preset image registration algorithm, the first affine matrix between the bright-field histology image and the translated fluorescence sequencing image is solved.

[0031] In one embodiment, acquiring a fluorescence sequencing image of a target specimen includes:

[0032] Obtain the initial sequencing image of the target specimen and calculate the skewness coefficient of the initial sequencing image;

[0033] When the skewness coefficient is less than the preset skewness threshold, the initial sequencing image is phase-reversed to obtain the reversed initial sequencing image;

[0034] The reversed initial sequencing image is subjected to image enhancement processing to obtain the fluorescence sequencing image of the target specimen.

[0035] According to a second aspect, one embodiment provides an image registration device based on spatial omics and in situ sequencing, comprising:

[0036] The data acquisition module is used to acquire bright-field histological images, fluorescence sequencing images, and spatial genomic metadata of the target specimen;

[0037] A matrix calculation module is used to calculate the global translation parameter between the bright-field histological image and the fluorescence sequencing image, and to perform affine optimization based on the global translation parameter, the bright-field histological image, and the fluorescence sequencing image to obtain a first affine matrix between the bright-field histological image and the fluorescence sequencing image; wherein, the global translation parameter is used to characterize the displacement relationship between the bright-field histological image and the fluorescence sequencing image;

[0038] The index calculation module is used to calculate a first convergence index based on the first affine matrix, the bright-field histology image, and the fluorescence sequencing image; wherein, the first convergence index is used to characterize the correlation between the fluorescence sequencing image after transformation using the first affine matrix and the bright-field histology image;

[0039] The matrix determination module is used to take the first affine matrix as the final affine matrix if the first convergence index is greater than or equal to the preset convergence threshold, and to calculate the final affine matrix based on the local correlation between the bright field histology image and the fluorescence sequencing image if the first convergence index is less than the convergence threshold.

[0040] An image registration module is used to register the spatial genomic metadata using the final affine matrix, so as to map the spatial genomic metadata onto the bright-field histological image.

[0041] According to a third aspect, one embodiment provides an image registration device based on spatial omics and in situ sequencing, comprising:

[0042] Memory, used to store programs;

[0043] A processor for implementing the image registration method by executing a program stored in the memory.

[0044] According to a fourth aspect, one embodiment provides a computer program product including a computer program and / or instructions that, when executed by a processor, implement the image registration method described above.

[0045] According to the above embodiments, the image registration method, device, equipment, and computer program product based on space omics and in situ sequencing calculate the global translation parameter between the bright-field histology image and the fluorescence sequencing image. Affine optimization is performed based on the global translation parameter, the bright-field histology image, and the fluorescence sequencing image to obtain a first affine matrix between the two images. Since the global translation parameter characterizes the displacement relationship between the bright-field histology image and the fluorescence sequencing image, and the first affine matrix is ​​obtained based on the global translation parameter, the fluorescence sequencing image transformed using the first affine matrix is ​​transformed from a global perspective. A first convergence index is calculated based on the first affine matrix, the bright-field histology image, and the fluorescence sequencing image. The first convergence index measures the correlation between the fluorescence sequencing image transformed using the first affine matrix and the bright-field histology image. If the first convergence index is greater than or equal to a preset convergence threshold, the first affine matrix is ​​used as the final affine matrix. If the first convergence criterion is less than the convergence threshold, the final affine matrix is ​​calculated based on the local correlation between the bright-field histology image and the fluorescence sequencing image. Calculating the final affine matrix from the perspective of local correlation provides an alternative scheme to global transformations, preventing the need to restart the calculation when the first convergence criterion is less than the convergence threshold, which would affect the efficiency of image registration. The final affine matrix is ​​then used to register the spatial genomic metadata, mapping it onto the bright-field histology image, thus achieving registration between the spatial genomic metadata and the bright-field histology image. Attached Figure Description

[0046] Figure 1 This is a flowchart of the image registration method based on spatial omics and in situ sequencing in the embodiments of this application;

[0047] Figure 2 This is a flowchart illustrating the acquisition of a fluorescence sequencing image of a target specimen in one embodiment;

[0048] Figure 3 This is a flowchart illustrating an embodiment of affine optimization based on global translation parameters, bright-field histology images, and fluorescence sequencing images to obtain a first affine matrix between bright-field histology images and fluorescence sequencing images.

[0049] Figure 4 This is a flowchart illustrating the calculation of the final affine matrix using a feature-matching-based affine matrix calculation method in one embodiment.

[0050] Figure 5 This is a flowchart illustrating the acquisition of multiple key feature points in a fluorescence sequencing image in one embodiment;

[0051] Figure 6 This is a flowchart illustrating how to obtain the phase correlation coefficients of bright-field histology images and fluorescence sequencing images in one embodiment, and how to construct the final affine matrix based on the phase correlation coefficients.

[0052] Figure 7 This is a flowchart illustrating the registration of spatial genome metadata using the final affine matrix in one embodiment;

[0053] Figure 8 This is a reference diagram showing the evolution of energy terrain at different scales in one embodiment;

[0054] Figure 9 This is a schematic diagram of an image registration device based on spatial omics and in situ sequencing in one embodiment. Detailed Implementation

[0055] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the invention. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to the present invention are not shown or described in the specification. This is to avoid obscuring the core parts of the invention with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0056] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0057] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this invention, unless otherwise specified, include both direct and indirect connections (linkages).

[0058] Spatial omics is a technique that resolves high-throughput molecular expression profiles (such as transcriptomics and proteomics) while preserving the inherent spatial coordinates of tissue sections. Examples include 10x Visium spatial transcriptome sequencing, spatiotemporal omics techniques such as Stereo-seq, and Multiplexed Error-Robust Fluorescence in Situ Hybridization (MERFISH). Its process involves complex physical and chemical steps: tissue embedding, frozen sectioning, chemical permeation, hybridization, enzymatic reactions, and cyclic fluorescence imaging. These processes inevitably produce macroscale displacements (such as translation, rotation, and scaling). More seriously, local "tearing," folding, and anisotropic shrinkage caused by chemical treatment and thermal cycling increase the difficulty of establishing a physical mapping between morphological histological images and high-throughput genome sequencing coordinates. The following schemes are commonly used in related technologies to achieve image registration: (1) Traditional medical image registration (such as Elastix and ANTs): This type of method uses mutual information or mean square error as indicators to measure macroscopic images for CT / MRI. However, in spatial omics, it often fails due to problems such as tissue texture repetition and mode inversion (bright field and fluorescence), and the optimizer is prone to getting trapped in local minima. (2) Deep learning-based registration (such as VoxeIMorph): Deformation field is learned unsupervised by CNN. This type of method requires a large amount of training data and has insufficient generalization ability for chemically induced topological fractures (tears or damage), and often fails to achieve the sub-pixel accuracy required for single cell attribution. (3) Pure feature point method (such as OpenSlide SIFT): It only relies on key point detection. In spatial omics, sequencing artifacts (dust, non-specific fluorescence, etc.) will generate a large number of "pseudo" feature points, causing the failure of traditional geometric consistency verification based on Random Sample Consensus (RANSAC).

[0059] In summary, the relevant technologies have the following defects: (1) Uncontrollable nonlinear anisotropic tearing: Tissue transparency and thermal cycling will trigger uneven deformation. The standard linear affine model cannot characterize these local elastic fractures, resulting in significant errors in the assignment of transcripts to cells. (2) Convergence failure caused by mode inversion: Bright field absorption and fluorescence emission are often inversely correlated in terms of intensity gradient. Standard metrics will be trapped by this "peak-valley" mismatch, causing the optimizer to stagnate or diverge. (3) Lack of hierarchical backoff mechanism: Most processes only execute a single path algorithm, rely on a predefined sequence of steps, and lack adaptive strategies for different image features or complex conditions, resulting in insufficient robustness and stability. In cases of extreme crowding or partial tissue degradation, failure of the first step will cause the entire pipeline to collapse, and there is a lack of automated recovery protocols.

[0060] To address the aforementioned issues, this application proposes a deterministic, robust, and stable image registration method based on spatial omics and in situ sequencing. In this method, bright-field histological images, fluorescence sequencing images, and spatial genomic metadata of the target specimen are acquired. A global translation parameter between the bright-field histological image and the fluorescence sequencing image is calculated. Affine optimization is performed based on the global translation parameter, the bright-field histological image, and the fluorescence sequencing image to obtain a first affine matrix between the two images. The global translation parameter characterizes the displacement relationship between the bright-field histological image and the fluorescence sequencing image. A first convergence index is calculated based on a first affine matrix, a bright-field histological image, and a fluorescence sequencing image. The first convergence index is used to characterize the correlation between the fluorescence sequencing image and the bright-field histological image after transformation using the first affine matrix. If the first convergence index is greater than or equal to a preset convergence threshold, the first affine matrix is ​​used as the final affine matrix. If the first convergence index is less than the convergence threshold, the final affine matrix is ​​calculated based on the local correlation between the bright-field histological image and the fluorescence sequencing image. The final affine matrix is ​​used to register the spatial genomic metadata to map the spatial genomic metadata onto the bright-field histological image.

[0061] The image registration method based on spatial omics and in situ sequencing provided in this application is described below with reference to the accompanying drawings.

[0062] Figure 1 A flowchart of an image registration method based on spatial omics and in situ sequencing provided in an embodiment of this application is shown below, which will be described in detail below.

[0063] Step S10: Obtain bright-field histological images, fluorescence sequencing images, and spatial genomic metadata of the target specimen.

[0064] In some embodiments, bright-field histological images, fluorescence sequencing images, and spatial genomic metadata are complementary datasets of the same target specimen in different aspects. Bright-field histological images are tissue sections captured by optical microscopy and stained with H&E or other dyes, captured before sequencing begins to provide morphological background and cellular structure information of the target specimen. Fluorescence sequencing images are multi-cycle fluorescence microscopy images captured during in situ sequencing, with each imaging cycle capturing fluorescence signals from specific nucleotide incorporations. These fluorescence signals represent the spatial distribution of genetic information, constructing spatial genetic information layer by layer. Spatial genomic metadata contains the genetic sequence information of the target specimen and a FastQ / BAM file of a spatial barcode matrix, linking each detected transcript to physical coordinates (X, Y) on the sequencing chip.

[0065] Step S20: Calculate the global translation parameter between the bright-field histology image and the fluorescence sequencing image, and perform affine optimization based on the global translation parameter, the bright-field histology image, and the fluorescence sequencing image to obtain the first affine matrix between the bright-field histology image and the fluorescence sequencing image.

[0066] In some embodiments, a global translation parameter between a bright-field histological image and a fluorescence sequencing image is calculated using phase correlation technology. This global translation parameter characterizes the displacement relationship between the two images. Phase correlation is a frequency domain technique that is highly robust to noise and can effectively determine large-scale translations between images without explicit feature detection. The purpose of calculating the global translation parameter is to provide a preliminary coarse alignment between the bright-field histological image and the fluorescence sequencing image. This macroscopic translation is crucial for achieving sufficient image overlap, facilitating the effective operation of subsequent image registration methods.

[0067] In some embodiments, the Enhanced Correlation Coefficient (ECC) image registration algorithm can be used to perform affine optimization on bright-field histology images and fluorescence sequencing images to obtain a first affine matrix between the two images. Affine optimization refers to iteratively adjusting and optimizing the parameters of the affine transformation (including rotation, scaling, shearing, and translation components) to achieve more accurate alignment. The Enhanced Correlation Coefficient algorithm is a gradient-based subpixel image registration algorithm that solves for the affine or homography transformation matrix by maximizing the correlation coefficient between images. Essentially, it actively solves for the affine transformation parameters by maximizing the correlation to find the optimal global rigid alignment. The global translation parameters can be used as initialization parameters in the affine optimization process to obtain the first affine matrix.

[0068] Step S30: Calculate the first convergence metric based on the first affine matrix, bright-field histology image, and fluorescence sequencing image.

[0069] In some embodiments, the first convergence index is used to characterize the correlation between the fluorescence sequencing image and the bright-field histology image after transformation by the first affine matrix. The fluorescence sequencing image can be transformed by the first affine matrix first, and then the correlation coefficient between the transformed fluorescence sequencing image and the bright-field histology image can be calculated to obtain the first convergence index.

[0070] For example, the first convergence index ,in, Represents bright-field histological images. Represents a fluorescent sequencing image. Let represent the first affine matrix. This represents matrix multiplication.

[0071] Step S40: Determine whether the first convergence index is greater than or equal to the preset convergence threshold.

[0072] For example, the preset convergence threshold is 0.85.

[0073] If the first convergence index is greater than or equal to the preset convergence threshold, proceed to step S50: use the first affine matrix as the final affine matrix; if the first convergence index is less than the convergence threshold, proceed to step S60: calculate the final affine matrix based on the local correlation between the bright-field histology image and the fluorescence sequencing image.

[0074] In some embodiments, if the first convergence index is greater than or equal to the preset convergence threshold, it means that the currently calculated first affine matrix can achieve good subsequent image registration. In this case, the first affine matrix can be directly used as the final affine matrix for subsequent image registration. If the first convergence index is less than the convergence threshold, the affine matrix needs to be recalculated. Considering that image registration cannot be achieved well using global translation parameters, the final affine matrix is ​​calculated based on the local correlation between bright-field histology images and fluorescence sequencing images to ensure accurate subsequent image registration.

[0075] Step S70: Register the spatial genomic metadata using the final affine matrix to map the spatial genomic metadata onto the bright-field histological image.

[0076] In some embodiments, registering spatial genomic metadata using a final affine matrix essentially involves translating or mapping the physical coordinates (X,Y) of the spatial genomic metadata to the histological location on a bright-field histological image using the final affine matrix.

[0077] According to the image registration method based on spatial omics and in situ sequencing in the above embodiments, a global translation parameter between a bright-field histological image and a fluorescence sequencing image is calculated. Affine optimization is performed based on the global translation parameter, the bright-field histological image, and the fluorescence sequencing image to obtain a first affine matrix between the bright-field histological image and the fluorescence sequencing image. Since the global translation parameter characterizes the displacement relationship between the bright-field histological image and the fluorescence sequencing image, the first affine matrix is ​​obtained based on the global translation parameter, so that the fluorescence sequencing image transformed using the first affine matrix is ​​transformed from a global perspective. A first convergence index is calculated based on the first affine matrix, the bright-field histological image, and the fluorescence sequencing image. The first convergence index is used to measure the correlation between the fluorescence sequencing image transformed using the first affine matrix and the bright-field histological image. If the first convergence index is greater than or equal to a preset convergence threshold, the first affine matrix is ​​used as the final affine matrix. If the first convergence criterion is less than the convergence threshold, the final affine matrix is ​​calculated based on the local correlation between the bright-field histology image and the fluorescence sequencing image. Calculating the final affine matrix from the perspective of local correlation provides an alternative scheme to global transformations, preventing the need to restart the calculation when the first convergence criterion is less than the convergence threshold, which would affect the efficiency of image registration. The final affine matrix is ​​then used to register the spatial genomic metadata, mapping it onto the bright-field histology image, thus achieving registration between the spatial genomic metadata and the bright-field histology image.

[0078] Please refer to Figure 2 In some embodiments, obtaining fluorescence sequencing images of the target specimen includes steps S11 to S14, which are described in detail below.

[0079] Step S11: Obtain the initial sequencing image of the target sample and calculate the skewness coefficient of the initial sequencing image.

[0080] In some embodiments, the skewness coefficient of the initial sequencing image is calculated using the following formula:

[0081]

[0082] in, This represents the skewness coefficient of the initial sequencing image. Represents the expectation operator, A random variable representing the pixel intensity in the initial sequencing image. This represents the mean pixel intensity, which is a first-order matrix about the origin. The standard deviation of pixel intensity.

[0083] In some embodiments, the skewness coefficient is the histogram skewness coefficient.

[0084] Step S12: Determine whether the skewness coefficient is less than the preset skewness threshold.

[0085] When the skewness coefficient is less than the preset skewness threshold, step S13 is executed: the initial sequencing image is phase-reversed to obtain the reversed initial sequencing image.

[0086] In some embodiments, when the skewness coefficient is less than a preset skewness threshold, it indicates that the modal distribution of the initial sequencing image has been reversed. This usually means that there is a darker signal on a brighter background, or vice versa. In this case, the initial sequencing image needs to be phase-reversed to obtain the reversed initial sequencing image.

[0087] In some embodiments, phase reversal can be expressed as I inv (x,y)=max(I seq )-I seq (x,y), where I seq (x,y) represents the pixel intensity value of the initial sequencing image at a specific pixel location, max(I seq ) represents the maximum possible pixel intensity value of the initial sequencing image within the dynamic range. For an 8-bit initial sequencing image, max(I seq The value is 255. More generally, it can be the actual maximum pixel value present in the initial sequencing image or the maximum intensity level predefined by the image format.

[0088] In some embodiments, the purpose of phase inversion is to invert the intensity scale; if a pixel is dark (low intensity), it will become brighter (high intensity) after inversion, and vice versa. This ensures that the gradient direction of the inverted initial sequencing image is consistent with the gradient direction of the bright-field histology image, which is crucial for subsequent registration steps.

[0089] Step S14: Perform image enhancement processing on the reversed initial sequencing image to obtain the fluorescence sequencing image of the target specimen.

[0090] In some embodiments, image enhancement processing of the reversed initial sequencing image can be performed by applying contrast-limited adaptive histogram equalization to normalize the feature scale, thereby obtaining a fluorescence sequencing image of the target specimen.

[0091] In some embodiments, considering that bright-field histology images and fluorescence sequencing images capture different physical phenomena from the same tissue specimen, bright-field histology images capture tissue morphology, such as cell boundaries, nuclei, and tissue structures. Bright-field histology images measure the light absorption of stained tissue components, with darker areas typically representing higher staining concentrations (e.g., hematoxylin uptake by the nucleus). In contrast, fluorescence sequencing images label the light emission of molecular probes, capturing molecular activities within the same morphology, such as gene expression and protein expression; their brighter areas represent higher molecular signal intensities. This creates an inherent inverse relationship between the two modalities; that is, areas appearing darker (highly stained) in bright-field histology images typically correspond to brighter, highly bioactive areas in fluorescence sequencing images. To reconcile the bright-field and fluorescence modalities, modality inversion is achieved by calculating a skewness coefficient, ensuring that the two modalities share a consistent gradient direction in subsequent optimization.

[0092] Please refer to Figure 3 In some embodiments, step S20: performing affine optimization based on global translation parameters, bright-field histology image and fluorescence sequencing image to obtain a first affine matrix between bright-field histology image and fluorescence sequencing image, including steps S21 to S22, which are described in detail below.

[0093] Step S21: Use global translation parameters to translate the fluorescence sequencing image to obtain the translated fluorescence sequencing image.

[0094] In some embodiments, translating the fluorescence sequencing image using global translation parameters is essentially initializing the image using these parameters. For subsequent image registration algorithms, a good starting point (initialization) helps avoid local minima and significantly speeds up convergence. Without a good initial translation, affine optimization may struggle to find the correct global transformation, especially for images with large displacements.

[0095] Step S22: Based on the preset image registration algorithm, the bright-field histology image and the translated fluorescence sequencing image, solve for the first affine matrix between the bright-field histology image and the fluorescence sequencing image.

[0096] In some embodiments, during the process of solving the first affine matrix between bright-field histology images and fluorescence sequencing images, the parameters of the affine transformation (such as rotation, scaling, shearing, and translation components) are iteratively adjusted and optimized to achieve more accurate alignment. The image registration algorithm can employ an enhanced correlation coefficient algorithm, which solves the affine or homography transformation matrix by maximizing the correlation coefficient between images. This algorithm is robust to photometric distortion. The above algorithm does not merely refine a solved affine matrix, but actively solves or optimizes the affine transformation parameters by maximizing the correlation. These affine transformation parameters are encapsulated in the affine matrix.

[0097] In some embodiments, the fluorescence sequencing image is translated using a global translation parameter, and then an affine optimization process is driven by an image registration algorithm. Starting from the state after translation by the global translation parameter, the affine matrix is ​​iteratively adjusted to find the optimal global rigid alignment by maximizing the correlation between images.

[0098] In some embodiments, the final affine matrix is ​​calculated based on the local correlation between bright-field histology images and fluorescence sequencing images, including:

[0099] The final affine matrix is ​​calculated using a feature-matching-based affine matrix calculation method and / or a phase-correlation-based affine matrix calculation method.

[0100] In some embodiments, the final affine matrix can be calculated using either a feature-matching-based affine matrix calculation method or a phase-correlation-based affine matrix calculation method. Alternatively, a second affine matrix can be calculated first using a feature-matching-based affine matrix calculation method (or a phase-correlation-based affine matrix calculation method). A second convergence index is then calculated based on the second affine matrix, bright-field histology image, and fluorescence sequencing image. If the second convergence index is greater than or equal to a preset convergence threshold, the second affine matrix is ​​used as the final affine matrix. If the second convergence index is less than the preset convergence threshold, a third affine matrix is ​​calculated using a phase-correlation-based affine matrix calculation method (or a phase-correlation-based affine matrix calculation method). A third convergence index is then calculated based on the third affine matrix, bright-field histology image, and fluorescence sequencing image. If the third convergence index is greater than or equal to a preset convergence threshold, the third affine matrix is ​​used as the final affine matrix. The affine matrix calculation methods used for the second and third affine matrices differ.

[0101] In some embodiments, when the first convergence metric is less than a preset convergence threshold, the final affine matrix is ​​calculated based on the local correlation between the bright-field histology image and the fluorescence sequencing image, thus shifting from a global, robust but less accurate method to a more local, more accurate, and computationally intensive method. If the first convergence metric fails to reach a predefined quality threshold, the system automatically switches to a more robust or refined method.

[0102] Please refer to Figure 4 In some embodiments, the final affine matrix is ​​calculated using a feature-matching-based affine matrix calculation method, including steps S23 to S24, which are described in detail below.

[0103] Step S23: Obtain multiple key feature points from the fluorescence sequencing image, and use a preset clustering algorithm to identify the target feature point clusters corresponding to the multiple key feature points.

[0104] In some embodiments, a density-based spatial clustering of applications with noise (DBSCAN) algorithm is used to identify target feature clusters corresponding to multiple key feature points. These target feature clusters typically correspond to genuine biological anchor clusters and can effectively identify and isolate "noise points" (i.e., outliers). In this embodiment, the aforementioned noise points typically refer to sequencing artifacts, nonspecific fluorescence, or other spurious features. By filtering out these noise points and focusing on the target feature clusters, a refined and robust set of feature points can be obtained for subsequent registration. The density-based spatial clustering of applications with noise algorithm is used to screen key points with high spatial topological consistency, thereby distinguishing genuine biological anchor clusters from sequencing "salt grain" noise or nonspecific fluorescence artifacts.

[0105] Step S24: Calculate the homography matrix corresponding to the target feature point cluster based on the preset parameter estimation method, and use the homography matrix as the final affine matrix.

[0106] In some embodiments, the homography matrix corresponding to the target feature point cluster is calculated by a homography estimation method based on Random Sample Consensus (RANSAC), and the homography matrix is ​​used as the final affine matrix.

[0107] Please refer to Figure 5 In some embodiments, step S23: acquiring multiple key feature points of the fluorescence sequencing image, including steps S231 to S232, will be described in detail below.

[0108] Step S231: Use different types of feature detectors to extract features from the fluorescence sequencing image and obtain the feature points output by each feature detector.

[0109] In some embodiments, different types of feature detectors include, but are not limited to, Scale Invariant Feature Transform (SIFT) detectors, Speeded Up Robust Features (SURF) detectors, Oriented FAST and Rotated BRIEF (ORB) detectors, Accelerated-KAZE (AKAZE) detectors, and deep learning-based geometric point detectors, wherein SuperPoint, a deep learning-based end-to-end visual feature detection and description method, or DISK (Dense Interest Points with Keypoint Adaptation) feature extractor is employed.

[0110] Step S232: Traverse the feature points output by each feature detector to construct a union, and use the feature points in the union as multiple key feature points.

[0111] In some embodiments, the feature points output by each feature detector are traversed and a union set is constructed, which takes advantage of the different types of feature detectors, making the final multiple key feature points more robust and less susceptible to the limitations of any single feature detector.

[0112] In some embodiments, the final affine matrix is ​​calculated using a phase-dependent affine matrix calculation method, including:

[0113] The phase correlation coefficients of bright-field histology images and fluorescence sequencing images were obtained, and the final affine matrix was constructed based on the phase correlation coefficients.

[0114] In some embodiments, the translation parameters between bright-field histology images and fluorescence sequencing images can be calculated by frequency domain phase difference, and then the scaling factor and rotation angle can be calculated by logarithmic polar coordinate transformation. The final affine matrix can be constructed based on the translation parameters, scaling factor and rotation angle.

[0115] Please refer to Figure 6 In some embodiments, the phase correlation coefficients of bright-field histology images and fluorescence sequencing images are obtained, and the final affine matrix is ​​constructed based on the phase correlation coefficients, including steps S25 to S27, which are described in detail below.

[0116] Step S25: Perform a first downsampling process on the bright-field histology image and the fluorescence sequencing image respectively to obtain a first downsampled bright-field histology image and a first downsampled fluorescence sequencing image. Perform a second downsampling process on the bright-field histology image and the fluorescence sequencing image respectively to obtain a second downsampled bright-field histology image and a second downsampled fluorescence sequencing image.

[0117] In some embodiments, the first downsampling process and the second downsampling process respectively employ a first resolution ratio and a second resolution ratio, wherein the first resolution ratio and the second resolution ratio are different.

[0118] For example, with a first resolution ratio of 4 and a second resolution ratio of 2, a first downsampling process is performed on the bright-field histology image and the fluorescence sequencing image based on the first resolution ratio to obtain a first downsampled bright-field histology image. and the first downsampling fluorescence sequencing image Based on the second resolution ratio, the bright-field histological image and the fluorescence sequencing image were subjected to a second downsampling process to obtain the second downsampled bright-field histological image. Second downsampled fluorescence sequencing images .

[0119] Step S26: Calculate the first phase correlation coefficient based on the first downsampled bright-field histological image and the first downsampled fluorescence sequencing image, and calculate the second phase correlation coefficient based on the second downsampled bright-field histological image and the second downsampled fluorescence sequencing image.

[0120] For example, the first phase correlation coefficient T coarse =PhaseCorrelation( The second phase correlation coefficient T fine =PhaseCorrelation( ).

[0121] Step S27: Construct the final affine matrix based on the first phase correlation coefficient and the second phase correlation coefficient.

[0122] In some embodiments, the final affine matrix A = , This represents matrix multiplication.

[0123] Please refer to Figure 7 In some embodiments, step S50: registering spatial genome metadata using the final affine matrix includes steps S51 to S52, which are described in detail below.

[0124] Step S51: Extract dense corresponding blocks from bright-field histology images and fluorescence sequencing images, and calculate the initial deformation field based on the dense corresponding blocks.

[0125] In some embodiments, a global translation parameter is used to translate the fluorescence sequencing image to obtain a translated fluorescence sequencing image. Dense corresponding blocks are extracted from the bright-field histology image and the translated fluorescence sequencing image. The extracted dense corresponding blocks represent local image regions around key points or dense grid points on the image.

[0126] In some embodiments, the following methods are provided to extract dense corresponding blocks and illustrate their implicit correspondences: (1) Local correlation / normalized cross correlation (NCC): For each point in the dense grid on a fixed image, a local search (e.g., using normalized cross correlation or other similarity measures) is performed during the image movement to find the best matching block and generate a set of dense displacement vectors. (2) Feature point interpolation: If there is a set of sparse but highly reliable feature correspondences (e.g., the target feature point clusters corresponding to multiple key feature points identified by the preset clustering algorithm in step S23), these correspondences can be used as "control points". Based on the above sparse "control points", a dense set of correspondences is generated by interpolation on the entire image as the basis for subsequent thin plate spline (TPS). TPS is a non-rigid coordinate transformation method that simulates the minimization of bending energy of a physical thin plate under point load. In this embodiment, it is used to model and correct local nonlinear tissue distortion. (3) Optical flow method: directly calculates the dense displacement field between two images, identifies the movement mode of each pixel, and the resulting dense vector field provides corresponding relationship information.

[0127] In some embodiments, non-rigid elastic correction can also be achieved through B-spline free deformation or optical flow fields to address local tissue tearing and restore the biological spatial context.

[0128] In some embodiments, multiple control points of dense corresponding blocks are identified. Non-rigid elastic correction is essentially a non-rigid transformation model, aiming to smoothly interpolate the deformation field from a set of discrete control points. The initial deformation field is solved by minimizing this non-rigid transformation model. These multiple control points can be regularly spaced grid points on a fixed image, found in a moving image through local image similarity (e.g., NCCs in image blocks surrounding each grid point), or feature points in a cluster of target feature points corresponding to multiple key feature points identified using a preset clustering algorithm in step S23, or a combination of both. A linear equation system is constructed based on the multiple control points to solve for the TPS parameters defining the TPS transformation. This equation system includes bending energy minimization constraints, ensuring the smoothness of the resulting deformation field. Its core is a kernel matrix constructed based on radial basis functions and affine component matrices. For each pixel in the image, the transformed coordinates and displacement vector corresponding to that pixel are calculated using the solved TPS parameters, and a dense nonlinear deformation field, i.e., the initial deformation field, is constructed. Each vector represents how much and in what direction each pixel in the moving image needs to be moved to align with the fixed image.

[0129] Step S52: Calculate the final deformation field based on the initial deformation field and the final affine matrix, invert the final deformation field, and map the inverted final deformation field onto the spatial genome metadata.

[0130] In some embodiments, the initial deformation field is The final affine matrix is ​​A, and the final deformation field is... , This represents matrix multiplication. The final deformation field simultaneously captures global rigid body motion and local non-rigid body distortion.

[0131] In some embodiments, the final deformation field is inverted and mapped onto the spatial genomic metadata. The inverse of the final deformation field is used to map the target coordinates back to the source coordinates for sampling. Simultaneously, the final deformation field can be used to resample fluorescence sequencing images or directly transform the coordinates of the spatial genomic metadata, typically using interpolation, such as bilinear or bicubic interpolation, to ensure the transformed image maintains its quality.

[0132] In some embodiments, the final deformation field is inverted and directly applied to spatial genomic metadata, which represents the main molecular outputs of the experiment, where each detected gene transcript is associated with physical coordinates (X, Y) on the sequencing chip. Since the sequencing chip is identical to the object entity captured in the fluorescence sequencing image, the spatial genomic metadata and the fluorescence sequencing image share the same distorted coordinate system. By applying the inverted final deformation field, the physical coordinate labels are transferred to their precise locations on the bright-field histological image, effectively "anchoring" molecular data to the morphological structure of the tissue. In summary, the goal of image registration is to align modalities so that each molecular signal from the fluorescence sequencing image or spatial genomic metadata can be accurately mapped to its corresponding morphological location in the bright-field histological image, thereby enabling comprehensive structural and functional analysis.

[0133] In some embodiments, the initial deformation field implicitly represents a mapping from source coordinates to target coordinates. The final deformation field is calculated based on the initial deformation field and the final affine matrix. The inverted final deformation field is mapped onto the fluorescence sequencing image or spatial genomic metadata, correcting local chemically induced shrinkage, tearing, and other non-rigid distortions in the image. This allows for precise alignment with the bright-field histological image at the sub-pixel level, resulting in a highly accurate, non-rigidly registered sequencing image or coordinates of the transformed spatial genomic metadata that accurately reflects its histological position.

[0134] In some embodiments, for the registered bright-field histology image and fluorescence sequencing image, the images are divided into multiple grid blocks. For each grid block, the local normalized cross-correlation value between the grid block in the registered bright-field histology image and the grid block at the corresponding position in the registered fluorescence sequencing image is calculated. Any grid block with a local normalized cross-correlation value lower than a preset convergence threshold is marked as "unconverged", thereby triggering a brute-force search. A sliding window normalized cross-correlation search is performed within the constrained search radius to ensure zero blind zone coverage and resolve local residual alignment deviations.

[0135] In some embodiments, compared with industry standard tools such as Elastix, the image registration method based on spatial omics and in situ sequencing of this application has the following advantages: (1) Extremely robust: Multiple affine matrix calculation methods can be adapted to multiple tissue types, and can still be successfully aligned on samples where traditional algorithms fail. At the same time, the cascaded hierarchical structure ensures recovery even under extreme optical congestion or staining artifacts. (2) Single-cell accuracy: High-fidelity coordinate recovery is achieved, ensuring that gene transcripts can be accurately assigned to the cell nucleus. (3) Parameter-free automation: The self-guided multi-scale schedule reduces / eliminates manual heuristic parameter tuning. (4) Subpixel resolution is achieved: Coordinates are recovered under extreme distortion using TPS transformation. (5) Bright field-fluorescence intensity mismatch is solved by skewness-aware adaptive phase reversal feedback loop.

[0136] Please refer to Figure 8 In some embodiments, multi-scale quasi-convexity is incorporated to validate the image registration method based on spatial omics and in situ sequencing in steps S10 to S70. Multi-scale quasi-convexity is a theoretical condition where multi-scale Gaussian smoothing eliminates local minima in the objective function, making the search terrain quasi-convex on a large scale, thereby ensuring global convergence. The robustness of the above image registration method is based on the operation of energy terrain and operates in scale-space scheduling. Figure 8 (a) shows the registration energy terrain evolution under large-scale smoothing. The terrain exhibits quasi-convexity, and strictly convex terrain ensures global convergence, allowing the optimizer to avoid the local minima "traps" unique to high-frequency biological textures. Figure 8 (b) in the figure represents the registration energy-topographic evolution under intermediate-scale smoothing, which shows a potential local minimum. Figure 8 (c) represents the registration energy terrain evolution under the original scale smoothing, which is a highly non-convex terrain with multiple local traps.

[0137] Please refer to Figure 9 In one embodiment, an image registration device based on spatial omics and in situ sequencing is provided, comprising:

[0138] The data acquisition module 100 is used to acquire bright-field histological images, fluorescence sequencing images, and spatial genomic metadata of the target specimen.

[0139] The matrix calculation module 200 is used to calculate the global translation parameters between the bright-field histology image and the fluorescence sequencing image. Based on the global translation parameters, the bright-field histology image and the fluorescence sequencing image, affine optimization is performed to obtain the first affine matrix between the bright-field histology image and the fluorescence sequencing image. The global translation parameters are used to characterize the displacement relationship between the bright-field histology image and the fluorescence sequencing image.

[0140] The index calculation module 300 is used to calculate a first convergence index based on a first affine matrix, a bright-field histological image, and a fluorescence sequencing image; wherein, the first convergence index is used to characterize the correlation between the fluorescence sequencing image and the bright-field histological image after transformation using the first affine matrix.

[0141] The matrix determination module 400 is used to take the first affine matrix as the final affine matrix if the first convergence index is greater than or equal to the preset convergence threshold, and to calculate the final affine matrix based on the local correlation between the bright field histology image and the fluorescence sequencing image if the first convergence index is less than the convergence threshold.

[0142] Image registration module 500 is used to register spatial genomic metadata using a final affine matrix to map the spatial genomic metadata onto a bright-field histological image.

[0143] One embodiment provides an image registration device based on spatial omics and in situ sequencing, comprising:

[0144] Memory, used to store programs;

[0145] A processor is used to implement an image registration method by executing a program stored in memory.

[0146] One embodiment provides a computer program product including a computer program and / or instructions, which, when executed by a processor, implement an image registration method.

[0147] In the above embodiments, implementation can be achieved, in whole or in part, by software, hardware, firmware, or any combination thereof. Furthermore, as those skilled in the art will understand, the principles herein can be reflected in a computer program product on a computer-readable storage medium pre-loaded with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CDs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to form a machine, such that instructions executing on the computer or other programmable data processing apparatus can generate means for performing a specified function. These computer program instructions can also be stored in a computer-readable storage medium that can instruct the computer or other programmable data processing apparatus to operate in a particular manner, such that instructions stored in the computer-readable storage medium can form an article of manufacture, including means for implementing the specified function. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to perform a series of operational steps on the computer or other programmable apparatus to produce a computer-implemented process, such that instructions executing on the computer or other programmable apparatus can provide steps for implementing the specified function.

[0148] This document describes various exemplary embodiments with reference to them. However, those skilled in the art will recognize that changes and modifications can be made to the exemplary embodiments without departing from the scope of this document. For example, various operational steps and components for performing operational steps can be implemented in different ways depending on the specific application or considering any number of cost functions associated with the operation of the system (e.g., one or more steps can be deleted, modified, or combined with other steps).

[0149] While the principles herein have been illustrated in various embodiments, numerous modifications to the structures, arrangements, proportions, elements, materials, and components, particularly suited to specific environments and operational requirements, may be used without departing from the principles and scope of this disclosure. These modifications and other alterations or alterations will be included within the scope of this document. Those skilled in the art will recognize that many changes can be made to the details of the above embodiments without departing from the fundamental principles of the invention.

Claims

1. An image registration method based on space omics and in situ sequencing, characterized in that, include: Acquire bright-field histological images, fluorescence sequencing images, and spatial genomic metadata of the target specimen; Calculate the global translation parameter between the bright-field histological image and the fluorescence sequencing image, and perform affine optimization based on the global translation parameter, the bright-field histological image, and the fluorescence sequencing image to obtain a first affine matrix between the bright-field histological image and the fluorescence sequencing image; wherein, the global translation parameter is used to characterize the displacement relationship between the bright-field histological image and the fluorescence sequencing image; A first convergence index is calculated based on the first affine matrix, the bright-field histological image, and the fluorescence sequencing image; wherein, the first convergence index is used to characterize the correlation between the fluorescence sequencing image transformed by the first affine matrix and the bright-field histological image; If the first convergence index is greater than or equal to the preset convergence threshold, the first affine matrix is ​​used as the final affine matrix; if the first convergence index is less than the convergence threshold, the final affine matrix is ​​calculated based on the local correlation between the bright-field histology image and the fluorescence sequencing image. The spatial genomic metadata is registered using the final affine matrix to map the spatial genomic metadata onto the bright-field histological image.

2. The image registration method as described in claim 1, characterized in that, The calculation of the final affine matrix based on the local correlation between the bright-field histology image and the fluorescence sequencing image includes: The final affine matrix is ​​calculated using a feature-matching-based affine matrix calculation method and / or a phase-correlation-based affine matrix calculation method. The final affine matrix is ​​calculated using a feature-matching-based affine matrix calculation method, including: Multiple key feature points of the fluorescence sequencing image are obtained, and a preset clustering algorithm is used to identify the target feature point clusters corresponding to the multiple key feature points; The homography matrix corresponding to the target feature point cluster is calculated based on a preset parameter estimation method, and the homography matrix is ​​used as the final affine matrix. The final affine matrix is ​​calculated using a phase-dependent affine matrix calculation method, including: The phase correlation coefficients of the bright-field histology image and the fluorescence sequencing image are obtained, and the final affine matrix is ​​constructed based on the phase correlation coefficients.

3. The image registration method as described in claim 2, characterized in that, The acquisition of multiple key feature points of the fluorescence sequencing image includes: The fluorescence sequencing image is used to extract features using different types of feature detectors, and the feature points output by each feature detector are obtained. The feature points output by each feature detector are traversed to construct a union, and the feature points in the union are used as multiple key feature points.

4. The image registration method as described in claim 2, characterized in that, The step of obtaining the phase correlation coefficient between the bright-field histology image and the fluorescence sequencing image, and constructing the final affine matrix based on the phase correlation coefficient, includes: The bright-field histological image and the fluorescence sequencing image are subjected to a first downsampling process to obtain a first downsampled bright-field histological image and a first downsampled fluorescence sequencing image, respectively. The bright-field histological image and the fluorescence sequencing image are then subjected to a second downsampling process to obtain a second downsampled bright-field histological image and a second downsampled fluorescence sequencing image, respectively. The first downsampling process and the second downsampling process use a first resolution ratio and a second resolution ratio, respectively, and the first resolution ratio and the second resolution ratio are different. A first phase correlation coefficient is calculated based on the first downsampled bright-field histological image and the first downsampled fluorescence sequencing image, and a second phase correlation coefficient is calculated based on the second downsampled bright-field histological image and the second downsampled fluorescence sequencing image. The final affine matrix is ​​constructed based on the first phase correlation coefficient and the second phase correlation coefficient.

5. The image registration method as described in claim 1, characterized in that, The registration of the spatial genome metadata using the final affine matrix includes: Dense corresponding blocks are extracted from the bright-field histological image and the fluorescence sequencing image, and the initial deformation field is calculated based on the dense corresponding blocks; The final deformation field is calculated based on the initial deformation field and the final affine matrix. The final deformation field is then inverted and mapped onto the spatial genome metadata.

6. The image registration method as described in claim 1, characterized in that, The affine optimization based on the global translation parameters, the bright-field histology image, and the fluorescence sequencing image to obtain the first affine matrix between the bright-field histology image and the fluorescence sequencing image includes: The fluorescence sequencing image is shifted using the global translation parameters to obtain a shifted fluorescence sequencing image. Based on a preset image registration algorithm, the first affine matrix between the bright-field histology image and the translated fluorescence sequencing image is solved.

7. The image registration method as described in claim 1, characterized in that, Obtain fluorescence sequencing images of the target specimen, including: Obtain the initial sequencing image of the target specimen and calculate the skewness coefficient of the initial sequencing image; When the skewness coefficient is less than the preset skewness threshold, the initial sequencing image is phase-reversed to obtain the reversed initial sequencing image; The reversed initial sequencing image is subjected to image enhancement processing to obtain the fluorescence sequencing image of the target specimen.

8. An image registration device based on space omics and in situ sequencing, characterized in that, include: The data acquisition module is used to acquire bright-field histological images, fluorescence sequencing images, and spatial genomic metadata of the target specimen; A matrix calculation module is used to calculate the global translation parameter between the bright-field histological image and the fluorescence sequencing image, and to perform affine optimization based on the global translation parameter, the bright-field histological image, and the fluorescence sequencing image to obtain a first affine matrix between the bright-field histological image and the fluorescence sequencing image; wherein, the global translation parameter is used to characterize the displacement relationship between the bright-field histological image and the fluorescence sequencing image; The index calculation module is used to calculate a first convergence index based on the first affine matrix, the bright-field histology image, and the fluorescence sequencing image; wherein, the first convergence index is used to characterize the correlation between the fluorescence sequencing image after transformation using the first affine matrix and the bright-field histology image; The matrix determination module is used to take the first affine matrix as the final affine matrix if the first convergence index is greater than or equal to the preset convergence threshold, and to calculate the final affine matrix based on the local correlation between the bright field histology image and the fluorescence sequencing image if the first convergence index is less than the convergence threshold. An image registration module is used to register the spatial genomic metadata using the final affine matrix, so as to map the spatial genomic metadata onto the bright-field histological image.

9. An image registration device based on space omics and in situ sequencing, characterized in that, include: Memory, used to store programs; A processor for implementing the image registration method as described in any one of claims 1-7 by executing a program stored in the memory.

10. A computer program product comprising a computer program and / or instructions, characterized in that, When the computer program and / or instructions are executed by the processor, they implement the image registration method as described in any one of claims 1-7.

Images