Spatial omics single cell data acquisition method and apparatus, and electronic device

Abstract

The invention discloses a spatial omics single cell data acquisition method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a dyeing image of a biological sample, and acquiring a gene expression visual image of the biological sample; carrying out registration on the dyed image and the gene expression visualization image to obtain a registered image; performing cell segmentation on the registered image to obtain a cell mask pattern; determining cells of background gene molecules of unknown cells in the gene expression visualized image according to the cell mask graph, and marking the cells to obtain a space group molecular expression matrix with cell markers. Compared with the prior art, the method has the advantages that high-precision spatial single cell data is obtained through the processing flows of gene expression matrix graph, splicing, registration, segmentation and correction.

Description

Spatial omics single-cell data acquisition method, device and electronic equipment

[0001] Cross-references to related art

[0002] This application is based on PCT patent application number PCT / CN2022 / 102505 and application date June 29, 2022, based on PCT patent application number PCT / CN2022 / 102424 and application date June 29, 2022, based on PCT patent application number PCT / CN2022 / 102360 and application date June 29, 2022, based on PCT patent application number PCT / CN2022 / 102443 and application date June 29, 2022, based on PCT patent application number PCT / CN2022 / 110779 and application date August 8, 2022, and claims priority to the above PCT patent applications. The entire contents of the above PCT patent applications are hereby incorporated into this application by reference. Technical Field

[0003] The present application relates to the field of data processing technology, and in particular to a method and device for acquiring spatial omics single-cell data and an electronic device. Background Art

[0004] Spatial omics techniques can reveal the functional and structural heterogeneity of internal spaces within tissues at different stages of development, creating favorable conditions for in-depth study of tissue structure and intercellular communication. Currently, some spatial transcriptomics technologies have advanced to capture the entire transcriptome, but in most cases, the expression matrix they provide lacks the resolution to reach the single-cell level. Therefore, how to obtain single-cell data is an urgent challenge.

[0005] Summary of the Invention

[0006] The present application provides a method, device, electronic device and storage medium for acquiring spatial omics single-cell data.

[0007] According to the first aspect of the present application, a method for acquiring spatial omics single-cell data is provided, comprising:

[0008] Obtaining a staining image of the biological sample and obtaining a gene expression visualization image of the biological sample;

[0009] registering the stained image with the gene expression visualization image to obtain a registered image;

[0010] performing cell segmentation on the registered image to obtain a cell mask image;

[0011] The cells to which background gene molecules of unknown cells belong in the gene expression visualization image are determined according to the cell mask image, and the cells to which they belong are marked to obtain a spatial component molecular expression matrix with cell labels.

[0012] Optionally, obtaining a stained image of the biological sample includes:

[0013] Acquire a stained image of the biological sample through a microscope; or

[0014] Acquire a stitched staining image of the biological sample composed of a plurality of sub-staining images through a microscope; or

[0015] A plurality of sub-stained images of the biological sample are acquired through the microscope, and the plurality of sub-stained images are spliced ​​to obtain a spliced ​​stained image.

[0016] Optionally, obtaining a stained image of the biological sample includes:

[0017] Acquire multiple sub-stained images of the biological sample through a microscope; perform a first stitching on the multiple sub-stained images of the biological sample to obtain a first stained image;

[0018] determining whether to perform a second stitching on the plurality of sub-stained images of the biological sample according to the image quality of the first stained image;

[0019] If not, determining the first stained image as the stained image;

[0020] If so, a second stitching is performed on the multiple sub-stained images of the biological sample to obtain the stained image.

[0021] Optionally, acquiring a plurality of sub-stained images of the biological sample through the microscope, and splicing the plurality of sub-stained images to obtain a spliced ​​stained image includes:

[0022] respectively determining overlapping areas of the plurality of sub-stained images and the template image;

[0023] Cutting the parts of the multiple sub-stained images and the template image determined to be overlapping areas into a plurality of stained image overlapping sub-images and a plurality of template image overlapping sub-images of the same number, respectively, to pair them into a plurality of overlapping sub-image pairs;

[0024] Obtaining the offset between each sub-stained image and the template image by performing frequency domain calculation on the multiple overlapping sub-image pairs; and

[0025] Each of the sub-stained images is translated by the offset amount toward the direction of the template image, and is then spliced ​​together with the template image.

[0026] Optionally, acquiring a plurality of sub-stained images of the biological sample through the microscope, and splicing the plurality of sub-stained images to obtain a spliced ​​stained image includes:

[0027] acquiring the plurality of sub-stained images and template information about trajectory lines or trajectory points of the biological sample;

[0028] Pre-stitching the multiple sub-stained images to obtain pre-stitched coordinates of the multiple sub-stained images;

[0029] selecting at least one first reference image from the plurality of sub-stained images according to features of the trajectory lines or trajectory points, and deriving a global template according to template information of the biological sample and pre-spliced ​​coordinates of the first reference image; and

[0030] For a second reference image other than the first reference image in the multiple sub-stained images, the offset between the pre-stitched coordinates of the second reference image and the corresponding template coordinates of the second reference image in the global template is calculated, and the coordinates of the second reference image are adjusted based on the offset to stitch together and generate the stitched stained image.

[0031] Optionally, registering the stained image with the gene expression visualization image to obtain a registered image includes:

[0032] Obtaining a trajectory line in the staining image, and rotating the staining image based on an angle between the trajectory line and a horizontal direction; wherein the staining image and the gene expression visualization image include a plurality of trajectory lines and / or a plurality of trajectory points, wherein the trajectory point is formed by the intersection of two trajectory lines, and each of the plurality of trajectory lines has a corresponding index number;

[0033] Enlarging or reducing the stained image to obtain a third reference image, wherein the third reference image has the same scale as the gene expression visualization image;

[0034] determining a target offset based on the gene expression visualization image and the third reference image;

[0035] The third reference image is moved based on the target offset to obtain the registered image.

[0036] Optionally, after the stained image is magnified or reduced to obtain a third reference image, the method further includes:

[0037] determining a similarity between the third reference image and the gene expression visualization image;

[0038] The position corresponding to the maximum similarity value is taken as the rough registration result.

[0039] Optionally, registering the stained image with the gene expression visualization image to obtain a registered image includes:

[0040] Acquire a staining image and a gene expression visualization image, wherein each of the staining image and the gene expression visualization image includes a plurality of trajectory lines and a plurality of trajectory points, wherein the trajectory point is formed by the intersection of two of the trajectory lines, and each of the trajectory lines has a corresponding index number;

[0041] Preprocessing the stained image to obtain a candidate image set;

[0042] Determine a binary image set corresponding to the candidate image set;

[0043] Determining, based on the similarity between the binary graph set and the gene expression visualization image, trajectory point pairs having a corresponding relationship between the binary graph set and the gene expression visualization image;

[0044] determining offset information based on the trajectory point pairs;

[0045] A target image is processed based on the offset information to obtain a registered image, wherein the target image is an image in the candidate image set corresponding to the track point pair.

[0046] Optionally, before performing cell segmentation on the registered image to obtain a cell mask image, the method further includes:

[0047] Tissue segmentation is performed on the registered image to obtain a tissue mask image.

[0048] Optionally, performing tissue segmentation on the registered image to obtain a tissue mask image includes:

[0049] Obtaining gene expression levels of the target tissue to be segmented in the registered image and determining a gene expression matrix of the registered image, wherein the gene expression matrix arranges the gene expression levels according to pixel positions;

[0050] Determining image data by assigning the gene expression amount at each pixel position in the gene expression matrix to the corresponding pixel position in the registered image;

[0051] performing image processing on the image data to determine foreground pixels of the image data;

[0052] The connected areas of the image data are marked according to the foreground pixels to obtain the tissue mask image.

[0053] Optionally, after performing tissue segmentation on the registered image to obtain a tissue mask image, the method further includes:

[0054] Calculate the outer contour image and inner contour image of the grayscale image corresponding to the tissue mask image respectively;

[0055] Dividing the outer contour image and the inner contour image into N segments respectively, and pairing each segment of the outer contour image with each segment of the inner contour image in pairs to obtain N image combinations, each of which includes a segment of the outer contour image and a segment of the inner contour image;

[0056] Calculating the pixel density ratio of a segment of the outer contour image to a segment of the inner contour image in each of the image combinations to obtain N pixel density ratios, where the pixel density ratio is the ratio of the pixel proportion of the segment of the outer contour image to the segment of the inner contour image in each of the image combinations, and the pixel proportion is the ratio of the pixel points to the total pixel points;

[0057] The grayscale image quality corresponding to the tissue mask image is calculated according to the pixel density ratio.

[0058] Optionally, performing cell segmentation on the registered image to obtain a cell mask image includes:

[0059] Inputting the registered image into the image segmentation model to obtain a segmentation result of each pixel in the registered image, wherein the segmentation result includes the pixel belonging to a cell, a cell outline, or an image background;

[0060] The pixel points belonging to cells or cell outlines in the segmentation results are determined to be pixel points belonging to cells, and the segmentation results of the registered image are output to obtain the cell mask image.

[0061] Optionally, performing cell segmentation on the registered image to obtain a cell mask image includes:

[0062] Preprocessing the gene expression visualization image to obtain a preprocessed image;

[0063] Binarizing the preprocessed image to obtain an initial mask image;

[0064] According to the initial mask image, a watershed algorithm based on distance transformation is used to segment the connected domain where cells are adhered, so as to obtain the segmented cell mask image.

[0065] Optionally, performing cell segmentation on the registered image to obtain a cell mask image includes:

[0066] Determining, based on the stained image, the labeled regions and cell areas corresponding to each tissue type of the target object;

[0067] Determining the area of ​​a to-be-processed sub-region contained in each of the annotated regions based on the cell area, wherein the area of ​​the to-be-processed sub-region is positively correlated with the cell area, and the to-be-processed sub-region is used to represent the minimum processing unit of the corresponding annotated region;

[0068] According to the area of ​​each of the sub-regions to be processed, the corresponding marked region is gridded to obtain grid coordinates corresponding to each of the marked regions;

[0069] Data corresponding to the gridded coordinates are extracted from the gene expression visualization image and superimposed onto the gridded coordinates to obtain the cell treatment result of the target object.

[0070] Optionally, determining the cells to which background gene molecules of unknown cells belong in the gene expression visualization image belong based on the cell mask image includes:

[0071] Based on the cell mask image, each gene expression feature of the gene expression visualization image is determined as a cell gene expression feature located inside the cell and a background gene expression feature located outside the cell;

[0072] For each of the cell gene expression features, a probability distribution model is fitted based on the spatial position and the signal value of the gene expression feature;

[0073] Determining, based on the probability distribution model, a probability value of a target background gene expression feature located within a preset range of a target cell belonging to the target cell;

[0074] Outputting a correction result according to the probability value to correct the background gene expression characteristics.

[0075] Optionally, determining the cells to which background gene molecules of unknown cells belong in the gene expression visualization image belong based on the cell mask image includes:

[0076] Determining the initial gene molecules belonging to the target cells and the background gene molecules of the unknown cells according to the gene expression visualization image;

[0077] Obtaining coordinate information of the initial gene molecule;

[0078] Determining the center point coordinates of the target cell according to the coordinate information;

[0079] Constructing a Voronoi diagram according to the center point coordinates and the coordinate information;

[0080] The cells to which the background gene molecules within the Voronoi diagram belong are determined.

[0081] Optionally, determining the cells to which background gene molecules of unknown cells belong in the gene expression visualization image belong based on the cell mask image includes:

[0082] Determining the initial number of gene molecules belonging to the target cell in the gene expression visualization image and the initial contour area of ​​the target region occupied by the gene molecules belonging to the target cell;

[0083] Determining the gene molecule density of the target cell according to the number of molecules and the initial contour area;

[0084] Determine the number of background gene molecules within a preset region of the target region and the area of ​​the preset region;

[0085] Determining the background gene molecule density according to the number of background gene molecules and the area of ​​the region;

[0086] The cell to which the background gene molecule belongs is determined according to the gene molecule density and the background gene molecule density.

[0087] Optionally, determining the cells to which background gene molecules of unknown cells belong in the gene expression visualization image belong based on the cell mask image includes:

[0088] determining a target region occupied by gene molecules belonging to target cells in the gene expression visualization image;

[0089] Determine the spatial coordinates and gene expression characteristic signal values ​​of background gene molecules of unknown cells within a preset calibration range of the target area;

[0090] Inputting the spatial coordinates and the gene expression characteristic signal values ​​into a probability distribution model to obtain a probability score of the background gene molecule belonging to the target cell; the probability distribution model is fitted based on the spatial coordinates and gene expression characteristic signal values ​​of the gene molecule sample belonging to the target cell;

[0091] The cell to which the background gene molecule belongs is determined according to the probability score.

[0092] Optionally, the cell labeling includes:

[0093] The cells are stained with ssDNA staining solution and cell boundary fluorescent staining solution, wherein the ssDNA staining solution includes a buffer and an ssDNA reagent, and the cell boundary fluorescent staining solution includes a fluorescent dye.

[0094] Optionally, the method further includes:

[0095] Acquiring a staining image or a gene expression visualization image to be tested; and

[0096] Input the staining image or gene expression visualization image to be tested into a directed target detection network to obtain a target detection result,

[0097] The directed target detection network includes:

[0098] The backbone layer is configured to perform feature extraction on the staining image or gene expression visualization image to be tested to obtain multiple feature maps at different scales;

[0099] A Neck layer is configured to receive the multiple feature maps from the Backbone layer and perform cross-scale fusion on the multiple feature maps to obtain multiple fused feature maps; and

[0100] The detection head layer is configured to predict the multiple fused images to obtain a target detection result about the staining image or gene expression visualization image to be tested,

[0101] The target detection result includes: classification information, target frame coordinate prediction information, target prediction information and angle prediction information.

[0102] Optionally, the dyeing image plane where the dyeing image is located includes a first direction and a second direction perpendicular to the first direction, and the dyeing image includes a plurality of first straight track lines transverse to the second direction and a plurality of second straight track lines transverse to the first direction.

[0103] The method for determining the coordinates of points in the stained image includes:

[0104] Sampling steps include:

[0105] Apply a first sampling strip to perform equally spaced sampling in the first direction on the stained image along the first direction to determine a first candidate point set, wherein for each equally spaced sampling in the first direction, perform the following sub-steps:

[0106] Accumulating and summing pixel values ​​of pixels within the sampling area of ​​the stained image selected by the first sampling strip during the first direction equally spaced sampling to obtain a series of first pixel value sums; and

[0107] Determine the extreme points of the series of first pixel value sums, and for each determined extreme point: determine a corresponding first candidate point obtained by the first-direction equally-interval sampling, wherein the coordinates of the corresponding first candidate point in the second direction are the coordinates of the second-direction position corresponding to the extreme point in the second direction, and the coordinates of the corresponding first candidate point in the first direction are determined based on the position of the first sampling strip in the first direction during the first-direction equally-interval sampling.

[0108] The first candidate point set is formed by first candidate points obtained by sampling in the first direction at equal intervals each time;

[0109] Apply a second sampling strip to perform equally spaced sampling in the second direction on the stained image along the second direction to determine a second set of candidate points, wherein for each equally spaced sampling in the second direction, perform the following sub-steps:

[0110] Accumulating and summing pixel values ​​of pixels within the sampling area of ​​the stained image selected by the second sampling strip during the second direction equally spaced sampling to obtain a series of second pixel value sums; and

[0111] Determine the extreme points of the series of second pixel value sums, and for each determined extreme point: determine a corresponding second candidate point obtained by the second-direction equally-interval sampling, where the coordinates of the corresponding second candidate point in the first direction are the coordinates of the first-direction position corresponding to the extreme point in the first direction, and the coordinates of the corresponding second candidate point in the second direction are determined based on the position of the second sampling strip in the second direction during the second-direction equally-interval sampling.

[0112] The second candidate point set is formed by the second candidate points obtained by sampling in the second direction at equal intervals each time;

[0113] The grouping steps include:

[0114] dividing the first candidate points in the first candidate point set into a plurality of first candidate point groups; and

[0115] dividing the second candidate points in the second candidate point set into a plurality of second candidate point groups;

[0116] The straight line fitting steps include:

[0117] Performing straight line fitting on the first candidate point in each first candidate point group to obtain a corresponding first straight line analytical expression; and

[0118] Performing straight line fitting on the second candidate points in each second candidate point group to obtain a corresponding second straight line analytical expression; and

[0119] Coordinate calculation steps include:

[0120] Calculate the coordinates of the intersection of each first straight line analytical expression and each second straight line analytical expression.

[0121] Optionally, after labeling the cells to obtain a spatial component molecular expression matrix with cell labels, the method further comprises:

[0122] Acquiring a first image of a biological sample, the first image including stained cell boundaries of the biological sample;

[0123] acquiring a second image of the biological sample, the second image comprising stained genetic material of the biological sample;

[0124] Obtaining gene expression information of the biological sample;

[0125] registering the second image with the first image to obtain a registered image of the second image and the first image;

[0126] Registering the second image with the gene expression level information to obtain a registered image of the second image and the gene expression level information;

[0127] Acquire a registered image of the first image and the gene expression amount information according to a registered image of the second image and the first image, and a registered image of the second image and the gene expression amount information;

[0128] Acquiring cell boundaries of the biological sample according to the first image and the registered image of the gene expression information;

[0129] The gene expression information of the cells of the biological sample is obtained according to the cell boundaries based on the first image and the registered image of the gene expression information.

[0130] In a second aspect, a spatial omics single-cell data acquisition device is also provided, comprising:

[0131] an acquisition unit, configured to acquire a stained image of the biological sample and a visualized image of gene expression of the biological sample;

[0132] a registration unit, configured to register the stained image with the gene expression visualization image to obtain a registered image;

[0133] a first segmentation unit, configured to perform cell segmentation on the registered image to obtain a cell mask image;

[0134] a correction unit, configured to determine, based on the cell mask image, to which the background gene molecules of unknown to which cells belong in the gene expression visualization image belong;

[0135] The labeling unit is used to label the cells to obtain a spatial component molecular expression matrix with cell labels.

[0136] Optionally, the acquiring unit is further configured to:

[0137] Acquire a stained image of the biological sample through a microscope; or

[0138] Acquire a stitched staining image of the biological sample composed of a plurality of sub-staining images through a microscope; or

[0139] A plurality of sub-stained images of the biological sample are acquired through the microscope, and the plurality of sub-stained images are spliced ​​to obtain a spliced ​​stained image.

[0140] Optionally, the acquiring unit includes:

[0141] a first acquisition module, configured to acquire a plurality of sub-stained images of the biological sample through a microscope; and perform a first splicing operation on the plurality of sub-stained images of the biological sample to obtain a first stained image;

[0142] a judgment module, configured to judge whether to perform a second splicing on the plurality of sub-stained images of the biological sample according to the image quality of the first stained image;

[0143] a determination module, configured to, if no, determine the first stained image as the stained image;

[0144] A second acquisition module is configured to perform a second splicing on the multiple sub-stained images of the biological sample to obtain the stained image.

[0145] Optionally, the first acquisition module is further configured to:

[0146] respectively determining overlapping areas of the plurality of sub-stained images and the template image;

[0147] Cutting the parts of the multiple sub-stained images and the template image determined to be overlapping areas into a plurality of stained image overlapping sub-images and a plurality of template image overlapping sub-images of the same number, respectively, to pair them into a plurality of overlapping sub-image pairs;

[0148] Obtaining the offset between each sub-stained image and the template image by performing frequency domain calculation on the multiple overlapping sub-image pairs; and

[0149] Each of the sub-stained images is translated by the offset amount toward the direction of the template image, and is then spliced ​​together with the template image.

[0150] Optionally, the first acquisition module is further configured to:

[0151] acquiring the plurality of sub-stained images and template information about trajectory lines or trajectory points of the biological sample;

[0152] Pre-stitching the multiple sub-stained images to obtain pre-stitched coordinates of the multiple sub-stained images;

[0153] selecting at least one first reference image from the plurality of sub-stained images according to features of the trajectory lines or trajectory points, and deriving a global template according to template information of the biological sample and pre-spliced ​​coordinates of the first reference image; and

[0154] For a second reference image other than the first reference image in the multiple sub-stained images, the offset between the pre-stitched coordinates of the second reference image and the corresponding template coordinates of the second reference image in the global template is calculated, and the coordinates of the second reference image are adjusted based on the offset to stitch together and generate the stitched stained image.

[0155] Optionally, the registration unit includes:

[0156] an acquisition module, configured to acquire a trajectory line in the staining image and rotate the staining image based on an angle between the trajectory line and a horizontal direction; wherein the staining image and the gene expression visualization image include a plurality of trajectory lines and / or a plurality of trajectory points, wherein a trajectory point is formed by the intersection of two trajectory lines, and each of the plurality of trajectory lines has a corresponding index number;

[0157] a processing module, configured to magnify or reduce the stained image to obtain a third reference image, wherein the third reference image has the same scale as the gene expression visualization image;

[0158] a first determining module, configured to determine a target offset based on the gene expression visualization image and the third reference image;

[0159] A registration module is configured to move the third reference image based on the target offset to obtain the registered image.

[0160] Optionally, the registration unit further includes:

[0161] a second determining module, configured to determine the similarity between the third reference image and the gene expression visualization image after the processing module amplifies or reduces the stained image to obtain a third reference image;

[0162] The coarse registration module is used to take the position corresponding to the maximum similarity as the coarse registration result.

[0163] Optionally, the registration unit is further configured to:

[0164] Acquire a staining image and a gene expression visualization image, wherein each of the staining image and the gene expression visualization image includes a plurality of trajectory lines and a plurality of trajectory points, wherein the trajectory point is formed by the intersection of two of the trajectory lines, and each of the trajectory lines has a corresponding index number;

[0165] Preprocessing the stained image to obtain a candidate image set;

[0166] Determine a binary image set corresponding to the candidate image set;

[0167] Determining, based on the similarity between the binary graph set and the gene expression visualization image, trajectory point pairs having a corresponding relationship between the binary graph set and the gene expression visualization image;

[0168] determining offset information based on the trajectory point pairs;

[0169] A target image is processed based on the offset information to obtain a registered image, wherein the target image is an image in the candidate image set corresponding to the track point pair.

[0170] Optionally, the device further includes:

[0171] The second segmentation unit is configured to perform tissue segmentation on the registered image to obtain a tissue mask image before the first segmentation unit performs cell segmentation on the registered image to obtain a cell mask image.

[0172] Optionally, the second segmentation unit is further configured to:

[0173] Obtaining gene expression levels of the target tissue to be segmented in the registered image and determining a gene expression matrix of the registered image, wherein the gene expression matrix arranges the gene expression levels according to pixel positions;

[0174] Determining image data by assigning the gene expression amount at each pixel position in the gene expression matrix to the corresponding pixel position in the registered image;

[0175] performing image processing on the image data to determine foreground pixels of the image data;

[0176] The connected areas of the image data are marked according to the foreground pixels to obtain the tissue mask image.

[0177] Optionally, the device further includes:

[0178] a calculation unit, configured to perform tissue segmentation on the registered image by the second segmentation unit to obtain a tissue mask image, and then respectively calculate an outer contour image and an inner contour image of the grayscale image corresponding to the tissue mask image;

[0179] a pairing unit, configured to divide the outer contour image and the inner contour image into N segments, and pair each segment of the outer contour image with each segment of the inner contour image to obtain N image combinations, each of which includes a segment of the outer contour image and a segment of the inner contour image;

[0180] The calculation unit is further configured to respectively calculate a pixel density ratio of a segment of the outer contour image to a segment of the inner contour image in each of the image combinations to obtain N pixel density ratios, wherein the pixel density ratio is a ratio of a pixel point to a total pixel point.

[0181] The calculation unit is further configured to calculate the grayscale image quality corresponding to the tissue mask image according to the pixel density ratio.

[0182] Optionally, the first segmentation unit is further configured to:

[0183] Inputting the registered image into the image segmentation model to obtain a segmentation result of each pixel in the registered image, wherein the segmentation result includes the pixel belonging to a cell, a cell outline, or an image background;

[0184] The pixel points belonging to cells or cell outlines in the segmentation results are determined to be pixel points belonging to cells, and the segmentation results of the registered image are output to obtain the cell mask image.

[0185] Optionally, the first segmentation unit is further configured to:

[0186] Preprocessing the gene expression visualization image to obtain a preprocessed image;

[0187] Binarizing the preprocessed image to obtain an initial mask image;

[0188] According to the initial mask image, a watershed algorithm based on distance transformation is used to segment the connected domain where cells are adhered, so as to obtain the segmented cell mask image.

[0189] Optionally, the first segmentation unit is further configured to:

[0190] Determining, based on the stained image, the labeled regions and cell areas corresponding to each tissue type of the target object;

[0191] Determining the area of ​​a to-be-processed sub-region contained in each of the annotated regions based on the cell area, wherein the area of ​​the to-be-processed sub-region is positively correlated with the cell area, and the to-be-processed sub-region is used to represent the minimum processing unit of the corresponding annotated region;

[0192] According to the area of ​​each of the sub-regions to be processed, the corresponding marked region is gridded to obtain grid coordinates corresponding to each of the marked regions;

[0193] Data corresponding to the gridded coordinates are extracted from the gene expression visualization image and superimposed onto the gridded coordinates to obtain the cell treatment result of the target object.

[0194] Optionally, the correction unit is further configured to:

[0195] Based on the cell mask image, each gene expression feature of the gene expression visualization image is determined as a cell gene expression feature located inside the cell and a background gene expression feature located outside the cell;

[0196] For each of the cell gene expression features, a probability distribution model is fitted based on the spatial position and the signal value of the gene expression feature;

[0197] Determining, based on the probability distribution model, a probability value of a target background gene expression feature located within a preset range of a target cell belonging to the target cell;

[0198] Outputting a correction result according to the probability value to correct the background gene expression characteristics.

[0199] Optionally, the correction unit is further configured to include:

[0200] Determining the initial gene molecules belonging to the target cells and the background gene molecules of the unknown cells according to the gene expression visualization image;

[0201] Obtaining coordinate information of the initial gene molecule;

[0202] Determining the center point coordinates of the target cell according to the coordinate information;

[0203] Constructing a Voronoi diagram according to the center point coordinates and the coordinate information;

[0204] The cells to which the background gene molecules within the Voronoi diagram belong are determined.

[0205] Optionally, the correction unit is further configured to include:

[0206] Determining the initial number of gene molecules belonging to the target cell in the gene expression visualization image and the initial contour area of ​​the target region occupied by the gene molecules belonging to the target cell;

[0207] determining the gene molecule density of the target cell according to the number of molecules and the initial contour area;

[0208] Determine the number of background gene molecules within a preset region of the target region and the area of ​​the preset region;

[0209] Determining the background gene molecule density according to the number of background gene molecules and the area of ​​the region;

[0210] The cell to which the background gene molecule belongs is determined according to the gene molecule density and the background gene molecule density.

[0211] Optionally, the correction unit is further configured to include:

[0212] determining a target region occupied by gene molecules belonging to target cells in the gene expression visualization image;

[0213] Determine the spatial coordinates and gene expression characteristic signal values ​​of background gene molecules of unknown cells within a preset calibration range of the target area;

[0214] Inputting the spatial coordinates and the gene expression characteristic signal values ​​into a probability distribution model to obtain a probability score of the background gene molecule belonging to the target cell; the probability distribution model is fitted based on the spatial coordinates and gene expression characteristic signal values ​​of the gene molecule sample belonging to the target cell;

[0215] The cell to which the background gene molecule belongs is determined according to the probability score.

[0216] Optionally, the marking unit is further used to:

[0217] The cells are stained with ssDNA staining solution and cell boundary fluorescent staining solution, wherein the ssDNA staining solution includes a buffer and an ssDNA reagent, and the cell boundary fluorescent staining solution includes a fluorescent dye.

[0218] Optionally, the device further includes a detection unit, including:

[0219] Acquiring a staining image or a gene expression visualization image to be tested; and

[0220] Input the staining image or gene expression visualization image to be tested into a directed target detection network to obtain a target detection result,

[0221] The directed target detection network includes:

[0222] The backbone layer is configured to perform feature extraction on the staining image or gene expression visualization image to be tested to obtain multiple feature maps at different scales;

[0223] A Neck layer is configured to receive the multiple feature maps from the Backbone layer and perform cross-scale fusion on the multiple feature maps to obtain multiple fused feature maps; and

[0224] The detection head layer is configured to predict the multiple fused images to obtain a target detection result about the staining image or gene expression visualization image to be tested,

[0225] The target detection result includes: classification information, target frame coordinate prediction information, target prediction information and angle prediction information.

[0226] Optionally, the dyeing image plane where the dyeing image is located includes a first direction and a second direction perpendicular to the first direction, and the dyeing image includes a plurality of first straight track lines transverse to the second direction and a plurality of second straight track lines transverse to the first direction.

[0227] The method for determining the coordinates of points in the stained image includes:

[0228] Sampling steps include:

[0229] Apply a first sampling strip to perform equally spaced sampling in the first direction on the stained image along the first direction to determine a first candidate point set, wherein for each equally spaced sampling in the first direction, perform the following sub-steps:

[0230] Accumulating and summing pixel values ​​of pixels within the sampling area of ​​the stained image selected by the first sampling strip during the first direction equally spaced sampling to obtain a series of first pixel value sums; and

[0231] Determine the extreme points of the series of first pixel value sums, and for each determined extreme point: determine a corresponding first candidate point obtained by the first-direction equally-interval sampling, wherein the coordinates of the corresponding first candidate point in the second direction are the coordinates of the second-direction position corresponding to the extreme point in the second direction, and the coordinates of the corresponding first candidate point in the first direction are determined based on the position of the first sampling strip in the first direction during the first-direction equally-interval sampling.

[0232] The first candidate point set is formed by first candidate points obtained by sampling in the first direction at equal intervals each time;

[0233] Apply a second sampling strip to perform equally spaced sampling in the second direction on the stained image along the second direction to determine a second set of candidate points, wherein for each equally spaced sampling in the second direction, perform the following sub-steps:

[0234] Accumulating and summing pixel values ​​of pixels within the sampling area of ​​the stained image selected by the second sampling strip during the second direction equally spaced sampling to obtain a series of second pixel value sums; and

[0235] Determine the extreme points of the series of second pixel value sums, and for each determined extreme point: determine a corresponding second candidate point obtained by the second-direction equally-interval sampling, where the coordinates of the corresponding second candidate point in the first direction are the coordinates of the first-direction position corresponding to the extreme point in the first direction, and the coordinates of the corresponding second candidate point in the second direction are determined based on the position of the second sampling strip in the second direction during the second-direction equally-interval sampling.

[0236] The second candidate point set is formed by the second candidate points obtained by sampling in the second direction at equal intervals each time;

[0237] The grouping steps include:

[0238] dividing the first candidate points in the first candidate point set into a plurality of first candidate point groups; and

[0239] dividing the second candidate points in the second candidate point set into a plurality of second candidate point groups;

[0240] The straight line fitting steps include:

[0241] Performing straight line fitting on the first candidate point in each first candidate point group to obtain a corresponding first straight line analytical expression; and

[0242] Performing straight line fitting on the second candidate points in each second candidate point group to obtain a corresponding second straight line analytical expression; and

[0243] Coordinate calculation steps include:

[0244] Calculate the coordinates of the intersection of each first straight line analytical expression and each second straight line analytical expression.

[0245] Optionally, the device further includes an analysis unit, configured to:

[0246] Acquiring a first image of a biological sample, the first image including stained cell boundaries of the biological sample;

[0247] acquiring a second image of the biological sample, the second image comprising stained genetic material of the biological sample;

[0248] Obtaining gene expression information of the biological sample;

[0249] registering the second image with the first image to obtain a registered image of the second image and the first image;

[0250] Registering the second image with the gene expression level information to obtain a registered image of the second image and the gene expression level information;

[0251] Acquire a registered image of the first image and the gene expression amount information according to a registered image of the second image and the first image, and a registered image of the second image and the gene expression amount information;

[0252] Acquiring cell boundaries of the biological sample according to the first image and the registered image of the gene expression information;

[0253] The gene expression information of the cells of the biological sample is obtained according to the cell boundaries based on the first image and the registered image of the gene expression information.

[0254] According to a third aspect of the present application, an electronic device is provided, including:

[0255] at least one processor; and

[0256] a memory communicatively connected to the at least one processor; wherein,

[0257] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0258] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0259] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program, wherein the computer program implements the method as described in the first aspect above when executed by a processor.

[0260] The spatial omics single-cell data acquisition method, device, electronic device and storage medium provided in this application obtain a stained image of a biological sample and obtain a gene expression visualization image of the biological sample; align the stained image with the gene expression visualization image to obtain an aligned image; perform cell segmentation on the aligned image to obtain a cell mask map; determine the cells to which the background gene molecules of the unknown cells in the gene expression visualization image belong based on the cell mask map, and mark the cells to which they belong to obtain a spatial group molecular expression matrix with cell labels. Compared with the related art, the embodiment of the present application obtains high-precision spatial single-cell data through a processing flow of gene expression matrix map, splicing, registration, segmentation and correction.

[0261] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0262] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present application.

[0263] FIG1 is a schematic diagram of a flow chart of a method for acquiring spatial omics single-cell data provided in an embodiment of the present application;

[0264] FIG2 is a schematic diagram of acquiring spatial omics single-cell data provided in an embodiment of the present application;

[0265] FIG3 is a schematic diagram of a process for obtaining a staining image of a biological sample according to an embodiment of the present application;

[0266] FIG4 is a schematic diagram of a flow chart of a method for stitching multiple sub-dyed images provided in an embodiment of the present application;

[0267] FIG5 is a schematic flow chart of another method for stitching multiple sub-dyed images provided in an embodiment of the present application;

[0268] FIG6 is a schematic diagram of a spatial distribution of multiple fields of view provided in an embodiment of the present application;

[0269] FIG7 is a schematic diagram of a pre-stitching result of an image stitching method provided in an embodiment of the present application;

[0270] FIG8a is a schematic diagram showing the positional relationship between two adjacent stained images in a row direction obtained by scanning in the horizontal direction during pre-stitching according to an embodiment of the present application;

[0271] FIG8 b is a schematic diagram showing the positional relationship between two adjacent stained images in a column direction obtained by scanning in the vertical direction during pre-stitching provided by an embodiment of the present application;

[0272] FIG9 is a schematic diagram of a single field of view template for stitching biological samples provided in an embodiment of the present application;

[0273] FIG10 is a schematic diagram of a construction process of a single field of view template for splicing biological samples provided in an embodiment of the present application;

[0274] FIG11 is a flow chart of a method for registering a staining image with a gene expression visualization image provided in an embodiment of the present application;

[0275] FIG12 is a schematic diagram of a dyeing image provided in an embodiment of the present application;

[0276] FIG13 is a schematic diagram of a third reference image provided in an embodiment of the present application;

[0277] FIG14a is a third reference image with multiple periods provided by an embodiment of the present application.

[0278] FIG14b is a visualization image of gene expression at multiple cycles provided in an embodiment of the present application;

[0279] FIG15 is a schematic diagram of a preset period determined with a first offset as the center, provided by an embodiment of the present application;

[0280] FIG16 is a schematic flow chart of another method for registering a staining image with a gene expression visualization image provided in an embodiment of the present application;

[0281] FIG17a is a schematic diagram of a candidate image provided in an embodiment of the present application;

[0282] FIG17 b is a schematic diagram of an initial binary image provided in an embodiment of the present application;

[0283] FIG17c is a schematic diagram of a target binary image provided in an embodiment of the present application;

[0284] FIG18 is a schematic flow chart of another method for registering a staining image with a gene expression visualization image provided in an embodiment of the present application;

[0285] FIG19 is a flowchart of determining image data of a tissue segmentation method for a sample image provided by an embodiment of the present application;

[0286] FIG20 is a schematic diagram of image data of a tissue segmentation method for a sample image provided by an embodiment of the present application;

[0287] FIG21 is a schematic diagram of a 2D convolution of a tissue segmentation method for a sample image provided by an embodiment of the present application;

[0288] FIG22 is a schematic diagram of low-level features of a tissue segmentation method for a sample image provided by an embodiment of the present application;

[0289] FIG23 is a flow chart of a method for calculating the quality of a grayscale image corresponding to a tissue mask provided in an embodiment of the present application;

[0290] FIG24 is a schematic diagram showing the relationship between an external contour image and an internal contour image provided by an embodiment of the present application;

[0291] FIG25 is a schematic diagram showing the relationship between another external contour image and an internal contour image provided by an embodiment of the present application;

[0292] FIG26 is a flow chart of a method for performing cell segmentation on a registered image provided in an embodiment of the present application;

[0293] FIG27 shows the segmentation effect of an image segmentation model provided in an embodiment of the present application on a mouse brain image;

[0294] FIG28 is a flow chart of another method for performing cell segmentation on registered images provided in an embodiment of the present application;

[0295] FIG29 is a schematic diagram showing a process flow of another cell segmentation processing method provided in an embodiment of the present application;

[0296] FIG30 shows a schematic diagram of an example process of the method according to the embodiment of the present application;

[0297] FIG31 shows an example diagram of the effect of the gene expression map provided in the embodiment of the present application;

[0298] FIG32 shows an example of a sharpened image obtained through sharpening processing according to an embodiment of the present application;

[0299] FIG33 is a schematic diagram showing the effect of the initial mask image provided by an embodiment of the present application;

[0300] FIG34 is a schematic diagram showing the effect of an output mask image provided by an embodiment of the present application;

[0301] FIG35 is a flow chart of another method for performing cell segmentation on a registered image provided in an embodiment of the present application;

[0302] FIG36 is a flow chart of a method for determining the cells to which background gene molecules of unknown cells belong in a gene expression visualization image provided by an embodiment of the present application;

[0303] FIG37 is a schematic diagram of the correspondence between a DNB and a cell membrane provided in an embodiment of the present application;

[0304] FIG38 is a schematic diagram of a correction result of a background DNB correction provided in an embodiment of the present application;

[0305] FIG39 is a flow chart of another method for determining the cells to which background gene molecules of unknown cells belong in a gene expression visualization image provided by an embodiment of the present application;

[0306] FIG40 is a flow chart of another method for determining the cells to which background gene molecules of unknown cells belong in a gene expression visualization image provided by an embodiment of the present application;

[0307] FIG41 is a schematic diagram of the center point of a target cell provided in an embodiment of the present application;

[0308] FIG42 is a flow chart of another method for determining the cells to which background gene molecules of unknown cells belong in a gene expression visualization image provided by an embodiment of the present application;

[0309] FIG43 is a schematic flow chart of a method for determining point coordinates in a stained image provided in an embodiment of the present application;

[0310] FIG44 is a schematic diagram of a dyeing image provided in an embodiment of the present application;

[0311] FIG45 is a flow chart of a directed target detection method provided in an embodiment of the present application;

[0312] FIG46 is a schematic flow chart of a method for obtaining cellular gene expression information of a biological sample provided in an embodiment of the present application;

[0313] FIG47 is a schematic diagram of the structure of a spatial omics single-cell data acquisition device provided in an embodiment of the present application;

[0314] FIG48 is a schematic diagram of the structure of another spatial omics single-cell data acquisition device provided in an embodiment of the present application;

[0315] Figure 49 is a schematic block diagram of an example electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0316] The following description of exemplary embodiments of the present application is made in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0317] The following describes the spatial omics single-cell data acquisition method, device, electronic device and storage medium of the embodiments of the present application with reference to the accompanying drawings.

[0318] FIG1 is a schematic diagram of a flow chart of a method for acquiring spatial omics single-cell data provided in an embodiment of the present application.

[0319] As shown in Figure 1, the method includes the following steps:

[0320] Step 101, obtaining a staining image of a biological sample and obtaining a gene expression visualization image of the biological sample;

[0321] Based on spatial resolved technology (SOP), biological samples are stained and photographed. The stained images are obtained by photographing the biological samples on the spatiotemporal chip under a microscope.

[0322] As a feasible method in the embodiment of the present application, staining can be achieved by but not limited to the following techniques, such as: Hematoxylin and Eosin Staining (HE), ssDNA and DAPI. Specifically, the embodiment of the present application does not limit this.

[0323] The obtained microscope photographs (stained images) are generally in tif format, including but not limited to 16-bit or 8-bit, and can be color images or grayscale images. Specifically, this embodiment of the application does not limit this.

[0324] The biological samples are spatially sequenced, and the gene expression matrix of the biological samples is obtained through standard processes such as sequencing sequence alignment analysis, and the gene expression matrix is ​​converted into a gene expression visualization image.

[0325] The spatiotemporal chip has regularly arranged sites for collecting genes from biological samples. Each site can capture the gene sequence of the biological sample. Gene sequencing can obtain the gene sequence or the number of bases at each site. The number of gene sequences at that site can then be used as the value of an element in the gene expression matrix, so that the elements of the gene expression matrix correspond to the sites of the spatiotemporal chip (for example, one element corresponds to one site, and the element in the first row and first column of the gene expression matrix corresponds to the site in the upper left corner of the spatiotemporal chip). By analogy, the gene expression matrix can be obtained, and then a gene expression visualization image can be drawn based on the gene expression matrix. The pixels in the gene expression visualization image correspond to the elements of the gene expression matrix, and the pixel values ​​of the pixels correspond to the values ​​of the elements. Therefore, the gene expression visualization image contains spatial information of the same biological sample.

[0326] For the implementation method of sequencing sequence comparison analysis, please refer to the methods in the relevant technology, and the embodiments of this application will not be described in detail here.

[0327] Obtaining a gene expression visualization image of the biological sample includes, but is not limited to, the following methods: performing spatial omics sequencing on the biological sample to obtain a gene expression matrix; and converting the gene expression matrix into a corresponding gene expression visualization image.

[0328] Step 102, registering the stained image with the gene expression visualization image to obtain a registered image;

[0329] In some embodiments, registration is achieved based on line features in the stitched stained image;

[0330] Alternatively, in some embodiments, accurate automatic registration at pixel-level precision is achieved through coordinate calculation using track points on the image and tissue contours as references;

[0331] Alternatively, in some embodiments, registration is achieved by splicing a tissue mask of the stained image. Specifically, the embodiments of the present application do not limit the registration method.

[0332] Step 103, performing cell segmentation on the registered image to obtain a cell mask image;

[0333] The registered images are fed into an image segmentation model to output a cell mask that includes the cell nucleus.

[0334] As a feasible method of an embodiment of the present application, before performing cell segmentation, tissue segmentation can also be performed first. After the tissue segmentation is completed, only the tissue area can be focused on without paying attention to the outside of the tissue, thereby improving the signal-to-noise ratio of the sample data.

[0335] Step 104, determining the cells to which background gene molecules of unknown cells belong in the gene expression visualization image according to the cell mask image, and marking the cells to obtain a spatial component molecular expression matrix with cell labels;

[0336] After the cell segmentation is completed in step 103, at least the cell nucleus is included, but there are still some background gene molecules that are not marked as a cell.

[0337] Reassign background genes at DNA nanoball (DNB) locations that are not assigned to cells. Centered around that location, select a specific area (e.g., 100 x 100 pixels) and fit a Gaussian mixture model to each of the cells contained within. Assuming there are N cells, there will be N probability values ​​for that location. The cell with the highest probability is selected as the cell assigned to that location. Repeat this process for each unassigned expression location. This secondary correction improves the signal-to-noise ratio and alleviates accuracy issues in cell segmentation and registration.

[0338] In practice, to prevent over-assignment, a probability threshold is set when selecting cells with the highest probability as belonging cells. This allows for cell attribution marking. When the probability is too low, the cells are not marked. There is no specific limit on the probability threshold.

[0339] Figure 1 illustrates the method for acquiring spatial omics single-cell data. Referring to Figure 2, Figure 2 is a schematic diagram of acquiring spatial omics single-cell data provided in an embodiment of the present application. The figure is based on a mouse brain slice as an example, where A corresponds to step 101, C corresponds to step 102, D corresponds to step 103, and E corresponds to step 104.

[0340] Acquire a stained image of a biological sample and obtain a gene expression visualization image of the biological sample; align the stained image with the gene expression visualization image to obtain a registered image; perform cell segmentation on the registered image to obtain a cell mask; determine the cells to which background gene molecules of unknown cells in the gene expression visualization image belong based on the cell mask, and label the cells to obtain a spatial component molecular expression matrix with cell labels. Compared with related technologies, the embodiments of the present application obtain high-precision spatial single-cell data through a processing flow of gene expression matrix diagram, splicing, alignment, segmentation, and correction.

[0341] As a feasible approach in the embodiment of the present application, when obtaining a stained image of a biological sample in step 101, the image may be obtained in, but not limited to, the following three ways:

[0342] Method 1: obtaining a stained image of the biological sample through a microscope;

[0343] Method 2: obtaining a stitched staining image of the biological sample by stitching together a plurality of sub-staining images through a microscope;

[0344] Method three: obtaining a plurality of sub-stained images of the biological sample through the microscope, and splicing the plurality of sub-stained images to obtain a spliced ​​stained image.

[0345] Methods 1 and 2 are the results of stitching performed under a microscope. Method 1 directly obtains a large image (stained image) through a microscope, while Method 2 obtains multiple small images (sub-stained images) and then stitches them together using a microscope to obtain a stained image. Method 3 is the stitching result of stained images obtained by stitching using a preset algorithm when the results of microscope stitching do not meet the requirements. The specific methods for obtaining stained images are not limited in the embodiments of this application.

[0346] In some embodiments, as shown in FIG3 , FIG3 is a flow chart of a method for obtaining a stained image of a biological sample provided in an embodiment of the present application. The method includes:

[0347] Step 1011 : Acquire multiple sub-stained images of the biological sample through a microscope; and perform a first stitching on the multiple sub-stained images of the biological sample to obtain a first stained image.

[0348] The stitching can be performed in the following ways, as shown in FIG4 , but is not limited to:

[0349] Step 10111, respectively determining overlapping areas of the plurality of sub-stained images and the template image;

[0350] In some examples, determining the overlapping area of ​​the sub-stained image and the template image may include: determining a matching direction between the sub-stained image and the template image; and selecting adjacent areas of the same size in the sub-stained image and the template image in the determined matching direction as the overlapping area.

[0351] For example, the matching direction (or stitching direction) can be determined based on the direction of movement of the camera when capturing the two images. If the sub-dyed image and the template image are two images captured sequentially while the camera moves horizontally, the matching direction between the two images can be determined as a horizontal direction. Of course, other available methods can also be used to determine the relative positional relationship between the two images and thereby determine the matching direction, which will not be elaborated here.

[0352] In some examples, the overlapping region is selected such that a ratio of actual overlapping portions in the sub-stained image and the template image in the overlapping region is greater than a predetermined threshold.

[0353] In some examples, the predetermined threshold is determined to be 10%, or 30%, etc. It should be noted that this value is only used as an example. In other examples, the predetermined threshold may also take other larger or smaller values.

[0354] Please continue to refer to FIG. 2 , this step corresponds to step B in FIG. 2 .

[0355] Step 10112: cropping the portions of the multiple sub-stained images and the template image that are determined to be overlapping areas into a plurality of stained image overlapping sub-images and a plurality of template image overlapping sub-images of the same number, so as to pair them into a plurality of overlapping sub-image pairs;

[0356] In some examples, the operation may include: performing noise reduction and / or windowing processing on the portions of the sub-stained image and the template image that are determined to be overlapping areas; merging the sub-stained image and the template image that have been subjected to noise reduction and / or windowing processing in a direction perpendicular to the matching direction, and resampling the portions of the sub-stained image and the template image that have been subjected to noise reduction and / or windowing processing with the same image cropping size to obtain the multiple sub-stained image overlapping sub-images and the multiple template image overlapping sub-images; and sequentially pairing the multiple sub-stained image overlapping sub-images and the multiple template image overlapping sub-images one by one in a direction perpendicular to the matching direction to form the multiple overlapping sub-image pairs.

[0357] The noise reduction process in the above operation can be performed, for example, using Gaussian filtering, but other noise reduction methods used in the field of image processing can also be used. Windowing can be used to obtain an image that suppresses spectral leakage. Examples of windows used include, but are not limited to, Hanning windows and Hamming windows.

[0358] In some examples of this application, a sliding window approach can be used to perform the aforementioned resampling. The sliding window approach can set a certain overlap region, and multiple stacked sub-images correspond to ensure that the cropped regions of the source image and template image are located in the overlap region and at the same coordinate system position. Any available approach can be used to implement resampling, such as using object detection to crop regions with more features, and is not limited to the aforementioned sliding window approach.

[0359] After obtaining multiple overlapping subimage pairs, the method shown in FIG1 may optionally further include removing, from the multiple overlapping subimage pairs, those overlapping subimage pairs that have fewer features than a specified standard and / or greater noise than a specified standard. For example, gradient detection may be used to filter out subimage pairs with fewer features and / or more noise. Gradient detection methods include, but are not limited to, Sobel and Canny methods. Furthermore, any other method capable of detecting the number of features and / or noise in an image can be used herein, and is not limited to gradient detection methods.

[0360] Step 10113, performing frequency domain calculation on the multiple overlapping sub-image pairs to obtain the offset between each sub-stained image and the template image; and

[0361] Optionally, the overlapping sub-image pairs subjected to frequency domain superposition resonance are the overlapping sub-image pairs obtained after removing sub-image pairs with fewer features and / or more noise in the above optional step.

[0362] In some examples, this step can be performed using a Dirac function. For example, it includes: stacking the cross-power spectra of the multiple overlapping sub-image pairs, and using the Dirac function to obtain an offset from the stacked cross-power spectra. Specifically, in some examples, each overlapping sub-image in the multiple overlapping sub-image pairs can be transformed from the position coordinate system to the frequency domain. This transformation can be accomplished, for example, by a two-dimensional image discrete Fourier transform, or by any other method that can achieve domain transformation, which will not be repeated here. Then, the cross-power spectrum corresponding to each overlapping sub-image pair in the multiple overlapping sub-image pairs can be calculated; an inverse discrete Fourier transform is performed on the cross-power spectrum corresponding to each overlapping sub-image pair obtained to obtain a Dirac function representation of the overlapping area; and the coordinates of the maximum peak of the obtained Dirac function representation in the overlapping area are determined as the offset.

[0363] In some examples, before accumulating the cross-power spectrum corresponding to each obtained overlapping sub-image pair, the features in the overlapping sub-image pair can be improved for each obtained overlapping sub-image pair to make them more obvious. For example, the overlapping sub-image pair can be transformed into a position coordinate system, and the overlapping sub-images transformed into the position coordinate system can be windowed to enhance the features in the overlapping sub-images. The window function used here can be a Hanning window or a Hamming window, or any other function that can enhance the features in the image. Enhancing the features in the overlapping sub-images can effectively improve the success rate of image registration of images with unclear features. Here, the method of enhancing image features used may not be limited to windowing, but may be any other method that can enhance image features.

[0364] In some embodiments, performing an inverse discrete Fourier transform on the cross-power spectrum corresponding to each obtained overlapping sub-image pair to obtain the Dirac function representation of the overlapping region may include: performing an inverse discrete Fourier transform on the cross-power spectrum corresponding to each obtained overlapping sub-image pair to obtain a stacked Dirac function representation; and performing weighted accumulation on the obtained stacked Dirac function representations to obtain the Dirac function representation of the overlapping region. In this scheme, overlapping sub-image pairs with more obvious features may be assigned a larger weighting coefficient. For example, the weighting coefficient of each overlapping sub-image pair may depend on the energy spectrum of the overlapping sub-image pair (e.g., the energy intensity of the maximum peak of the Dirac function) and / or the number of matches of the feature points of the overlapping sub-image pair and / or the image clarity of the overlapping sub-image pair, or may be based on any other metric that can characterize whether the overlapping sub-image pair has more obvious features.

[0365] Step 10114: Shift each of the sub-dyed images toward the direction of the template image by the offset amount, and splice them together with the template image.

[0366] Step 1012: determining whether to perform a second stitching on the plurality of sub-stained images of the biological sample according to the image quality of the first stained image;

[0367] In some embodiments, the image quality of the first staining image needs to be set according to the requirements for acquiring spatial omics single-cell data, which is not limited in the embodiments of the present application.

[0368] If not, execute step 1013; if so, execute step 1014.

[0369] Step 1013, determining the first stained image as the stained image;

[0370] Step 1014 : performing a second stitching on the multiple sub-stained images of the biological sample to obtain the stained image.

[0371] The specific implementation process of the second splicing method can be found in the relevant description of FIG4 , so it will not be described again here.

[0372] As shown in FIG5 , FIG5 is a flow chart of another method for splicing multiple sub-stained images to obtain a spliced ​​stained image provided by an embodiment of the present application, including:

[0373] Step 10115 , obtaining the plurality of sub-stained images and template information about the trajectory lines or trajectory points of the biological sample;

[0374] During the sequencing process, several images can be taken sequentially for a portion of a single biological sample. Each field of view image can have several track lines distributed horizontally and vertically, with the intersection of track lines being referred to as track crosses. Track lines and track points can be converted to each other; a track point can be obtained based on two intersecting track lines; or a track line can be obtained based on collinear track points. The template information may include index numbers for the multiple track lines on the biological sample. As known to those skilled in the art, the distribution pattern of track lines on a biological sample is already defined at the beginning of the biological sample design. For example, assuming there are 10 track lines in the vertical direction of a biological sample, the spacing between them can be, for example, 1, 2, 3, 4, 5, 4, 3, 2, and 1, respectively. Assuming the position of the first track line is the starting point (coordinate 0), the horizontal axis (x-axis) coordinates of the above 10 track lines are 0, 1, 3, 6, 10, 15, 19, 22, 24, and 25, respectively. The index numbers in the template information of the biological sample can be associated with the arrangement of the trajectory lines in the biological sample. The index numbers of the above 10 trajectory lines can be artificially specified as 0, 1, 2, 3, 4, 5, 6, 7, 8, and 0 respectively.

[0375] In addition, the template information of the acquired biological sample may further include periodicity. For example, for the 10 track lines in the vertical direction of the biological sample, they may be extended to the left and right by one or more periods respectively. For example, the spacing between the trajectory lines formed by expanding one cycle on the left and right is 1, 2, 3, 4, 5, 4, 3, 2, 1, 1, 2, 3, 4, 5, 4, 3, 2, 1, 1, 2, 3, 4, 5, 4, 3, 2, 1, 1, 2, 3, 4, 5, 4, 3, 2, 1. The horizontal axis (x-axis) coordinates of the 28 expanded trajectory lines are -25, -24, -22, -19, -15, -10, -6, -3, -1, 0, 1, 3, 6, 10, 15, 19, 22, 24, 25, 26, 28, 31, 35, 40, 44, 47, 49, 50, and their corresponding index numbers are 0, 1, 2, 3, 4, 5, 6, 7, 8, 0, 1, 2, 3, 4, 5, 6, 7, 8, 0.

[0376] It is worth noting that, generally speaking, the index number of the biological sample corresponding to a certain trajectory line is unique, and a certain index number on the biological sample may correspond to multiple trajectory lines. This is determined by the periodicity of the trajectory line design on the biological sample.

[0377] The microscope can capture images of biological samples, for example, by scanning the biological sample row by row or column by column. The spatial distribution of multiple images after scanning can be as shown in Figure 6. The multiple images can have, for example, n+1 columns and m+1 rows, including the image (r0, c0) from the first row and first column to the image (rm, cn) from the last row and last column. For the image (rm-1, cn-1) shown in Figure 3, the image can have up to 8 adjacent images. In addition, the microscope can also record the camera position information at the time of shooting when shooting an image. Subsequently, based on the camera position information recorded for any image, the corresponding position of the image and its adjacent images can be determined for the image.

[0378] FIG7 is a schematic diagram of a pre-stitching result of an image stitching method according to an embodiment of the present invention.

[0379] Step 10116: pre-stitching the multiple sub-stained images to obtain pre-stitched coordinates of the multiple sub-stained images;

[0380] Multiple stained images can be scanned, and the images of the left and right neighbors of the current stained image can be determined using, for example, a 2-nearest neighbor method based on the scanning order or the camera position information when the images were captured. The offset and overlap between the stained image and the left and right adjacent images can then be determined. During the scanning process, a fast Fourier transform (FFT) or a scale-invariant feature transform (SIFT) can be preferably used in conjunction with template information of the biological sample to obtain the offset and overlap between adjacent stained images. Based on the offset and overlap between adjacent stained images obtained during the image scanning process, pre-joined coordinates of the multiple stained images can be obtained.

[0381] FIG8a is a schematic diagram showing the positional relationship between two adjacent dyed images in the row direction obtained by scanning in the horizontal direction during pre-splicing; FIG8b is a schematic diagram showing the positional relationship between two adjacent dyed images in the column direction obtained by scanning in the vertical direction during pre-splicing.

[0382] Specifically, Figures 8a and 8b respectively show the deviation and overlap between adjacent stained images obtained after scanning the stained images in the horizontal and vertical directions. For horizontal scanning (refer to Figure 8a), the distance of horizontal overlap between two adjacent images in the row direction is called the overlap in the horizontal direction, and the distance of deviation in the vertical direction is called the offset in the vertical direction. Similarly, for vertical scanning (refer to Figure 8b), the distance of vertical overlap between two adjacent stained images in the column direction is called the overlap in the vertical direction, and the distance of deviation in the horizontal direction is called the offset in the horizontal direction. After the horizontal and vertical scanning are completed, the deviation matrix A and overlap matrix B of the multiple stained images in the corresponding directions can be obtained. Using the deviation matrix A and overlap matrix B, the coordinates of each stained image in space can be mapped, thereby completing the pre-stitched image of the multiple stained images shown in Figure 7.

[0383] However, only pre-stitching the stained image cannot achieve pixel-level accuracy, so it is necessary to further accurately adjust the coordinates of the stained image in steps S10117 and S10118 on this basis.

[0384] Step 10117: selecting at least one first reference image from the plurality of sub-stained images according to the features of the trajectory lines or trajectory points, and deriving a global template according to the template information of the biological sample and the pre-spliced ​​coordinates of the first reference image; and

[0385] The first reference image (or reference image) selected can be the image with the most obvious trajectory features or the image with the most trajectory points among the multiple stained images. The first reference image is, for example, the image (rm-1, cn-1) highlighted in Figures 6 and 7. Trajectory line detection is then performed on the selected image (rm-1, cn-1), and the spacing between the detected adjacent trajectory lines is matched with the spacing between the trajectory lines specified during the chip design mentioned above, thereby determining the index number corresponding to the trajectory line in the selected first reference image. In other words, the index number corresponding to each trajectory line in the first reference image can be determined based on the distance between adjacent trajectory lines in the first reference image. Then, a global template can be derived based on the index number of the first reference image and the pre-spliced ​​coordinates of the first reference image.

[0386] In one embodiment, since tissue slices on a biological sample will cover the trajectory line, and some trajectory points may be revealed at the location of the tissue cavity, the probability of complete trajectory points appearing in an image is higher than that of complete trajectory lines. Therefore, it is also preferable to select the image with the most trajectory points as the first reference image.

[0387] In one embodiment, an index number corresponding to each trajectory line in the first reference image is determined according to a distance between adjacent trajectory lines in the first reference image, and a global template is derived based on the index number of the first reference image and the pre-stitched coordinates of the first reference image.

[0388] FIG9 is a schematic diagram illustrating a global template for stitching biological samples according to an embodiment of the present invention; FIG10 is a schematic diagram illustrating a construction process of a global template for stitching biological samples according to an embodiment of the present invention.

[0389] Referring to the construction example of the global template shown in Figure 10, based on the position coordinates of the selected first reference image (rm-1, cn-1) in the pre-splicing stage and the index number of the trajectory line, a global template (or global reference system) based on the first reference image (rm-1, cn-1) can be obtained by performing left-right bidirectional periodic expansion in the horizontal direction and up-down bidirectional periodic expansion in the vertical direction.

[0390] The adjustment of the coordinates of the second reference image other than the selected first reference image (rm-1, cn-1) can be divided into two categories, namely, coordinate adjustment of the second reference image containing trajectory points and coordinate adjustment of the second reference image not containing trajectory points.

[0391] Coordinate adjustment is performed on the second reference image containing the trajectory points by adding the pre-stitched coordinates of each second reference image containing the trajectory points to the unique offset of the second reference image containing the trajectory points. The offset of the second reference image is the vector formed by the trajectory point on the second reference image pointing to the template point in the global template corresponding to the trajectory point on the second reference image. The offset, indicated by the arrow in the partially enlarged schematic diagram of FIG9 , represents the vector from the trajectory point on the biological sample to the template line trajectory point on the global template derived from the trajectory point.

[0392] Step 10118, for the second reference image other than the first reference image in the multiple sub-stained images, calculate the offset between the pre-stitched coordinates of the second reference image and the corresponding template coordinates of the second reference image in the global template, and adjust the coordinates of the second reference image based on the offset to stitch and generate the stitched stained image.

[0393] Obtaining the respective offsets of each trajectory point in each second reference image containing the trajectory point, and selecting, for each second reference image, an offset having a median angle or a median distance among the offsets as the unique offset of the second reference image; or,

[0394] Obtain the respective offsets of each trajectory point in each second reference image containing the trajectory point, sort the offsets according to vector lengths of the offsets, and select the offset at a preset sorting position as the unique offset of the second reference image.

[0395] The adjusted coordinates of the second reference image containing the trajectory point may be obtained by adding the unique offset of each of the second reference images containing the trajectory point to the respective unique offsets of the second reference images containing the trajectory point.

[0396] The coordinate adjustment of the second reference image that does not contain the track point can be performed using a neighbor adjustment method. The neighbor adjustment method includes: for each of the second reference images that do not contain the track point, the coordinate adjustment of the second reference image that does not contain the track point is performed based on the unique offset of the second reference image that contains the track point that is closest to the second reference image that does not contain the track point, and the deviation and overlap between the second reference image that does not contain the track point and the closest second reference image that contains the track point.

[0397] Specifically, due to issues such as tissue coverage, chip feature contamination, and microscope camera quality, some images may not contain track points. In this case, the second reference image without track points can be aligned based on the deviation and overlap between the second reference image without track points and the adjacent or closest adjusted second reference image containing track points. It should be noted that the closest distance metric is Manhattan distance, not Euclidean distance.

[0398] The coordinate adjustment of the second reference image further includes performing seam fusion processing on adjacent second reference images, for example, a linear weighted fusion method may be used.

[0399] In some embodiments, as shown in FIG11 , FIG11 is a schematic flow chart of a method for registering a staining image with a gene expression visualization image provided in an embodiment of the present application, including:

[0400] Step 1021: Obtain a trajectory line in the staining image, and rotate the staining image based on an angle between the trajectory line and the horizontal direction; wherein the staining image and the gene expression visualization image include multiple trajectory lines and / or multiple trajectory points, each trajectory point is formed by the intersection of two trajectory lines, and each trajectory line in the multiple trajectory lines has a corresponding index number;

[0401] Since the stained image is obtained by capturing a microscope image, the trajectory lines in the microscope image are not necessarily horizontal or vertical. Therefore, the first trajectory line in the microscope image can be selected as a reference, the angle between the first trajectory line and the horizontal direction can be calculated, and the microscope image can be rotated by the aforementioned angle to be horizontal. See Figure 12, which shows a schematic diagram of a stained image. As can be seen from Figure 12, the stained image is not horizontal; the first trajectory line is at an angle with the horizontal direction. Therefore, the stained image can be rotated to the horizontal direction after calculating the aforementioned angle.

[0402] Because gene expression visualization images are generated from a gene expression matrix and appear as numerous discrete points, they are less capable of detecting expression clustering features or edge features of biological samples. In a preferred implementation, tissue segmentation can be performed on the gene expression visualization image to obtain the biological sample outline as indicated by gene expression. Convolution enhancement is then performed on the approximate region containing the sample outline to enhance the edge features of the gene expression visualization image, facilitating subsequent registration of the microscope image with the gene expression visualization image.

[0403] Step 1022: zoom in or out the stained image to obtain a third reference image, wherein the third reference image has the same scale as the gene expression visualization image;

[0404] When the staining image is captured by a microscope, the gene expression visualization image can serve as a reference image. However, the scale ratios of the microscope staining image and the gene expression visualization image may differ. To align the staining image with the gene expression visualization image, a third reference image of the same size as the gene expression visualization image needs to be obtained by enlarging or reducing the staining image.

[0405] In one possible implementation, the scaling ratio of the staining image can be determined based on the distance between the staining image and the trajectory lines with corresponding index numbers in the gene expression visualization image. Specifically, a second trajectory line is obtained from the staining image, and a first spacing is determined based on the first and second trajectory lines of the staining image. A third and fourth trajectory lines are obtained from the gene expression visualization image, wherein the index number of the third trajectory line is the same as the index number of the first trajectory line, and the index number of the fourth trajectory line is the same as the index number of the second trajectory line, and a second spacing is determined based on the third and fourth trajectory lines. Then, based on the first and second spacings, a scaling ratio value between the staining image and the gene expression visualization image is determined, and the staining image is enlarged or reduced according to the aforementioned ratio value to obtain a third reference image with the same scale as the gene expression visualization image. The first and second trajectory lines are parallel; and the third and fourth trajectory lines are parallel.

[0406] When determining the first spacing based on the first and second trajectory lines, the calculation can be performed using trajectory points on the first and second trajectory lines. When two trajectory points have the same horizontal coordinate or vertical coordinate, the difference between the corresponding vertical coordinates or horizontal coordinates can be used as the first spacing. Similarly, the second spacing between the third and fourth trajectory lines can be determined based on the same principle. It should be noted that the method for calculating the scaling ratio of the staining image and the gene expression visualization image provided in the above embodiment is merely illustrative and is not limited to the above implementation.

[0407] Step 1023, determining a target offset based on the gene expression visualization image and the third reference image;

[0408] When the staining image is scaled to a third reference image with the same scale as the gene expression visualization image, the target offset can be determined based on the gene expression visualization image and the third reference image. The above embodiment shows that after the original staining image is rotated to the horizontal direction, there may be a 90-degree angle difference between the staining image and the gene expression visualization image. In order to further improve the accuracy of image processing, after the staining image is scaled to a third reference image with the same scale as the gene expression visualization image, the third reference image can be rotated according to a preset rotation angle, and the corresponding target offset can be determined at each rotation angle. This will be explained below in conjunction with a specific application scenario.

[0409] As shown in Figure 13, S represents the third reference image, V represents the gene expression visualization image, the black dot represents the center of gravity of the third reference image, and the gray dot represents the center of gravity of the gene expression visualization image. The four regions in Figure 13 represent the original third reference image and the images after the third reference image is rotated 90 degrees, 180 degrees, and 270 degrees clockwise, respectively. After the third reference image is rotated, the origin coordinates of the third reference image and the origin coordinates of the gene expression visualization image can be aligned. The target offset can then be determined based on the origin-aligned third reference image and the gene expression visualization image.

[0410] For any preset rotation angle, in order to process the image more conveniently and accurately, this embodiment provides a preferred implementation method, that is, first, the third reference image can be preliminarily positioned to a position roughly the same as the gene expression visualization image based on the centroid coordinates of the third reference image and the gene expression visualization image, and then the target offset is calculated based on the moved third reference image for precise positioning. In a specific implementation, the stained image is enlarged or reduced to obtain a fifth reference image, wherein the fifth reference image has the same proportion as the gene expression visualization image. Then, the first centroid coordinates of the fifth reference image and the second centroid coordinates of the gene expression visualization image are calculated respectively, the centroid offset is determined based on the first centroid coordinates and the second centroid coordinates, and the fifth reference image is moved based on the centroid offset to obtain a fourth reference image, that is, a preliminarily positioned fourth reference image is obtained. Among them, when calculating the centroid coordinates of the fifth reference image and the gene expression visualization image, a density centroid algorithm based on pixel grayscale values ​​can be used for implementation, and this embodiment does not limit this.

[0411] It should be noted that the image movement during the intermediate process can be implemented on a temporary image that copies the stained image. That is, the stained image is not moved for registration. After the calculation is completed to obtain the final target offset, the stained image is moved once again and registered with the gene expression visualization image.

[0412] After obtaining a preliminarily determined fourth reference image, the target offset can be determined based on the fourth reference image and the gene expression visualization image. In practical applications, the trajectory lines inherent to the spatiotemporal chip may have multiple periods. Therefore, the microscope image and gene expression visualization image obtained based on the spatiotemporal chip may also have multiple periods, that is, the fourth reference image and the gene expression visualization image have multiple periods. Each period has a corresponding sub-image, and each sub-image has the same number of trajectory lines and trajectory points, and the index numbers of the trajectory lines between the sub-images correspond one-to-one. As shown in Figures 14a and 14b, Figure 14a shows a third reference image with multiple periods, and Figure 14b shows a gene expression visualization image with multiple periods. In Figure 14a, only the sub-images of the first and second periods are shown. It can be seen that the sub-images of each period have the same number of index lines and index points, and each index line corresponds to an index number. Figure 14b also shows the sub-images corresponding to the first and second periods. The schematic diagrams of the third reference image and the gene expression visualization image provided in the above embodiment are only one possible implementation method and do not limit the specific form of the image. Therefore, when determining the target offset based on the fourth reference image and the gene expression visualization image, the effect of the cycle needs to be considered.

[0413] In a specific implementation, a first offset can be first determined based on the fourth reference image and the gene expression visualization image. This first offset is the undetermined offset between the fourth reference image and the gene expression visualization image. Then, with the first offset as the center, multiple second offsets corresponding to the first offset within a preset period are determined, i.e., each period within the preset period corresponds to a second offset. For any second offset, the fourth reference image is shifted based on the second offset to obtain a sixth reference image. A first similarity is then calculated between the sixth reference image and the gene expression visualization image, thereby obtaining the first similarity corresponding to each period within the preset period. Since the similarity determined above is obtained at any one rotation angle, the above steps need to be repeated at other preset rotation angles to obtain multiple first similarities corresponding to each rotation angle. Based on the multiple first similarities determined for all rotation angles and all periods, a first maximum similarity is determined, and the sixth reference image corresponding to this first maximum similarity is determined as the third reference image. The second offset corresponding to the third reference image can then be determined as the target offset.

[0414] Among them, when determining the first offset based on the fourth reference image and the gene expression visualization image, one possible implementation method is to first obtain multiple groups of trajectory point sets, where each group of trajectory point sets includes a first trajectory point and a second trajectory point, the first trajectory point is located in the fourth reference image, the second trajectory point is located in the gene expression visualization image, and the first trajectory point and the second trajectory point have the same index identifier. Since a trajectory point is formed by two intersecting trajectory lines, each trajectory line has an index number, the index identifier of the trajectory point can be composed of the index numbers of the two intersecting trajectory lines of the trajectory point. In this embodiment, since there are multiple trajectory points in both the fourth reference image and the gene expression visualization image, it is necessary to obtain trajectory points with a corresponding relationship to form a group of trajectory point sets, that is, trajectory points with the same index number. For any group of trajectory point sets in the multiple groups of trajectory point sets, the fourth offset can be determined based on the coordinates of the first trajectory point and the coordinates of the second trajectory point, and then the first offset is determined based on the multiple fourth offsets.

[0415] To facilitate a clearer understanding of this solution, before introducing the implementation of the third offset, the implementation of the fourth offset will be described first.

[0416] In a specific implementation, since a gene expression visualization image has multiple cycles, i.e., multiple sub-images, the first and second trajectory points obtained with the same index identifier may not belong to the corresponding cycle, resulting in the possibility of multiple second trajectory points being determined. When determining the fourth offset based on the coordinates of the first and second trajectory points, any first trajectory point in the fourth reference image can be selected and, based on the index identifier of the first trajectory point, multiple second trajectory points with the same identifier can be determined. To obtain the second trajectory point corresponding to the first trajectory point, the Euclidean distance between the first trajectory point and each second trajectory point can be calculated. After calculating multiple Euclidean distances, the second trajectory point corresponding to the minimum Euclidean distance is obtained, which is the second trajectory point corresponding to the first trajectory point. The fourth offset is then determined based on the coordinate difference between the second trajectory point and the first trajectory point corresponding to the minimum Euclidean distance. Since the fourth reference image contains multiple trajectory points, i.e., multiple sets of trajectory point sets, it is necessary to traverse all trajectory point sets and determine the fourth offset corresponding to each set of trajectory point sets using the method for determining the basic offset described above. In one possible implementation, among the multiple fourth offsets determined, fourth offsets that exceed a preset range may be first excluded, and then a target offset may be determined based on the remaining fourth offsets. For example, the median of the multiple fourth offsets may be selected as the target offset, or the average of all fourth offsets may be calculated as the target offset, to maximize the accuracy of the offset determination. It should be noted that this embodiment does not limit the specific method for determining the target offset based on the fourth offset.

[0417] The image movement within a preset period will be described below with reference to a specific application scenario.

[0418] Referring to FIG. 15 , FIG. 15 is a schematic diagram of a preset period determined with the first offset as the center.

[0419] In one possible implementation, the gene expression visualization image is used as a benchmark. As shown in FIG15 , the middle gray area represents the period where the second trajectory point is located. After determining the first offset based on the first and second trajectory points, a 5*5 period is selected with the first offset (the period to which the corresponding second trajectory point belongs) as the center, and the second offset corresponding to each period is determined. The difference between the second offsets corresponding to adjacent periods is the width of the sub-image. Similarly, the second offset corresponding to each period can be determined.

[0420] Then, the fourth reference image can be moved by the second offset corresponding to any period to obtain a sixth reference image, and the similarity between the sixth reference image and the gene expression visualization image can be calculated. When calculating the similarity, this embodiment provides a possible implementation method, namely, performing a frequency domain transform on the sixth reference image to obtain a first frequency domain image, performing a frequency domain transform on the gene expression visualization image to obtain a second frequency domain image, and calculating the Hamming distance between the first frequency domain image and the second frequency domain image as the similarity. The smaller the Hamming distance, the greater the similarity between the first frequency domain image and the second frequency domain image. After determining the Hamming distances corresponding to all preset rotation angles and all preset periods, the minimum Hamming distance is used as the first maximum similarity.

[0421] In this case, a frequency domain image can be obtained using a discrete cosine transform, that is, a discrete cosine transform is performed on the moved third reference image to obtain a first frequency domain image, and a discrete cosine transform is performed on the gene expression visualization image to obtain a second frequency domain image. Then, a low-frequency frequency domain image is intercepted from the first frequency domain image. The low-frequency frequency domain image is used as a reference to calculate the pixel mean of the low-frequency frequency domain image. In the low-frequency frequency domain image, the pixel values ​​of pixels with pixel values ​​higher than the pixel mean are set to 1, the pixel values ​​of pixels with pixel values ​​lower than the pixel mean are set to 0, and the pixel values ​​of pixels with pixel values ​​equal to the pixel mean are set to 0 or 1. Then, a hash fingerprint is obtained based on the reset low-frequency frequency domain image. Following the same execution steps, a hash fingerprint of the second frequency domain image is obtained. Then, based on the hash fingerprint of the first frequency domain image and the hash fingerprint of the second frequency domain image, the number of different bits in the two hash fingerprints is determined, which is used as the Hamming distance between the first frequency domain image and the second frequency domain image. It should be noted that the method for calculating similarity in the above embodiment is only an exemplary implementation method and is not limited to the above implementation method. Other achievable methods for calculating image similarity are also within the scope of protection of this application.

[0422] In the above-described embodiment of the present application, the fourth reference image can first be preliminarily shifted and positioned according to its center of gravity offset relative to the gene expression visualization image. In another possible implementation, when determining the third reference image, the fourth reference image is not preliminarily positioned. Instead, for any preset rotation angle, the fourth reference image is shifted by each period within a preset cycle to obtain a seventh reference image corresponding to each period. For example, the third center of gravity coordinate of the fourth reference image can be first calculated. With the period corresponding to the third center of gravity coordinate as the center, the fourth reference image is shifted by each period within the preset cycle to obtain a seventh reference image. A second similarity between the seventh reference image and the gene expression visualization image is then calculated to obtain a second similarity corresponding to each period within the preset cycle. The above steps are then repeated for other preset rotation angles to obtain the second similarity corresponding to each preset rotation angle and each period. A second maximum similarity is determined based on these multiple second similarities. The seventh reference image corresponding to the second maximum similarity is used as the third reference image, and the target offset is then determined based on this third reference image and the gene expression visualization image.

[0423] Specifically, first, a plurality of third offsets can be determined based on the gene expression visualization image and the third reference image, and the target offset can be determined based on the plurality of third offsets. The calculation principle of the third offset is the same as the principle of calculating the fourth offset in the above embodiment. That is, a plurality of groups of trajectory point sets are obtained, wherein each group of trajectory point sets includes a third trajectory point and a fourth trajectory point, the third trajectory point is located in the third reference image, the fourth trajectory point is located in the gene expression visualization image, and the third trajectory point and the fourth trajectory point have the same index identifier. For any group of trajectory point sets, the third offset is determined based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point. The specific implementation method of determining the third offset based on the coordinates of the third trajectory point and the coordinates of the fourth trajectory point can refer to the specific implementation method of determining the fourth offset based on the coordinates of the first trajectory point and the coordinates of the second trajectory point, which will not be repeated here.

[0424] Similarly, after determining the third offsets corresponding to multiple sets of trajectory point sets, you can first exclude the third offsets that exceed the preset range, and then select the median as the target offset among the multiple third offsets that meet the preset range, or calculate the average of multiple third offsets as the target offset, so as to maximize the accuracy of determining the offset.

[0425] Step 1024: Move the third reference image based on the target offset to obtain the registered image.

[0426] After determining the target offset based on the method provided in the above embodiment, a preset rotation angle corresponding to the target offset is determined, and the third reference image is moved based on the preset rotation angle and the target offset.

[0427] It should be noted that the image movement in the intermediate process is all implemented on a temporary image that copies the stained image. That is, the stained image is not moved for registration. After the calculation is completed to obtain the final target offset, the stained image is moved once again and aligned with the gene expression visualization image.

[0428] In the image processing method provided in the embodiment of the present application, the staining image and the gene expression visualization image can be aligned based on the image coordinates with pixel-level precision, thereby improving the accuracy of image processing and facilitating subsequent research and development and analysis using the aligned images.

[0429] When the third reference image and the gene expression visualization image have multiple periods, since the offset corresponding to the preset period is derived from the calculated offset using the theoretical period width, errors may exist in the actual image. To correct the errors in theoretical reasoning and further improve the accuracy of image processing, in a preferred implementation, after the third reference image is moved based on the target offset, the corrected offset can be reused based on the moved third reference image and the gene expression visualization image to determine the final offset. The third reference image is then further moved based on the determined final offset, and the third reference image is aligned with the gene expression visualization image to improve the accuracy of the alignment. For example, based on the moved third reference image and the gene expression visualization image, multiple sets of trajectory point sets are obtained, wherein each set of trajectory point sets includes a fifth trajectory point and a sixth trajectory point, the fifth trajectory point being located in the third reference image and the sixth trajectory point being located in the gene expression visualization image, and the fifth trajectory point and the sixth trajectory point having the same index identifier. For any set of trajectory point sets, a fifth offset is determined based on the coordinates of the fifth trajectory point and the coordinates of the sixth trajectory point. After determining the fifth offset corresponding to each of the multiple sets of trajectory point sets, a final offset is determined based on the multiple fifth offsets. The implementation of determining the fifth offset and determining the final offset based on the multiple fifth offsets can be found in the above embodiment and will not be repeated here.

[0430] As a feasible method of an embodiment of the present application, after the step of magnifying or reducing the stained image to obtain a third reference image, the method further includes: determining the similarity between the third reference image and the gene expression visualization image; and using the position corresponding to the maximum similarity as the coarse registration result. This similarity is used as the coarse registration, and the method shown in Figure 11 is used as the fine registration.

[0431] In some embodiments, as shown in FIG16 , FIG16 is a schematic flow chart of a method for registering a staining image with a gene expression visualization image provided in an embodiment of the present application, including:

[0432] Step 1025: pre-process the stained image to obtain a candidate image set;

[0433] Before registering the staining image with the gene expression visualization image, the staining image is actually taken by the microscope and is affected by external factors. There are certain differences between the staining image and the gene expression visualization image. For example, the staining image is not in a horizontal position. Therefore, in order to facilitate subsequent image registration, the staining image can be preprocessed first.

[0434] Specifically, a first trajectory and a second trajectory of the staining image are obtained, and the first and second trajectory lines can be randomly determined. The staining image is then rotated based on the angle between the first trajectory line and the horizontal direction, that is, the staining image is rotated to a horizontal position. A first spacing between the two trajectory lines is determined based on the first and second trajectory lines. A third trajectory line and a fourth trajectory line are obtained from the gene expression visualization image, wherein the index number of the third trajectory line is the same as the index number of the first trajectory line, and the index number of the fourth trajectory line is the same as the index number of the second trajectory line. A second spacing is determined based on the spacing between the third and fourth trajectory lines, and the rotated staining image is adjusted based on the ratio of the first spacing to the second spacing to obtain a candidate image. In other words, the spacing between the trajectory lines of the staining image and the gene expression visualization image may have different scaling ratios. To achieve accurate image registration, the staining image can be scaled to the same scale as the gene expression visualization image. When determining the first spacing between the two trajectory lines based on the first and second trajectory lines, two trajectory points with the same horizontal coordinate in the first and second trajectory lines can be obtained. The difference between the vertical coordinates of the two trajectory lines is the first spacing. Alternatively, two track points with the same vertical coordinate are obtained from the first track line and the second track line, and the difference between the horizontal coordinates of the two track lines is the first distance.

[0435] After acquiring the candidate image, since the candidate image is currently horizontal but may differ from the gene expression visualization image by a multiple of 90°, multiple rotation angles can be determined. The candidate image is rotated based on each rotation angle, thereby forming a candidate image set with the rotated images corresponding to each rotation angle. For example, the multiple rotation angles can be 90°, 180°, and 270°. After rotation, any image in the candidate image set still has the same index number as the stained image.

[0436] Step 1026, determining a binary image set corresponding to the candidate image set;

[0437] A binary image is an image binarization, which is to set the grayscale value of each pixel on the image to 0 or 255, so that the entire image presents a clear black and white effect, which is convenient for subsequent image registration. Optionally, a binary image set can be determined in the following way: for any fourth image in the candidate image set, segmentation is performed based on the tissue contour of the fourth image to obtain a tissue segmentation image. When performing tissue segmentation, it can be implemented based on a tissue segmentation algorithm, or it can be obtained by manually circling the tissue contour using a tissue segmentation tool. The tissue segmentation image is then binarized to obtain an initial binary image. For any of the multiple boundaries of the initial binary image, the boundary point closest to the boundary in the tissue contour is determined, and in the trajectory line parallel to the boundary, the boundary trajectory line closest to the boundary point is determined, and it is ensured that the boundary trajectory line does not pass through the tissue contour, thereby determining the target binary image based on the boundary trajectory line corresponding to each boundary. The target binary images corresponding to the multiple fourth images in the candidate image set constitute a binary image set. That is, the target binary image is the minimum binary image that includes the entire biological tissue, and the boundary of the target binary image is composed of trajectory lines. Referring to Figure 17a, which is a schematic diagram of a fourth image provided in an embodiment of the present application, the tissue contours of the fourth image are segmented to obtain a tissue segmentation image. This tissue segmentation image is then binarized, resulting in an initial binary image as shown in Figure 17b. Based on the initial binary image, a minimum binary image encompassing all tissues is determined, i.e., a target binary image as shown in Figure 17c.

[0438] Step 1027: determining, based on the similarity between the binary graph set and the gene expression visualization image, trajectory point pairs having a corresponding relationship between the binary graph set and the gene expression visualization image;

[0439] In practical applications, a gene expression visualization image may have multiple cycles, with the gene expression visualization image for each cycle being identical. That is, the index numbers of the gene expression visualization image may have multiple cycles. Based on this, when determining trajectory point pairs, for any binary image in the binary image set, multiple regions corresponding to the binary image are determined in the gene expression visualization image based on the coordinates corresponding to the vertices of the binary image and the coordinates corresponding to the trajectory points in the gene expression visualization image. The determined multiple regions correspond one-to-one to the multiple cycles. The vertices of the binary image are the trajectory points in the binary image, and the coordinates corresponding to the vertices can be determined by the index numbers corresponding to the two trajectory lines that intersect to form the vertices. For any of the multiple regions, a similarity index is determined between the binary image and the region, thereby determining multiple similarity indices between the multiple binary images and the multiple regions. Based on the multiple similarity indices, trajectory point pairs corresponding to the binary image set and the gene expression visualization image are determined. The similarity index can represent the degree of similarity between the image corresponding to the binary image and the image corresponding to the region, with a larger similarity index indicating a higher degree of similarity.

[0440] Among them, when determining the similarity index between the binary image and the region, it can be achieved in the following way: determining the binary image matrix corresponding to the binary image. Since the grayscale values ​​of the pixels of the binary image can only be 0 or 255, optionally, the elements of the binary image matrix for pixels with a grayscale value of 0 can be set to 0, and the elements of the binary image matrix for pixels with a grayscale value of 255 can be set to 1. In other words, the binary image is a matrix composed of 0 and 1. When the gene expression visualization image is a gene expression visualization image, the gene expression matrix corresponding to the gene expression visualization image can be determined. Each pixel of the gene expression visualization image corresponds to a gene expression level in the gene expression matrix. The similarity index is then determined using the binary image matrix and the gene expression matrix corresponding to the region. For example, the binary image matrix can be dot-multiplied with the gene expression matrix, that is, the binary image matrix can be multiplied with the elements at corresponding positions in the gene expression matrix, and then all elements in the matrix obtained by the dot multiplication are summed to obtain the similarity index. Thus, based on the similarity index between the binary image set and multiple regions of the gene expression visualization image, trajectory point pairs with corresponding relationships can be determined.

[0441] As can be seen from the above embodiments, the stained image can be obtained by processing the first original image using the trajectory positioning algorithm, or by scaling the first original image to a preset ratio and then processing it using the trajectory positioning algorithm. The following describes methods for determining trajectory point pairs based on multiple similarity metrics, respectively, for the two aforementioned methods of obtaining stained images and gene expression visualization images.

[0442] When using a trajectory positioning algorithm to process the first and second original images to obtain a stained image and a gene expression visualization image, after obtaining multiple similarity indices between multiple binary images in a binary image set and multiple regions, the maximum value of the multiple similarity indices can be obtained, and the binary image corresponding to the maximum value is determined to be the first target image, and the region corresponding to the maximum value is determined to be the first target region. Then, the first vertex in the first target image and the first index point in the first target region with the same coordinates as the first vertex can be obtained, and the first vertex and the first index point are determined to be a trajectory point pair with a corresponding relationship. The first vertex can be any vertex in the first target image.

[0443] After scaling the first and second original images according to a preset ratio, the scaled first and second original images are processed using a trajectory positioning algorithm to obtain a stained image and a gene expression visualization image. After obtaining multiple similarity indices between multiple binary images in the binary image set and multiple regions, a ranking result of the multiple similarity indices can be determined. Based on the ranking result, a preset number of similarity indices are obtained, and the multiple binary images and the multiple regions corresponding to the preset number of similarity indices are determined. For example, the first preset number of similarity indices with the largest values ​​can be selected. The specific value of the preset number is not limited to this embodiment of the present application and can be determined based on actual application scenarios. Because errors may occur when using scaled images for registration, to further improve image processing accuracy, the images before scaling can be obtained for the binary images corresponding to the preset number of similarity indices. That is, after determining the multiple binary images corresponding to the preset number of similarity indices, the multiple binary images are scaled up according to the inverse of the preset scaling ratio to determine multiple second target images, i.e., the images before scaling. Multiple candidate similarity indices are then determined based on the binary image matrices corresponding to the multiple second target images and the gene expression matrices corresponding to the multiple regions. Obtain a maximum value among multiple candidate similarity indices, and determine that the binary image corresponding to the maximum value is the third target image, and the region corresponding to the maximum value is the third target region. Obtain a second vertex in the third target image and a second index point in the third target region having the same coordinates as the second vertex, and determine that the second vertex and the second index point are a corresponding trajectory point pair. The second vertex can be any vertex in the third target image.

[0444] Step 1028, determining offset information based on the trajectory point pair;

[0445] Optionally, the coordinates corresponding to the two trajectory points of the regular point pair may be obtained, and the difference between the horizontal coordinate and the vertical coordinate may be determined as the offset information, ie, the offset.

[0446] Step 1029 : Processing a target image based on the offset information to obtain a registered image, wherein the target image is an image in the candidate image set corresponding to the trajectory point pair.

[0447] After the offset information is determined, the target image can be moved based on the offset information, that is, the image corresponding to the track point in the candidate image set can be moved to achieve image registration.

[0448] It should be noted that the image processing and movement in the intermediate process are all implemented on the temporary image of the copied staining image, that is, the staining image is not moved for alignment. After the calculation is completed and the final offset information is obtained, the staining image can be moved according to the rotation direction and offset information of the target image and aligned with the gene expression visualization image.

[0449] The image processing method provided in this application can achieve image processing with pixel-level precision without the need for manual image registration, thereby improving the accuracy of image processing.

[0450] Before performing cell segmentation on the registered image to obtain a cell mask image, the method further includes: step 105 , performing tissue segmentation on the registered image to obtain a tissue mask image.

[0451] In some embodiments, as shown in FIG18 , FIG18 is a flow chart of a method for performing tissue segmentation on a registered image provided by an embodiment of the present application, including:

[0452] Step 1051, obtaining the gene expression levels of the target tissue to be segmented in the registered image and determining a gene expression matrix of the registered image, wherein the gene expression matrix arranges the gene expression levels according to pixel positions;

[0453] In one embodiment, target tissue can include but not limited to any one or more of sample tissue, biological tissue, muscle tissue, bone tissue, cell tissue.Gene expression matrix characterizes the expression of gene in different positions (row and column position of corresponding gene expression matrix).Each gene expression amount data includes gene identifier, gene coordinates and gene expression amount, characterizes the expression amount of gene in a position.Take cell tissue as example, the gene expression amount of cell tissue can be determined by carrying out transcription process to the DNA of cell tissue.In addition, gene expression amount is generally in utf-8 format.

[0454] In this embodiment, the gene expression level at each pixel position in the sample image is obtained and arranged according to the row and column coordinates of the pixel position to obtain a gene expression matrix. The gene expression matrix is ​​consistent with the size of the sample image. When the gene expression level at each pixel position is 0, it indicates that the gene of the target tissue is not present at that pixel position. The gene expression level at each pixel position in the gene expression matrix can represent the gene expression level of one or more genes. If the gene expression matrix is ​​inconsistent with the size of the sample image, both images or one of the images can be scaled.

[0455] Step 1052, determining image data by assigning the gene expression value at each pixel position in the gene expression matrix to the corresponding pixel position in the registered image;

[0456] determining an image matrix having the same size as the gene expression matrix;

[0457] The gene expression amount of each pixel position in the gene expression matrix is ​​assigned to the corresponding pixel position in the image matrix and all pixel positions are normalized to obtain image data of the sample image.

[0458] Referring to FIG19 , the gene expression amount at each pixel position in the gene expression matrix is ​​stored in a dictionary with the row and column coordinates of each pixel position as the key and the gene expression amount as the value. Create an all-zero image matrix of the same size as the gene expression matrix, and assign the row and column coordinates of the gene expression amount stored in the dictionary to the same position in the image matrix, that is, the gene expression amount at the same pixel position in the image matrix is ​​the same as the gene expression amount at the same pixel position in the gene expression matrix. The assigned image matrix is ​​normalized so that the pixel value of each pixel position is within the interval [0, 255], thereby obtaining image data that is more accurate relative to the sample image and reducing the impact on subsequent tissue segmentation images. The image data is shown in FIG20 . The format of the image data is preferably TIFF format, but can also be JPG format, etc. It is not limited to a specific form of expression and can be selected according to actual conditions.

[0459] In one embodiment, a gene expression matrix is ​​generated by obtaining the gene expression amount at each pixel position and converting the gene expression matrix into image data, thereby reducing the process of repeatedly photographing and correcting tissue images through the original microscope, improving segmentation efficiency, and making the image data more accurate in displaying the target tissue compared to the initial sample image, and achieving better segmentation effect for the target tissue.

[0460] Step 1053, performing image processing on the image data to determine foreground pixels of the image data;

[0461] In one embodiment, before step S1053, the tissue segmentation method further includes: referring to FIG21 , performing convolution on the image data to extract low-frequency features of the image data. The convolution method preferably employs 2D convolution, but may also employ convolution methods such as dilated convolution and depthwise separable convolution to improve computational accuracy and expand the field of view. The image data after convolution is shown in FIG22 .

[0462] The foreground pixels of the image data are determined by image processing on the convolved image data.

[0463] Step 1054 : Mark the connected areas of the image data according to the foreground pixels to obtain the tissue mask image.

[0464] In one embodiment, the connected region refers to an image region composed of foreground pixels having the same pixel value and adjacent positions in the image data, that is, a segmented region of the target tissue.

[0465] In some embodiments, after tissue segmentation to obtain a tissue mask image, as shown in FIG23 , FIG23 is a flow chart of a method for calculating the quality of a grayscale image corresponding to a tissue mask image provided by an embodiment of the present application, including:

[0466] Step 1061 , respectively calculating and obtaining an outer contour image and an inner contour image of a grayscale image corresponding to the tissue mask image;

[0467] The outer contour image and inner contour image of the target area tissue grayscale image can be obtained based on matrix operations. In this embodiment, the outer contour image and the inner contour image are band-shaped annular contours, and the outer contour image can surround the inner contour image.

[0468] Step 1062 , dividing the outer contour image and the inner contour image into N segments respectively, and pairing each segment of the outer contour image with each segment of the inner contour image to obtain N image combinations, each of which includes a segment of the outer contour image and a segment of the inner contour image;

[0469] In order to more intuitively demonstrate the relationship between the outer contour image and the inner contour image, Figure 24 is a schematic structural diagram of the relationship between an outer contour image and an inner contour image provided in an embodiment of the present disclosure, and Figure 25 is a schematic structural diagram of another relationship between an outer contour image and an inner contour image provided in an embodiment of the present application. As shown in Figure 24, the outer contour image and the inner contour image are divided into N segments respectively, that is, the outer contour image is evenly divided into N segments, and the inner contour image is evenly divided into N segments. A segment is taken from the N segments of the outer contour image and paired with a segment from the N segments of the inner contour image to form an image combination. After the pairing is completed, N image combinations can be obtained.

[0470] Step 1063: Calculate the pixel density ratio of one outer contour image and one inner contour image in each of the image combinations respectively, obtaining N pixel density ratios. The pixel density ratio is the ratio of the pixel occupancy of one outer contour image and one inner contour image in each of the image combinations. The pixel occupancy is the ratio of the pixel points to the total pixel points.

[0471] Before calculating the pixel density ratio, it is necessary to first calculate the pixel occupancy of each outer contour image and each inner contour image. Calculating the pixel occupancy of one outer contour image includes dividing the number of pixel points within this outer contour image by the total number of pixel points within this outer contour image. In this embodiment, the pixel points are pixel points with a gray value greater than 127. The method for calculating the pixel occupancy of one inner contour image is the same as that for calculating the pixel occupancy of one outer contour image, that is, dividing the number of pixel points within this inner contour image by the total number of pixel points within this inner contour image. By analogy, finally, the pixel occupancy of all segments of the outer contour images and the pixel occupancy of all segments of the inner contour images are calculated.

[0472] After completing the above calculation of the pixel occupancy, it is also necessary to calculate the ratio of the pixel occupancy of one outer contour image to the pixel occupancy of one inner contour image. This ratio is the pixel density ratio. The one outer contour image and the one inner contour image exist in one image combination, that is, they are paired with each other. After completing the calculation of the ratio of the pixel occupancy of one outer contour image to the pixel occupancy of one inner contour image in all image combinations, finally, N pixel density ratios are obtained.

[0473] Step 1064: Calculate the quality of the grayscale image corresponding to the tissue mask according to the pixel density ratio.

[0474] Calculating the quality of the grayscale image of the target area tissue according to the pixel density ratio includes but is not limited to the following implementation methods. For example: substituting the average value of the pixel density ratio into a preset function to calculate the value describing the quality of the grayscale image of the target area tissue. The preset function can be a linear function or a non-linear function.

[0475] To facilitate the understanding of the above refined content, this embodiment provides an exemplary description in combination with formulas. For example: the average value of the effective pixel density ratio is x. When x is greater than 0 and less than 200, the formula (1) can be substituted to obtain the value describing the quality of the grayscale image of the target area tissue. Formula (1) is as follows: <s

[0476] F(x) = 100 * x^0.7 (0 < x < 200) Formula (1)

[0477] F(x) = 0 (x <= 0) Formula (2)

[0478] F(x)=100(x>=200) Formula (3)

[0479] The above formula is only for illustrative purposes and can be replaced by any function. The embodiments of the present application do not limit the formula.

[0480] In some embodiments, as shown in FIG26 , FIG26 is a flow chart of a method for performing cell segmentation on a registered image provided in an embodiment of the present application, including:

[0481] Step 1031 , input the registered image into the image segmentation model to obtain segmentation results of each pixel in the registered image, wherein the segmentation results include pixel points belonging to cells, pixel points belonging to cell contours, or pixel points of the image background.

[0482] By inputting the registered image into the above-mentioned image segmentation model, the probability value of each pixel in the registered image belonging to a cell, a cell outline, or the image background can be obtained. The initial segmentation result of each pixel in the registered image is determined based on whether the probability value is greater than a threshold value, that is, it is determined whether each pixel in the registered image belongs to a cell pixel, a cell outline pixel, or an image background pixel. In practical applications, the registered image can be expanded into a three-channel (such as RGB three-channel) image and input into the image segmentation model to obtain the initial segmentation result of each pixel in each channel image, that is, each pixel in each channel image belongs to the target object, the target object outline, or the image background. The initial segmentation results of each pixel in each channel image are then combined to determine the initial segmentation result of each pixel in the registered image.

[0483] Step 1032 , determining the pixel points belonging to cells or the pixel points of cell contours in the segmentation result as the pixel points belonging to cells, outputting the segmentation result of the registered image, and obtaining the cell mask image.

[0484] The pixel points belonging to cells or cell outlines are fused to achieve cell segmentation, that is, the pixel points of cell objects or cell outlines are determined as pixel points belonging to cells, image segmentation is performed, and the segmentation results of the aligned image are output.

[0485] In the embodiment of the present application, a watershed algorithm may be used to filter target objects whose areas are smaller than a threshold in the segmentation results of the registered image, and / or to repair incomplete target objects.

[0486] By filtering out target objects with an area smaller than a threshold and / or repairing incomplete target objects, the target objects are corrected, completing post-processing of the segmentation results of the registered image and improving the accuracy of image segmentation. For example, when the target objects are cells, algorithms such as cell filtering and correction can filter out very small cells and correct incomplete cells, resulting in more accurate cell segmentation.

[0487] The image segmentation model used in the embodiment of the present application adopts a pyramid squeezing attention module in the encoder module, which improves the feature extraction capability of the encoder module and enables the image segmentation model to pay more attention to target objects with significant features, thereby improving the segmentation accuracy of the image segmentation model.

[0488] The present invention is applied to cell segmentation in spatiotemporal omics. Figure 27 shows the segmentation effect of the image segmentation model of the present invention on a mouse brain image. A portion of the mouse brain image in the above figure is captured for demonstration. As can be seen in the lower left figure, there is cell adhesion in the mouse brain image, with some cell boundaries severely blurred. The image segmentation model proposed in the present invention can be used in the lower right figure to segment independent cells from the adhered cells, providing relatively reliable boundaries.

[0489] In some embodiments, as shown in FIG28 , FIG28 is a flow chart of a method for performing cell segmentation on a registered image provided in an embodiment of the present application, including:

[0490] Step 1033, preprocessing the gene expression visualization image to obtain a preprocessed image;

[0491] Since the gene expression visualization image is a scatter plot, it is not convenient for cell segmentation, so preprocessing is required to process the cell gene expression visualization image to obtain a preprocessed image with enhanced boundary effect.

[0492] Step 1034, binarizing the pre-processed image to obtain an initial mask image;

[0493] In this embodiment, the Otsu method may be used to perform binarization processing on the pre-processed image to obtain an initial mask image.

[0494] Step 1035 : Segment the connected domains where cells are adhered using a watershed algorithm based on distance transformation according to the initial mask image to obtain the segmented cell mask image.

[0495] The Watershed Algorithm considers image segmentation based on the composition of the watershed. The Watershed Algorithm considers image segmentation based on the composition of the watershed.

[0496] Furthermore, as a refinement and extension of the above embodiment, in order to fully illustrate the specific implementation process of the method of this embodiment, this embodiment provides a specific method as shown in FIG29 , which includes:

[0497] Step 201: Obtain a gene expression matrix including spatial positions.

[0498] A gene expression matrix including spatial locations is obtained from gene expression data of the cell. The gene expression data may include gene identifiers, coordinate locations, and total gene expression levels at the corresponding coordinate locations for multiple genes.

[0499] Step 202: Generate a gene expression visualization image of the cell based on the gene expression matrix.

[0500] As an optional method, step 202 may specifically include: first obtaining the coordinate positions of the expressed genes and the total gene expression levels at the corresponding coordinate positions in the gene expression matrix; then, generating a gene expression visualization image based on the coordinate positions of the expressed genes and the total gene expression levels at the corresponding coordinate positions, wherein the gene expression visualization image is a grayscale image, and the grayscale value of a pixel in the gene expression visualization image is the total gene expression level at the coordinate position corresponding to the pixel. This optional method can accurately generate a gene expression visualization image of the cell.

[0501] For example, in the specific use of cell segmentation, a gene expression matrix containing spatial positions is input, and an expression image is generated based on the coordinate positions of the expressed genes and the total gene expression amount at the corresponding positions. The specific form of the image is a grayscale image, and the grayscale value of the coordinate point is the total amount of gene expression at the coordinate.

[0502] As another optional method, step 202 may further specifically include: drawing a visualized image of the gene expression of the cell based on the gene expression matrix and the segmented mask map, wherein the spatial positions in the gene expression matrix correspond to the spatial positions in the segmented mask map. For example, a cell-based expression level map may be drawn based on the gene expression matrix and the segmented mask map, wherein the spatial positions in the gene expression matrix may correspond to the spatial positions in the segmented mask map. Through this optional method, a visualized image of the gene expression of the cell can be accurately generated.

[0503] Since the generated gene expression visualization image is a scatter plot, it is not easy to segment and needs to be processed. Specifically, the process shown in step 203 can be performed.

[0504] Step 203: pre-process the gene expression visualization image to obtain a median image, and sharpen the median image to obtain a sharpened image.

[0505] Because the gene expression visualization image is a scatter plot, it is not convenient for cell segmentation. Therefore, preprocessing is required. In this embodiment, the gene expression visualization image of the cell is first processed into a median map. The median map is then sharpened using the Laplacian operator to enhance the boundary effect of the median map, thereby obtaining a sharpened map.

[0506] Optionally, the gene expression visualization image is preprocessed to obtain a median map, which may specifically include: first, using a convolution kernel of a preset size (such as 13*13) to perform a convolution operation on the gene expression visualization image, so that the scattered points in the gene expression visualization image are adhered to obtain a first convolution map; then, detecting the local maximum point of the first convolution map based on the two-dimensional grayscale peak of the image; obtaining the pth percentile of the local maximum point, where p is a preset value, such as the pth percentile can be the 98th percentile value, or the 99th percentile value, etc.; if the pth percentile is within a preset range, using a first median filter to perform median filtering on the first convolution map to obtain a median map, where the filter size of the first median filter is determined according to the preset size.

[0507] Further optionally, after obtaining the pth percentile in the local maximum point, the method of this embodiment may also include: if the pth percentile is outside the preset range, determining the new size of the convolution kernel based on the pth percentile and the preset size; using the convolution kernel of the new size to perform a convolution operation on the gene expression visualization image, so that the scattered points in the gene expression visualization image are adhered to obtain a second convolution image; using a second median filter to perform median filtering on the second convolution image to obtain a median image, wherein the filter size of the second median filter is determined based on the new size.

[0508] Exemplarily, determining the new size of the convolution kernel based on the pth percentile and the preset size may specifically include: calculating the new size of the convolution kernel according to the formula K=N*(N / R), where K*K represents the new size of the convolution kernel, N*N represents the preset size, and R represents the pth percentile.

[0509] For example, first use a convolution kernel of size 13*13 (empirical value) to perform a convolution operation on the original image of the gene expression visualization image to make the scatter plot stick together, and obtain the first convolution image; then detect the local maximum point of the first convolution image based on the two-dimensional grayscale peak of the image, and take out the 99% percentile value R of all the local maximum points obtained. If the R value is too far from a set empirical threshold (too high or too low will affect subsequent processing), it is considered that the 13*13 convolution kernel is not suitable for the original image. At this time, calculate K=13*(13 / R), change the original 13*13 convolution kernel to a K*K convolution kernel, reprocess the original image, and obtain the second convolution image; if the R value is within the allowable range, use the first convolution image. The convolution image is presented as a stack of blocks of different grayscales. The grayscale span is quite dramatic, and some images show a phenomenon of hollow centers in clusters of clusters. To eliminate this problem, a filter size of 13*2.7=35 (2.7 can be an empirical value, and the decimal is rounded down) is used (if K needs to be calculated, the kernel size here is K*2.7) to perform median filtering on the second convolution image to obtain a median image.

[0510] In order to fill the voids in the point clusters, the median filter used above is relatively large, and the grayscale boundary of the median image may be relatively blurred. The laplacian operator is used to sharpen the median image to enhance the grayscale boundary and obtain a sharpened image. After completing the preprocessing, the cell segmentation process is now performed. Specifically, the process shown in steps 204 to 207 can be executed.

[0511] Step 204: binarize the sharpened image to obtain an initial mask image.

[0512] Use Otsu's method to binarize the sharpened image obtained in the previous step to obtain the initial mask image.

[0513] Step 205: Filter the connected domains whose areas do not meet the preset conditions in the initial mask image to obtain a filtered mask image.

[0514] Optionally, step 205 may specifically include filtering connected domains in the initial mask image whose areas are greater than a first preset threshold, or whose areas are less than a second preset threshold, to obtain a filtered mask image, where the first preset threshold is greater than the second preset threshold. For example, an empirical threshold may be used to filter connected domains in the initial mask whose areas are too small or too large to obtain the filtered mask image.

[0515] Step 206 , traverse each connected domain in the filter mask image, extract the area where the connected domain is located based on the minimum circumscribed rectangle of the connected domain, and use the watershed algorithm to segment the connected domain where cell adhesion exists to obtain a segmented mask image.

[0516] Optionally, step 206 may specifically include: setting the grayscale value of each pixel point in each connected domain in the filter mask image to a first value, and setting the grayscale value of each pixel point outside the connected domain to a second value; for each target pixel point in the connected domain whose grayscale value is the first value, remapping the grayscale value of the target pixel point to the distance from the target pixel point to the pixel point with the grayscale value of the second value closest to it, to obtain a distance map of the connected domain; binarizing the distance map of the connected domain to obtain a preset number of pixels in the connected domain that are farthest from the pixel point with the grayscale value of the second value; using the preset number of pixels as injection points of the watershed algorithm, and using the watershed function to perform watershed segmentation on the original mask of the connected domain in the filter mask image to obtain a segmented target connected domain, and covering the target connected domain with the filter mask image to obtain a segmented mask image.

[0517] For example, traverse each connected domain in the filter mask image, extract the area where the connected domain is located based on the minimum bounding rectangle of the connected domain, and use the watershed algorithm based on distance transformation to further segment the connected domain where cell adhesion may exist. The specific steps are as follows:

[0518] Step a: For each connected domain area extracted, the grayscale value of the points within the target connected domain is set to 1, and the grayscale value of the points outside the connected domain is set to 0 (including background and non-target connected domains). A distance transformation is performed on each point with a value of 1, and its grayscale value is remapped to the distance from the point to the nearest point with a value of 0 (the distance between adjacent points is 1), thereby obtaining the distance map of the connected domain.

[0519] Step b: Use an empirical threshold to binarize the distance map to obtain the number of points in the connected domain that are farthest from the point with a distance value of 0 (the number of points varies due to the different conditions of each connected domain).

[0520] Step c: Use the several points obtained in the previous step as the water injection points of the watershed algorithm, use the watershed function in OpenCV to perform watershed segmentation on the original mask of the connected domain, obtain the target connected domain after segmentation, and overlay the result onto the filter mask map.

[0521] Step d: Perform the above steps on each connected domain traversed to obtain a segmented mask image.

[0522] Step 207: Perform a closure operation on the segmented mask image to obtain a mask image of the cell segmentation result.

[0523] To make cell boundaries more regular, the segmented mask is closed to obtain the final mask. This final mask can be exported and saved for subsequent biological analysis at the cellular level in conjunction with the original expression matrix.

[0524] For example, as shown in Figure 30, a schematic diagram of an example process based on the method of this embodiment is shown. First, a gene expression matrix can be input, and a gene expression visualization image can be generated based on the matrix, as shown in Figure 31. Then, a 13*13 convolution kernel is used to perform a convolution operation on the gene expression visualization image, so that the scattered points in the gene expression visualization image are adhered, resulting in a first convolution map. At this point, a threshold is required for judgment. That is, the local maximum points of the first convolution map are detected based on the two-dimensional grayscale peak of the image. The 99% percentile value R of all the local maximum points obtained is taken. If the R value is too far from a set empirical threshold (too high or too low will affect subsequent processing), the 13*13 convolution kernel is considered unsuitable for the original image. A new convolution kernel size is calculated using a ratio, and then the new convolution kernel is used to process the original image to obtain a second convolution map. The median filter size is calculated using this ratio, and then the second convolution map is processed with a median filter of this size to obtain a median map. If the R value is within the allowable range, the first convolution map is used and processed with a median filter of size 35 to obtain a median map.

[0525] After obtaining the median image, the Laplacian operator was used to sharpen it, resulting in a sharpened image, as shown in Figure 32. The sharpened image was binarized using the large law method to obtain an initial mask image, as shown in Figure 33. Adhesion cells were then segmented using area filtering and the watershed algorithm. Finally, the cell mask image was output and saved, as shown in Figure 34.

[0526] Compared to existing cell segmentation methods, this embodiment provides a solution for cell segmentation based directly on gene expression visualization images. By combining multiple image processing methods, it delivers more reliable cell segmentation results. Cell segmentation is independent of imagery and eliminates the need for additional registration technology between imagery and gene expression visualizations. This eliminates the introduction of additional errors, reduces overall operational time and technical costs, and improves the efficiency and accuracy of cell segmentation.

[0527] In some embodiments, as shown in FIG35 , FIG35 is a flow chart of a method for performing cell segmentation on a registered image provided by an embodiment of the present application, including:

[0528] Step 1036 , determining the labeled regions and cell areas corresponding to each tissue type of the target object based on the stained image;

[0529] Specifically, the tissue information provided by the staining map and the prior knowledge related to the target object can be combined to determine the labeled areas corresponding to the various tissue types contained in the target object based on the staining map. The prior knowledge related to the target object can be which cell tissues the target object contains, or the cell morphology of these tissues. In an optional case, the tissue outlines of each cell tissue of the target object can be drawn using a specific manual tool, and the areas corresponding to different tissues can be labeled with different colors. The specific manual tool can be the open source tool 3Dslicer, which supports manual labeling, and can also be combined with the open source tool plant-seg network to label tissue areas.

[0530] The cell area in this step can be a pre-set reference cell area for the target tissue type. In other words, the cell area for each tissue type is pre-set. This pre-set cell area can be based on experience or the average value of cells of the same tissue type across different subjects. Therefore, when executing this step, the pre-set cell area can be directly obtained and used as the cell area corresponding to the marked region for subsequent processing.

[0531] Step 1037: determining the area of ​​a sub-region to be processed contained in each of the marked regions based on the cell area, wherein the area of ​​the sub-region to be processed is positively correlated with the cell area, and the sub-region to be processed is used to represent the minimum processing unit of the corresponding marked region;

[0532] Among them, the sub-region to be processed corresponding to the marked area can be used as the minimum processing unit of the marked area to process and analyze tissue cells. When using a gene chip to extract the position information of cells in the target object, a sub-region to be processed is extracted as a unit, and the positional relationship between the sub-regions to be processed is used to characterize the positional relationship between cells in the marked area. Because the area of ​​the sub-region to be processed is positively correlated with the cell area, that is, the larger the cell area of ​​the marked area determined in the previous step, the larger the sub-region to be processed divided by the marked area. Therefore, it can better express the positional relationship between cells in the marked domain corresponding to different tissue types.

[0533] Step 1038: gridding the corresponding marked area according to the area of ​​each sub-area to be processed to obtain grid coordinates corresponding to each marked area;

[0534] Specifically, an optional way to perform gridding processing on the marked area is to divide each marked area into grids with an area size equal to the area of ​​the corresponding sub-area to be processed, and obtain gridded coordinates.

[0535] Optionally, the center point coordinates of the grid can also be used as the gridded coordinates of the grid, wherein the center point coordinates of the grid can be determined as the center point coordinates of the grid by obtaining the coordinate interval where the grid is located in the coordinates established by the capture unit, calculating the center point coordinates of the coordinate interval, and determining them as the center point coordinates of the grid.

[0536] Step 1039 : extracting data corresponding to the gridded coordinates from the gene expression visualization image, and superimposing the data onto the gridded coordinates to obtain the cell processing result of the target object.

[0537] Specifically, the data corresponding to each gridded coordinate is extracted from the gene expression data to obtain the tissue cell processing data of the target object, which includes the expression levels of different genes of the target object in cell tissues of various tissue types. The corresponding gridded coordinates are further combined to generate the tissue cell processing results of the target object.

[0538] In one possible implementation, the gene expression data includes a gene identifier ID, gene coordinate data x, y, and gene expression amount MIDCount. Since the grid coordinates have been determined at this time, the coordinate range of each grid can be determined by the grid coordinates. Then compare the gene coordinate data x, y of each data with the coordinate range of each grid. If the gene coordinate data x, y falls within the coordinate range of a grid, it means that the data needs to be superimposed on this grid. At this time, the ID of the corresponding grid can be set for the data in the expression data, that is, the Cell ID. When the data is superimposed subsequently, according to the added Cell ID, it can be clear which grid each data falls in, that is, the center point coordinates of the grid to which the data needs to be superimposed.

[0539] In some embodiments, as shown in FIG36 , FIG36 is a flow chart of a method for determining the cells to which background gene molecules of unknown cells belong in a gene expression visualization image provided by an embodiment of the present application, including:

[0540] Step 1041 , based on the cell mask image, determining each gene expression feature of the gene expression visualization image as a cell gene expression feature located inside the cell and a background gene expression feature located outside the cell;

[0541] In this step, based on the cell mask image, each gene expression feature of the gene image is determined as a cell gene expression feature located inside the cell and a background gene expression feature located outside the cell.

[0542] In this embodiment, the gene expression feature is DNB. Referring to Figure 37, Figure 37 is a schematic diagram of the correspondence between DNB and cell membrane, where x and y are the DNB coordinates, geneID and MIDCount represent the gene information, and label represents the cell identity. Each DNB in ​​the stereo-seq spatial transcriptome data is mapped to the cell information to which the DNB belongs, and the remaining DNBs not in the cell are treated as background DNBs.

[0543] Step 1042, for each of the cell gene expression features, a probability distribution model is fitted using the spatial position and the signal value of the gene expression feature;

[0544] In this step, for each of the cell gene expression features, a probability distribution model is fitted through the spatial position and the signal value of the gene expression feature.

[0545] As a preferred implementation, in this embodiment, the probability distribution model can be a Gaussian mixture model, but different probability distribution models can be used, and in the judgment of the cells to which the background signal belongs, the Bayesian model can be used to judge the cells to which it belongs, and corresponding selection and adjustment can be made according to actual needs.

[0546] In this step, a Gaussian mixture model is fitted for the DNB in ​​each cell using the coordinates (x, y) and the DNB signal value.

[0547] Step 1043, based on the probability distribution model, determining a probability value of a target background gene expression feature within a preset range of a target cell belonging to the target cell;

[0548] In this step, based on the probability distribution model, the probability value of the target background gene expression feature within a preset range of a target cell belonging to the cell is determined. The target cell can be all cells in the biological sample or a portion of the cells in the biological sample.

[0549] Specifically, as an optional implementation, in this step, taking the target cell as cell a as an example, for cell a, the Gaussian mixture model is used to calculate the probability value of the background DNB within the central range of 50x50 pixels of cell a belonging to cell a.

[0550] Step 1044: Output a correction result based on the probability value to correct the background gene expression characteristics.

[0551] Correction for background gene expression signatures was performed based on probability values.

[0552] Specifically, in this step, it is determined whether the probability value is greater than or equal to the preset probability value. If so, the target background gene expression feature is used as the cell gene expression feature located in the target cell to correct the background gene expression feature. If not, the target background gene expression feature continues to be used as the background gene expression feature.

[0553] Referring to FIG38 , FIG32 is a schematic diagram of the correction result of background DNB correction. For background DNBs, those with a probability less than a preset value are regarded as background DNBs, and those with a probability greater than or equal to the preset value are regarded as cell DNBs.

[0554] In this embodiment, the preset probability value is not specifically limited and can be selected and adjusted accordingly according to actual needs.

[0555] As an optional embodiment, in this step, in response to the target background gene expression feature being simultaneously located outside at least two target cells, a probability value belonging to each corresponding target cell is determined respectively; the highest probability value is selected from the at least two determined probability values ​​as the probability value for outputting the correction result, and the target cell corresponding to the highest probability value is used as the cell to which the target background gene expression feature belongs.

[0556] Taking the target cells as cells a and b as an example, assuming that the background gene expression feature A is within the central range of 50x50 pixels of both cells a and b, the probability value of the background gene expression feature A belonging to cell a is calculated to be x1, and the probability value of the background gene expression feature A belonging to cell b is x2. If x1 is greater than x2, then the background gene expression feature A is determined to be the gene expression feature of cell a.

[0557] In some embodiments, as shown in FIG39 , FIG39 is a flow chart of a method for determining the cells to which background gene molecules of unknown cells belong in a gene expression visualization image provided by an embodiment of the present application, including:

[0558] Step 1045, determining the initial gene molecules belonging to the target cells and the background gene molecules of the unknown cells based on the gene expression visualization image;

[0559] Step 1046, obtaining the coordinate information of the initial gene molecule;

[0560] The microscope image and the gene image are aligned according to the coordinates, and the coordinate information of the initial gene molecules in the cell can be obtained through the cell segmentation boundary of the microscope image.

[0561] Step 1047, determining the center coordinates of the target cell according to the coordinate information;

[0562] Step 1048: construct a Voronoi diagram based on the center point coordinates and the coordinate information;

[0563] A Voronoi diagram (also known as a Thiessen polygon or Dirichlet diagram) consists of a set of continuous polygons formed by the perpendicular bisectors of the lines connecting two adjacent points. N distinct points on a plane are partitioned according to the nearest neighbor principle; each point is associated with its nearest neighbor region.

[0564] Step 1049: Determine the cells to which the background gene molecules within the Voronoi diagram belong.

[0565] According to the labels set for the background gene molecules within the Voronoi diagram, the background gene molecules within the Voronoi diagram can be determined to completely belong to the target cells, partially belong to the target cells, or not belong to the target cells at all.

[0566] In this embodiment, the coordinates of the target cell's initial gene molecules are used to determine the target cell's center coordinates. A Voronoi diagram is then constructed based on the target cell's center coordinates to identify the cells to which the background gene molecules within the Voronoi diagram belong. This improves both calibration efficiency and accuracy. Furthermore, this embodiment only requires determining the initial gene molecules within the target cell, eliminating the need for a map of the target cell's outline. This reduces the need for high-precision cell segmentation, reduces reliance on target cell morphological images, and enhances the robustness of the calibration scheme.

[0567] In some embodiments, the step of determining the initial gene molecules belonging to the target cells and the background gene molecules of the unknown cells based on the gene image comprises:

[0568] Obtain microscopic and genetic images of biological samples;

[0569] Perform image registration on microscope images and gene images of biological samples;

[0570] performing cell segmentation on the microscope image to obtain a cell segmentation result;

[0571] Determining a gene image of a target cell based on the cell segmentation result and the image registration result of the gene image;

[0572] According to the gene image of the target cell, the initial gene molecules belonging to the target cell and the background gene molecules of the unknown belonging cells are determined.

[0573] In some embodiments, the step of determining the coordinates of the center point of the target cell according to the coordinate information includes:

[0574] Counting the number of molecules of the initial gene molecules;

[0575] Calculating the sum of the abscissa values ​​and the sum of the ordinate values ​​of the initial gene molecules;

[0576] Determine a first ratio as the abscissa value of the center point coordinate; the first ratio is the ratio of the sum of the abscissa values ​​of the initial gene molecules to the number of the molecules;

[0577] The second ratio is determined as the ordinate value of the center point coordinate; the second ratio is the ratio of the sum of the ordinate values ​​of the initial gene molecules to the number of molecules.

[0578] In some embodiments, the step of determining the coordinates of the center point of the target cell according to the coordinate information includes:

[0579] Obtaining the maximum abscissa value, the minimum abscissa value, the maximum ordinate value, and the minimum ordinate value in the initial gene molecule;

[0580] Determine a third ratio as the abscissa value of the center point coordinate; the third ratio is half of the difference between the maximum abscissa value and the minimum abscissa value;

[0581] The fourth ratio is determined as the ordinate value of the center point coordinate; the fourth ratio is half of the difference between the maximum ordinate value and the minimum ordinate value.

[0582] In some embodiments, the gene image data correction method further comprises:

[0583] If the background gene molecules are within different Voronoi diagram ranges, the background gene molecules are input into a Gaussian mixture model to determine the probability scores of the background gene molecules belonging to target cells within a preset distance range;

[0584] The background gene molecules are corrected to the gene molecules belonging to the target cells with high probability scores.

[0585] In some embodiments, the step of determining the cells to which the background gene molecules within the Voronoi map belong comprises:

[0586] Generate target cell gene expression information based on the corrected results of the gene molecules belonging to the target cell.

[0587] In some embodiments, the step of constructing a Voronoi diagram based on the center point coordinates and the coordinate information includes:

[0588] Constructing a polygonal network for all the center point coordinates;

[0589] Determining edges of a Voronoi diagram based on the polygonal network;

[0590] A Voronoi diagram is constructed according to the edges of the Voronoi diagram.

[0591] In some embodiments, as shown in FIG40 , FIG40 is a flow chart of a method for determining the cells to which background gene molecules of unknown cells belong in a gene expression visualization image provided by an embodiment of the present application, including:

[0592] Step 10410, determining the initial number of gene molecules belonging to the target cells in the gene expression visualization image and the initial contour area of ​​the target region occupied by the gene molecules belonging to the target cells;

[0593] The convex hull algorithm is used to calculate the initial contour area of ​​the target region occupied by the gene molecules belonging to the target cell. The convex hull algorithm finds the convex hull of N points in the target region based on the cell segmentation results. The initial contour area is the area enclosed by the connection between these N points.

[0594] Step 10411, determining the gene molecule density of the target cell according to the number of molecules and the initial contour area;

[0595] In an alternative embodiment, the ratio of the number of molecules to the initial contour area is determined as the gene molecule density of the target cell.

[0596] Step 10412, determining the number of background gene molecules within a preset region of the target region and the area of ​​the preset region;

[0597] Background gene molecules are gene molecules whose cell of origin cannot be determined based on the gene image. The preset region is outside the target region but cannot exceed the area occupied by gene molecules belonging to neighboring cells. The preset region can be 5 pixels by 5 pixels.

[0598] In an optional embodiment, the preset region range may be a region determined based on the center point of the target cell as the center. The center point of the target cell is determined by the gene image. A schematic diagram of the center point of the target cell is shown in FIG41 .

[0599] Step 10413, determining the density of background gene molecules according to the number of background gene molecules and the area of ​​the region;

[0600] In an optional embodiment, the ratio of the number of background gene molecules to the area of ​​the region is determined as the background gene molecule density.

[0601] Step 10414: Determine the cell to which the background gene molecule belongs based on the gene molecule density and the background gene molecule density.

[0602] In this embodiment, the aforementioned cell characteristics are utilized to determine the target cell's gene density based on the initial number of target cell genes and the target cell's initial outline area in the gene image. By comparing the target cell's gene density with the background gene density, the cell to which the background gene molecules belong is determined based on the gene density and background gene density. This significantly improves the efficiency and accuracy of correcting target cell genes.

[0603] In some embodiments, as shown in FIG42 , FIG42 is a flow chart of a method for determining the cells to which background gene molecules of unknown cells belong in a gene expression visualization image provided by an embodiment of the present application, including:

[0604] Step 10415, determining the target area occupied by the gene molecules belonging to the target cells in the gene expression visualization image;

[0605] Step 10416, determining the spatial coordinates and gene expression characteristic signal values ​​of background gene molecules of unknown cells within a preset calibration range of the target region;

[0606] In this embodiment, the gene expression signature is DNB. By mapping each DNB in ​​the Stereo-seq spatial transcriptome data, the cell information to which the DNB belongs can be obtained. The remaining DNBs outside the cell are considered background DNBs. The spatial coordinates of the background gene molecules can be represented by the coordinates of the background DNBs.

[0607] Step 10417: Input the spatial coordinates and the gene expression characteristic signal values ​​into a probability distribution model to obtain a probability score of the background gene molecule belonging to the target cell; the probability distribution model is fitted based on the spatial coordinates and gene expression characteristic signal values ​​of the gene molecule sample belonging to the target cell;

[0608] When fitting a GMM (Gaussian mixture model), there are three independent variables: spatial coordinates (x, y) and the signal value. The dependent variable, or output, is the probability score for each gene molecule belonging to the target cell. The specific implementation process involves introducing the Gaussian mixture function from Scikit-learn (a machine learning library), automatically fitting the GMM model parameters to the input data, and obtaining the final probabilistic model.

[0609] The probability distribution model is obtained by fitting the spatial coordinates of the gene molecule samples and the gene expression characteristic signal value samples belonging to the target cells. As a preferred embodiment, in this embodiment, the probability distribution model can be a Gaussian mixture model.

[0610] Step 10418: Determine the cell to which the background gene molecule belongs based on the probability score.

[0611] Compare whether the probability score is greater than or equal to the probability threshold.

[0612] If the probability score is greater than or equal to the probability threshold, the background gene molecules are corrected to the gene molecules belonging to the target cell. If the probability score is less than the probability threshold, the background gene molecules remain background molecules of the unknown cell.

[0613] In an optional embodiment, the third quartile between the first probability score and the second probability score is determined as the probability threshold.

[0614] The first probability score is the probability score with the highest score value among the probability scores; the second probability score is the probability score with the lowest score value among the probability scores.

[0615] In this embodiment, based on a rough understanding of the target region of the target cell, the probability fraction of background gene molecules of unknown cells within a preset correction range belonging to the target cell is calculated, and the cell to which the background gene molecules belong is determined based on the probability fraction. This reduces the large errors in gene image data caused by gene molecule diffusion, improving correction efficiency. Furthermore, it does not rely on high-precision cell segmentation results, enhancing the robustness of the correction scheme.

[0616] Furthermore, this application also provides on-chip track cross detection, which is mainly used for subsequent stitching correction and registration. Both stitching and registration use the track detection results.

[0617] 43 shows a flow chart of a method 100 for determining point coordinates in a stained image according to an embodiment of the present invention. The method 100 may include a sampling step 1001 , a grouping step 1002 , a straight line fitting step 1003 , and a coordinate calculation step 1004 .

[0618] To facilitate understanding of the determination method of the present invention, the staining image involved herein will now be exemplified with reference to FIG. 44 . FIG. 38 shows an example staining image to which the present invention is applicable. The staining image can be acquired by a camera, staining image sensor, or the like. The staining image can be a staining image of a mechanical structure, a carrier, a slide, or a biochip, for example, having straight lines. As shown in FIG. 38 , the staining image plane in which the staining image resides may include a first direction X and a second direction Y perpendicular to the first direction X. The staining image includes a plurality of first straight lines transverse to the second direction Y (i.e., a plurality of dark lines in the staining image that are substantially parallel to the first direction X) and a plurality of second straight lines transverse to the first direction X (i.e., a plurality of dark lines in the staining image that are substantially parallel to the second direction Y). Specifically, there are 6 first straight lines and 8 second straight lines in this example staining image. Since each first straight line intersects with each second straight line, there are a total of 6×8=48 intersection points in this example staining image. As shown in FIG. 38 , the staining image can have a pixel length L and a pixel width W. In Figure 38, also shown is an example second sampling bar. As an illustration and not limitation, the pixel length of this sampling bar is identical with the pixel length of this example dyeing image, i.e., L, and the pixel width of this sampling bar is less than the pixel width of this example dyeing image, which is W2. In this article, a straight track line transverse to either direction refers to that this straight track line is parallel to the direction perpendicular to this either direction or is less than a preset value with the angle of the direction perpendicular to this either direction. The scope of the preset value can include 1 to 10 degree, 10 to 20 degree, 20 to 30 degree, 30 to 40 degree or 40 to 50 degree etc., and for example, the preset value can be 1 degree, 2 degrees, 3 degrees, 4 degrees, 5 degrees, 6 degrees, 7 degrees, 8 degrees, 9 degrees, 10 degrees.

[0619] The sampling step 1001 includes a first direction sampling step and a second direction sampling step.

[0620] In the first direction sampling step, a first sampling strip is used to perform first-direction equally spaced sampling along the first direction on the stained image to determine a first set of candidate points. For each first-direction equally spaced sampling, the following first-direction sampling sub-steps 1 and 2 are performed. The first direction sampling sub-step 1 includes accumulating pixel values ​​along the first direction for pixels within the sampling region of the stained image selected by the first sampling strip during the first-direction equally spaced sampling to obtain a series of first pixel value sums. The value of the series of first pixel value sums at any second-direction position in the second direction is equal to the sum of the pixel values ​​of all pixels within the sampling region aligned along the first direction at that second-direction position. The first direction sampling sub-step 2 includes determining extreme points of the series of first pixel value sums, and for each extreme point determined, determining a corresponding first candidate point obtained by the first-direction equally spaced sampling. The coordinates of the corresponding first candidate point in the second direction are the coordinates of the second-direction position corresponding to the extreme point in the second direction. The coordinates of the corresponding first candidate point in the first direction are determined based on the position of the first sampling strip in the first direction during the first-direction equally spaced sampling. In this way, the first candidate points obtained by sampling in the first direction at equal intervals each time form the first candidate point set.

[0621] In the second direction sampling step, a second sampling strip is used to perform second-direction equally spaced sampling of the stained image along the second direction to determine a second set of candidate points. For each second-direction equally spaced sampling, the following second-direction sampling sub-steps 1 and 2 are performed. The first second-direction sampling sub-step comprises: accumulating pixel values ​​along the second direction for pixels within the sampling region of the stained image selected by the second sampling strip during the second-direction equally spaced sampling period to obtain a series of second pixel value sums, wherein the value of the series of second pixel value sums at any first-direction position in the first direction is equal to the sum of the pixel values ​​of pixels within the sampling region aligned along the second direction at that first-direction position. The second second-direction sampling sub-step 2 comprises: determining extreme points of the series of second pixel value sums, and for each extreme point determined, determining a corresponding second candidate point obtained by the second-direction equally spaced sampling. The coordinates of the corresponding second candidate point in the first direction are the coordinates of the first-direction position corresponding to the extreme point in the first direction. The coordinates of the corresponding second candidate point in the second direction are determined based on the position of the second sampling strip in the second direction during the second-direction equally spaced sampling period. In this way, the second candidate points obtained by sampling in the second direction at equal intervals each time form the second candidate point set.

[0622] All extreme value points used in the sampling step 1001 may be the same maximum value point or the same minimum value point. For example, if the straight track line is a dark line, all extreme value points used may be the same minimum value point; if the straight track line is a bright line, all extreme value points used may be the same maximum value point.

[0623] In the grouping step 1002 , the first candidate points in the first candidate point set are divided into a plurality of first candidate point groups; and the second candidate points in the second candidate point set are divided into a plurality of second candidate point groups.

[0624] In one embodiment, the first candidate points in the first candidate point set are divided into a plurality of first candidate point groups using a first grouping method, wherein the first grouping method is configured such that the first candidate points in each first candidate point group are substantially arranged in a straight line transverse to the second direction. Furthermore, the second candidate points in the second candidate point set are divided into a plurality of second candidate point groups using a second grouping method, wherein the second grouping method is configured such that the second candidate points in each second candidate point group are substantially arranged in a straight line transverse to the first direction.

[0625] In the line fitting step 1003 , a line fitting is performed on the first candidate point in each first candidate point group to obtain a corresponding first line analytical expression; and a line fitting is performed on the second candidate point in each second candidate point group to obtain a corresponding second line analytical expression.

[0626] Performing linear fitting based on multiple points with known coordinates to obtain a linear expression is well known in the art and will not be described in detail here. Common linear fitting methods include: least squares method; gradient descent method; Gauss-Newton method and Lev-Massachusetts method.

[0627] In the coordinate calculation step 1004, the coordinates of the intersection points of each first linear analytical expression and each second linear analytical expression are calculated. The method of calculating the coordinates of the intersection points of two known linear analytical expressions is well known in the art and will not be described in detail here.

[0628] FIG45 is a directed target detection method provided by an embodiment of the present application, including:

[0629] Step 1005: Acquire a staining image or a gene expression visualization image to be tested.

[0630] Step 1006: Input the staining image or gene expression visualization image to be tested into a directed target detection network to obtain a target detection result.

[0631] Specifically, the stained image or gene expression visualization image to be tested is input into the trained YOLOV6-based directed target detection model, and the multi-level convolutional neural network trained in the directed target detection model is used to perform convolution operations and feature map fusion on the image to be detected, and output feature fusion maps of different scales; for feature fusion maps of different scales, corresponding predictors are used to predict category confidence, target frame position confidence, target detection confidence and angle classification confidence.

[0632] The Backbone layer adopts the RepVGG architecture, and the Neck layer adopts the FPN-PAN architecture.

[0633] The Head layer uses a decoupled head, so that a classification branch performs a category analysis to obtain the classification information, and a regression branch performs a regression analysis to respectively obtain the target frame coordinate prediction information, the target prediction information, and the angle prediction information. The classification information used to represent the category analysis result, the target frame coordinate prediction information used to determine the position of the target detection frame, the target prediction information used to determine whether there is a corresponding target in the target detection frame, and the angle prediction information used to determine the angle of the target detection frame. The corresponding target may include a track line or a track cross (also known as a track intersection). An embodiment of the present application provides a method for training a directed target detection network based on YOLOV6, the method comprising: obtaining multiple training images and label information including labels in the multiple training images; and inputting the multiple training images and the label information into the directed target detection network based on YOLOV6 to obtain setting parameters of the directed target detection network based on YOLOV6 through training; wherein, the directed target detection network based on YOLOV6 comprises: a backbone layer, configured to perform feature extraction on the training image to obtain multiple feature maps of different scales; a neck layer, configured to receive the multiple feature maps from the backbone layer and perform cross-scale fusion on the multiple feature maps to obtain multiple fused feature maps; and a detection head layer, configured to predict the multiple fused images to obtain prediction results about the training image, wherein the prediction results include: classification information, target frame coordinate prediction information, target prediction information and angle prediction information.

[0634] In some embodiments, the Backbone layer adopts a RepVGG architecture, and the Neck layer adopts an FPN-PAN architecture.

[0635] In some embodiments, the Head layer adopts a decoupling head so that category analysis is performed via a classification branch to obtain the classification information, and regression analysis is performed via a regression branch to respectively obtain the target frame coordinate prediction information, the target prediction information and the angle prediction information.

[0636] In some embodiments, inputting the plurality of training images and the label information into the YOLOV6-based directed target detection network to obtain the setting parameters of the YOLOV6-based directed target detection network through training includes: in each training,

[0637] Obtain the corresponding training images and corresponding label information for the current training;

[0638] Use the YOLOV6-based directed target detection network to predict the target in the corresponding training image to obtain a corresponding prediction result;

[0639] Calculating the loss function of the YOLOV6-based directed object detection network according to the corresponding label information and the corresponding prediction results; and

[0640] Based on the calculation result of the loss function, the setting parameters of the YOLOV6-based directed target detection network are adjusted until the calculation result of the loss function converges.

[0641] In some embodiments, the method further includes: preprocessing the acquired label information in the first data format to process the label information into data in the form of [x, y, w, h, θ, classID], and inputting the processed data into the YOLOV6-based directed target detection network.

[0642] Wherein, (x, y) represents the position of the center of the target detection frame, w represents the width of the long side of the target detection frame, h represents the height of the short side of the target detection frame, θ represents the angle of rotation of the long side of the target detection frame relative to the horizontal axis, and classID represents the category of the target detection frame.

[0643] In some embodiments, the first data form of the label information is [x1, y1, x2, y2, x3, y3, x4, y4, classID], where (x1, y1), (x2, y2), (x3, y3) and (x4, y4) respectively define the positions of the four vertices of the target detection box.

[0644] In some embodiments, the method further includes: post-processing the obtained prediction result in the form of [x, y, w, h, θ, conf, classID] to process the prediction result into data in a second data form, and presenting the data in the second data form to the user.

[0645] Wherein, (x, y) represents the position of the center of the target detection box, w represents the width of the long side of the target detection box, h represents the height of the short side of the target detection box, θ represents the angle of rotation of the long side of the target detection box relative to the horizontal axis, conf represents the confidence that the target detection box has the corresponding target, and classID represents the category of the target detection box.

[0646] In some embodiments, the second data format is [x1, y1, x2, y2, x3, y3, x4, y4, conf, classID], where (x1, y1), (x2, y2), (x3, y3) and (x4, y4) respectively define the positions of the four vertices of the target detection box.

[0647] Furthermore, as shown in FIG46 , FIG46 is a method for obtaining cell gene expression information of a biological sample provided in an embodiment of the present application, including:

[0648] Step 1101 , acquiring a first image of a biological sample, wherein the first image includes stained cell boundaries of the biological sample;

[0649] Step 1102 , obtaining a second image of the biological sample, where the second image includes stained genetic material of the biological sample;

[0650] Step 1103, obtaining gene expression information of the biological sample;

[0651] Step 1104: register the second image with the first image to obtain a registered image of the second image and the first image;

[0652] Step 1105: registering the second image with the gene expression level information to obtain a registered image of the second image and the gene expression level information;

[0653] Step 1106, obtaining a registered image of the first image and the gene expression quantity information based on the registered image of the second image and the first image, and the registered image of the second image and the gene expression quantity information;

[0654] Step 1107 , obtaining the cell boundaries of the biological sample based on the first image and the registered image of the gene expression information;

[0655] Step 1108 : Acquire gene expression information of cells of the biological sample according to the cell boundaries based on the first image and the registered image of gene expression information.

[0656] In some embodiments, cells in a biological sample are stained with an ssDNA stain and a cell boundary fluorescent stain, wherein the ssDNA stain comprises a buffer and an ssDNA reagent, and the cell boundary fluorescent stain comprises a fluorescent dye. The ssDNA stain can stain the genetic material of the cells, and the cell boundary fluorescent stain can stain the cell boundaries.

[0657] In some embodiments, the cell boundary comprises a cell wall or a cell membrane.

[0658] In some embodiments, the gene expression information of the cells of the tissue sample includes the position information, gene expression level and cell type of each site on the chip carrying the tissue sample.

[0659] In some embodiments, registering the second image with the gene expression level information to obtain a registered image of the second image and the gene expression level information includes:

[0660] The second image and the gene expression level information are registered according to the track line in the second image and the track line of the gene expression level information to obtain a registered image of the second image and the gene expression level information.

[0661] In some embodiments, the tissue sample is stained with a cell staining reagent and then imaged, wherein the imaging obtains a second image of the tissue sample through a FITC channel and a first image of the tissue sample through a DAPI channel, and the cell staining reagent includes: an ssDNA stain and a cell boundary fluorescent stain; the ssDNA stain includes a buffer and an ssDNA reagent; the cell boundary fluorescent stain includes a fluorescent dye.

[0662] It should be noted that the order of the above numbers does not represent the specific execution steps.

[0663] Corresponding to the above-mentioned method for acquiring spatial omics single-cell data, the present invention also provides a device for acquiring spatial omics single-cell data. Since the device embodiments of the present invention correspond to the above-mentioned method embodiments, details not disclosed in the device embodiments can be referred to the above-mentioned method embodiments and will not be further described in this invention.

[0664] FIG47 is a schematic diagram of the structure of a spatial omics single-cell data acquisition device provided in an embodiment of the present application, as shown in FIG47 , comprising:

[0665] an acquisition unit 21, configured to acquire a dyed image of a biological sample and a visualized image of gene expression of the biological sample;

[0666] a registration unit 22, configured to register the staining image with the gene expression visualization image to obtain a registered image;

[0667] a first segmentation unit 23, configured to perform cell segmentation on the registered image to obtain a cell mask image;

[0668] A correction unit 24 is configured to determine the cells to which background gene molecules of unknown cells belong in the gene expression visualization image according to the cell mask image;

[0669] The labeling unit 25 is used to label the cells to obtain a spatial component molecular expression matrix with cell labels.

[0670] Acquire a stained image of a biological sample and obtain a gene expression visualization image of the biological sample; align the stained image with the gene expression visualization image to obtain a registered image; perform cell segmentation on the registered image to obtain a cell mask; determine the cells to which background gene molecules of unknown cells in the gene expression visualization image belong based on the cell mask, and label the cells to obtain a spatial component molecular expression matrix with cell labels. Compared with related technologies, the embodiments of the present application obtain high-precision spatial single-cell data through a processing flow of gene expression matrix diagram, splicing, alignment, segmentation, and correction.

[0671] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the acquiring unit 21 is further configured to:

[0672] Acquire a stained image of the biological sample through a microscope; or

[0673] Acquire a stitched staining image of the biological sample composed of a plurality of sub-staining images through a microscope; or

[0674] A plurality of sub-stained images of the biological sample are acquired through the microscope, and the plurality of sub-stained images are spliced ​​to obtain a spliced ​​stained image.

[0675] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the acquiring unit 21 includes:

[0676] a first acquisition module, configured to acquire a plurality of sub-stained images of the biological sample through a microscope; and perform a first splicing operation on the plurality of sub-stained images of the biological sample to obtain a first stained image;

[0677] a judgment module, configured to judge whether to perform a second splicing on the plurality of sub-stained images of the biological sample according to the image quality of the first stained image;

[0678] a determination module, configured to, if no, determine the first stained image as the stained image;

[0679] A second acquisition module is configured to perform a second splicing on the multiple sub-stained images of the biological sample to obtain the stained image.

[0680] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the first acquisition module is further configured to:

[0681] respectively determining overlapping areas of the plurality of sub-stained images and the template image;

[0682] Cutting the parts of the multiple sub-stained images and the template image determined to be overlapping areas into a plurality of stained image overlapping sub-images and a plurality of template image overlapping sub-images of the same number, respectively, to pair them into a plurality of overlapping sub-image pairs;

[0683] Obtaining the offset between each sub-stained image and the template image by performing frequency domain calculation on the multiple overlapping sub-image pairs; and

[0684] Each of the sub-stained images is translated by the offset amount toward the direction of the template image, and is then spliced ​​together with the template image.

[0685] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the first acquisition module is further configured to:

[0686] acquiring the plurality of sub-stained images and template information about trajectory lines or trajectory points of the biological sample;

[0687] Pre-stitching the multiple sub-stained images to obtain pre-stitched coordinates of the multiple sub-stained images;

[0688] selecting at least one first reference image from the plurality of sub-stained images according to features of the trajectory lines or trajectory points, and deriving a global template according to template information of the biological sample and pre-spliced ​​coordinates of the first reference image; and

[0689] For a second reference image other than the first reference image in the multiple sub-stained images, the offset between the pre-stitched coordinates of the second reference image and the corresponding template coordinates of the second reference image in the global template is calculated, and the coordinates of the second reference image are adjusted based on the offset to stitch together and generate the stitched stained image.

[0690] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the registration unit 22 includes:

[0691] an acquisition module, configured to acquire a trajectory line in the staining image and rotate the staining image based on an angle between the trajectory line and a horizontal direction; wherein the staining image and the gene expression visualization image include a plurality of trajectory lines and / or a plurality of trajectory points, wherein a trajectory point is formed by the intersection of two trajectory lines, and each of the plurality of trajectory lines has a corresponding index number;

[0692] a processing module, configured to magnify or reduce the stained image to obtain a third reference image, wherein the third reference image has the same scale as the gene expression visualization image;

[0693] a first determining module, configured to determine a target offset based on the gene expression visualization image and the third reference image;

[0694] A registration module is configured to move the third reference image based on the target offset to obtain the registered image.

[0695] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the registration unit 22 further includes:

[0696] a second determining module, configured to determine the similarity between the third reference image and the gene expression visualization image after the processing module amplifies or reduces the stained image to obtain a third reference image;

[0697] The coarse registration module is used to take the position corresponding to the maximum similarity as the coarse registration result.

[0698] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the registration unit 22 is further configured to:

[0699] Acquire a staining image and a gene expression visualization image, wherein each of the staining image and the gene expression visualization image includes a plurality of trajectory lines and a plurality of trajectory points, wherein the trajectory point is formed by the intersection of two of the trajectory lines, and each of the trajectory lines has a corresponding index number;

[0700] Preprocessing the stained image to obtain a candidate image set;

[0701] Determine a binary image set corresponding to the candidate image set;

[0702] Determining, based on the similarity between the binary graph set and the gene expression visualization image, trajectory point pairs having a corresponding relationship between the binary graph set and the gene expression visualization image;

[0703] determining offset information based on the trajectory point pairs;

[0704] A target image is processed based on the offset information to obtain a registered image, wherein the target image is an image in the candidate image set corresponding to the track point pair.

[0705] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the apparatus further includes:

[0706] The second segmentation unit 26 is configured to perform tissue segmentation on the registered image to obtain a tissue mask image before the first segmentation unit performs cell segmentation on the registered image to obtain a cell mask image.

[0707] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the second segmentation unit 26 is further configured to:

[0708] Obtaining gene expression levels of the target tissue to be segmented in the registered image and determining a gene expression matrix of the registered image, wherein the gene expression matrix arranges the gene expression levels according to pixel positions;

[0709] Determining image data by assigning the gene expression amount at each pixel position in the gene expression matrix to the corresponding pixel position in the registered image;

[0710] performing image processing on the image data to determine foreground pixels of the image data;

[0711] The connected areas of the image data are marked according to the foreground pixels to obtain the tissue mask image.

[0712] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the apparatus further includes:

[0713] a calculation unit 27 configured to perform tissue segmentation on the registered image in the second segmentation unit to obtain a tissue mask image, and then respectively calculate an outer contour image and an inner contour image of the grayscale image corresponding to the tissue mask image;

[0714] a pairing unit 28 configured to divide the outer contour image and the inner contour image into N segments, and pair each segment of the outer contour image with each segment of the inner contour image to obtain N image combinations, each of which includes a segment of the outer contour image and a segment of the inner contour image;

[0715] The calculation unit 27 is further configured to calculate a pixel density ratio of a segment of the outer contour image to a segment of the inner contour image in each image combination, to obtain N pixel density ratios, wherein the pixel density ratio is a ratio of a pixel point to a total pixel point.

[0716] The calculation unit is further configured to calculate the grayscale image quality corresponding to the tissue mask image according to the pixel density ratio.

[0717] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the first segmentation unit 23 is further configured to:

[0718] Inputting the registered image into the image segmentation model to obtain a segmentation result of each pixel in the registered image, wherein the segmentation result includes the pixel belonging to a cell, a cell outline, or an image background;

[0719] The pixel points belonging to cells or cell outlines in the segmentation results are determined to be pixel points belonging to cells, and the segmentation results of the registered image are output to obtain the cell mask image.

[0720] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the first segmentation unit 23 is further configured to:

[0721] Preprocessing the gene expression visualization image to obtain a preprocessed image;

[0722] Binarizing the preprocessed image to obtain an initial mask image;

[0723] According to the initial mask image, a watershed algorithm based on distance transformation is used to segment the connected domain where cells are adhered, so as to obtain the segmented cell mask image.

[0724] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the first segmentation unit 23 is further configured to:

[0725] Determining, based on the stained image, the labeled regions and cell areas corresponding to each tissue type of the target object;

[0726] Determining the area of ​​a to-be-processed sub-region contained in each of the annotated regions based on the cell area, wherein the area of ​​the to-be-processed sub-region is positively correlated with the cell area, and the to-be-processed sub-region is used to represent the minimum processing unit of the corresponding annotated region;

[0727] According to the area of ​​each of the sub-regions to be processed, the corresponding marked region is gridded to obtain grid coordinates corresponding to each of the marked regions;

[0728] Data corresponding to the gridded coordinates are extracted from the gene expression visualization image and superimposed onto the gridded coordinates to obtain the cell treatment result of the target object.

[0729] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the correction unit 24 is further configured to:

[0730] Based on the cell mask image, each gene expression feature of the gene expression visualization image is determined as a cell gene expression feature located inside the cell and a background gene expression feature located outside the cell;

[0731] For each of the cell gene expression features, a probability distribution model is fitted based on the spatial position and the signal value of the gene expression feature;

[0732] Determining, based on the probability distribution model, a probability value of a target background gene expression feature located within a preset range of a target cell belonging to the target cell;

[0733] Outputting a correction result according to the probability value to correct the background gene expression characteristics.

[0734] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the correction unit 24 is further configured to include:

[0735] Determining the initial gene molecules belonging to the target cells and the background gene molecules of the unknown cells according to the gene expression visualization image;

[0736] Obtaining coordinate information of the initial gene molecule;

[0737] Determining the center point coordinates of the target cell according to the coordinate information;

[0738] Constructing a Voronoi diagram according to the center point coordinates and the coordinate information;

[0739] The cells to which the background gene molecules within the Voronoi diagram belong are determined.

[0740] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the correction unit 24 is further configured to include:

[0741] Determining the initial number of gene molecules belonging to the target cell in the gene expression visualization image and the initial contour area of ​​the target region occupied by the gene molecules belonging to the target cell;

[0742] Determining the gene molecule density of the target cell according to the number of molecules and the initial contour area;

[0743] Determine the number of background gene molecules within a preset region of the target region and the area of ​​the preset region;

[0744] Determining the background gene molecule density according to the number of background gene molecules and the area of ​​the region;

[0745] The cell to which the background gene molecule belongs is determined according to the gene molecule density and the background gene molecule density.

[0746] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the correction unit 24 is further configured to include:

[0747] determining a target region occupied by gene molecules belonging to target cells in the gene expression visualization image;

[0748] Determine the spatial coordinates and gene expression characteristic signal values ​​of background gene molecules of unknown cells within a preset calibration range of the target area;

[0749] Inputting the spatial coordinates and the gene expression characteristic signal values ​​into a probability distribution model to obtain a probability score of the background gene molecule belonging to the target cell; the probability distribution model is fitted based on the spatial coordinates and gene expression characteristic signal values ​​of the gene molecule sample belonging to the target cell;

[0750] The cell to which the background gene molecule belongs is determined according to the probability score.

[0751] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the marking unit 25 is further configured to:

[0752] The cells are stained with ssDNA staining solution and cell boundary fluorescent staining solution, wherein the ssDNA staining solution includes a buffer and an ssDNA reagent, and the cell boundary fluorescent staining solution includes a fluorescent dye.

[0753] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the apparatus further includes a detection unit 29 including:

[0754] Acquiring a staining image or a gene expression visualization image to be tested; and

[0755] Input the staining image or gene expression visualization image to be tested into a directed target detection network to obtain a target detection result,

[0756] The directed target detection network includes:

[0757] The backbone layer is configured to perform feature extraction on the staining image or gene expression visualization image to be tested to obtain multiple feature maps at different scales;

[0758] A Neck layer is configured to receive the multiple feature maps from the Backbone layer and perform cross-scale fusion on the multiple feature maps to obtain multiple fused feature maps; and

[0759] The detection head layer is configured to predict the multiple fused images to obtain a target detection result about the staining image or gene expression visualization image to be tested,

[0760] The target detection result includes: classification information, target frame coordinate prediction information, target prediction information and angle prediction information.

[0761] Optionally, the method further includes a determining unit 210, which is configured to:

[0762] The dyeing image plane where the dyeing image is located includes a first direction and a second direction perpendicular to the first direction, and the dyeing image includes a plurality of first straight track lines transverse to the second direction and a plurality of second straight track lines transverse to the first direction.

[0763] Sampling steps include:

[0764] Apply a first sampling strip to perform equally spaced sampling in the first direction on the stained image along the first direction to determine a first candidate point set, wherein for each equally spaced sampling in the first direction, perform the following sub-steps:

[0765] Accumulating and summing pixel values ​​of pixels within the sampling area of ​​the stained image selected by the first sampling strip during the first direction equally spaced sampling to obtain a series of first pixel value sums; and

[0766] Determine the extreme points of the series of first pixel value sums, and for each determined extreme point: determine a corresponding first candidate point obtained by the first-direction equally-interval sampling, wherein the coordinates of the corresponding first candidate point in the second direction are the coordinates of the second-direction position corresponding to the extreme point in the second direction, and the coordinates of the corresponding first candidate point in the first direction are determined based on the position of the first sampling strip in the first direction during the first-direction equally-interval sampling.

[0767] The first candidate point set is formed by first candidate points obtained by sampling in the first direction at equal intervals each time;

[0768] Apply a second sampling strip to perform equally spaced sampling in the second direction on the stained image along the second direction to determine a second set of candidate points, wherein for each equally spaced sampling in the second direction, perform the following sub-steps:

[0769] Accumulating and summing pixel values ​​of pixels within the sampling area of ​​the stained image selected by the second sampling strip during the second direction equally spaced sampling to obtain a series of second pixel value sums; and

[0770] Determine the extreme points of the series of second pixel value sums, and for each determined extreme point: determine a corresponding second candidate point obtained by the second-direction equally-interval sampling, where the coordinates of the corresponding second candidate point in the first direction are the coordinates of the first-direction position corresponding to the extreme point in the first direction, and the coordinates of the corresponding second candidate point in the second direction are determined based on the position of the second sampling strip in the second direction during the second-direction equally-interval sampling.

[0771] The second candidate point set is formed by the second candidate points obtained by sampling in the second direction at equal intervals each time;

[0772] The grouping steps include:

[0773] dividing the first candidate points in the first candidate point set into a plurality of first candidate point groups; and

[0774] dividing the second candidate points in the second candidate point set into a plurality of second candidate point groups;

[0775] The straight line fitting steps include:

[0776] Performing straight line fitting on the first candidate point in each first candidate point group to obtain a corresponding first straight line analytical expression; and

[0777] Performing straight line fitting on the second candidate points in each second candidate point group to obtain a corresponding second straight line analytical expression; and

[0778] Coordinate calculation steps include:

[0779] Calculate the coordinates of the intersection of each first straight line analytical expression and each second straight line analytical expression.

[0780] Furthermore, in a possible implementation of this embodiment, as shown in FIG48 , the apparatus further includes an analyzing unit 211 configured to:

[0781] Acquiring a first image of a biological sample, the first image including stained cell boundaries of the biological sample;

[0782] acquiring a second image of the biological sample, the second image comprising stained genetic material of the biological sample;

[0783] Obtaining gene expression information of the biological sample;

[0784] registering the second image with the first image to obtain a registered image of the second image and the first image;

[0785] Registering the second image with the gene expression level information to obtain a registered image of the second image and the gene expression level information;

[0786] Acquire a registered image of the first image and the gene expression amount information according to a registered image of the second image and the first image, and a registered image of the second image and the gene expression amount information;

[0787] Acquiring cell boundaries of the biological sample according to the first image and the registered image of the gene expression information;

[0788] The gene expression information of the cells of the biological sample is obtained according to the cell boundaries based on the first image and the registered image of the gene expression information.

[0789] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0790] Figure 49 shows a schematic block diagram of an example electronic device 300 that can be used to implement an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0791] As shown in FIG49 , the device 300 includes a computing unit 301 that can perform various appropriate actions and processes based on a computer program stored in a ROM (Read-Only Memory) 302 or a computer program loaded from a storage unit 308 into a RAM (Random Access Memory) 303. Various programs and data required for the operation of the device 300 can also be stored in the RAM 303. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An I / O (Input / Output) interface 305 is also connected to the bus 304.

[0792] Various components in device 300 are connected to I / O interface 305, including: an input unit 306, such as a keyboard, mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a magnetic disk, optical disk, etc.; and a communication unit 309, such as a network card, modem, wireless communication transceiver, etc. The communication unit 309 allows device 300 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0793] The computing unit 301 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units that run machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the spatial omics single-cell data acquisition method. For example, in some embodiments, the spatial omics single-cell data acquisition method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to execute the aforementioned spatial omics single-cell data acquisition method in any other appropriate manner (for example, by means of firmware).

[0794] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System on Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0795] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0796] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0797] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0798] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0799] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0800] It's important to note that artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). This encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0801] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0802] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A method for acquiring spatial omics single-cell data, characterized in that: include: Obtaining a staining image of the biological sample and obtaining a gene expression visualization image of the biological sample; registering the stained image with the gene expression visualization image to obtain a registered image; performing cell segmentation on the registered image to obtain a cell mask image; The cells to which background gene molecules of unknown cells belong in the gene expression visualization image are determined according to the cell mask image, and the cells to which they belong are marked to obtain a spatial component molecular expression matrix with cell labels.

2. The method according to claim 1, characterized in that The obtaining of a stained image of a biological sample comprises: Acquire a stained image of the biological sample through a microscope; or Acquire a stitched staining image of the biological sample composed of a plurality of sub-staining images through a microscope; or A plurality of sub-stained images of the biological sample are acquired through the microscope, and the plurality of sub-stained images are spliced ​​to obtain a spliced ​​stained image.

3. The method according to claim 2, characterized in that The obtaining of a stained image of the biological sample comprises: Acquire multiple sub-stained images of the biological sample through a microscope; perform a first stitching on the multiple sub-stained images of the biological sample to obtain a first stained image; determining whether to perform a second stitching on the plurality of sub-stained images of the biological sample according to the image quality of the first stained image; If not, determining the first stained image as the stained image; If so, a second stitching is performed on the multiple sub-stained images of the biological sample to obtain the stained image.

4. The method according to claim 3, characterized in that The step of acquiring a plurality of sub-stained images of the biological sample through the microscope and splicing the plurality of sub-stained images to obtain a spliced ​​stained image includes: respectively determining overlapping areas of the plurality of sub-stained images and the template image; Cutting the parts of the multiple sub-stained images and the template image determined to be overlapping areas into a plurality of stained image overlapping sub-images and a plurality of template image overlapping sub-images of the same number, respectively, to pair them into a plurality of overlapping sub-image pairs; Obtaining the offset between each sub-stained image and the template image by performing frequency domain calculation on the multiple overlapping sub-image pairs; and Each of the sub-stained images is translated by the offset amount toward the direction of the template image, and is then spliced ​​together with the template image.

5. The method according to claim 3, characterized in that The step of acquiring a plurality of sub-stained images of the biological sample through the microscope and splicing the plurality of sub-stained images to obtain a spliced ​​stained image includes: acquiring the plurality of sub-stained images and template information about trajectory lines or trajectory points of the biological sample; Pre-stitching the multiple sub-stained images to obtain pre-stitched coordinates of the multiple sub-stained images; selecting at least one first reference image from the plurality of sub-stained images according to features of the trajectory lines or trajectory points, and deriving a global template according to template information of the biological sample and pre-spliced ​​coordinates of the first reference image; and For a second reference image other than the first reference image in the multiple sub-stained images, the offset between the pre-stitched coordinates of the second reference image and the corresponding template coordinates of the second reference image in the global template is calculated, and the coordinates of the second reference image are adjusted based on the offset to stitch together and generate the stitched stained image.

6. The method according to claim 1, characterized in that The registering the stained image with the gene expression visualization image to obtain the registered image comprises: Obtaining a trajectory line in the staining image, and rotating the staining image based on an angle between the trajectory line and a horizontal direction; wherein the staining image and the gene expression visualization image include a plurality of trajectory lines and / or a plurality of trajectory points, wherein the trajectory point is formed by the intersection of two trajectory lines, and each of the plurality of trajectory lines has a corresponding index number; Enlarging or reducing the stained image to obtain a third reference image, wherein the third reference image has the same scale as the gene expression visualization image; determining a target offset based on the gene expression visualization image and the third reference image; The third reference image is moved based on the target offset to obtain the registered image.

7. The method according to claim 6, characterized in that After the stained image is magnified or reduced to obtain a third reference image, the method further includes: determining a similarity between the third reference image and the gene expression visualization image; The position corresponding to the maximum similarity value is taken as the rough registration result.

8. The method according to claim 1, characterized in that Registering the stained image with the gene expression visualization image to obtain a registered image includes: Preprocessing the stained image to obtain a candidate image set; Determine a binary image set corresponding to the candidate image set; Determining, based on the similarity between the binary graph set and the gene expression visualization image, trajectory point pairs having a corresponding relationship between the binary graph set and the gene expression visualization image; determining offset information based on the trajectory point pairs; A target image is processed based on the offset information to obtain a registered image, wherein the target image is an image in the candidate image set corresponding to the track point pair.

9. The method according to claim 1, characterized in that Before performing cell segmentation on the registered image to obtain a cell mask image, the method further includes: Tissue segmentation is performed on the registered image to obtain a tissue mask image.

10. The method according to claim 9, characterized in that The performing tissue segmentation on the registered image to obtain a tissue mask image includes: Obtaining gene expression levels of the target tissue to be segmented in the registered image and determining a gene expression matrix of the registered image, wherein the gene expression matrix arranges the gene expression levels according to pixel positions; Determining image data by assigning the gene expression amount at each pixel position in the gene expression matrix to the corresponding pixel position in the registered image; performing image processing on the image data to determine foreground pixels of the image data; The connected areas of the image data are marked according to the foreground pixels to obtain the tissue mask image.

11. The method according to claim 10, characterized in that After performing tissue segmentation on the registered image to obtain a tissue mask image, the method further includes: Calculate the outer contour image and inner contour image of the grayscale image corresponding to the tissue mask image respectively; Dividing the outer contour image and the inner contour image into N segments respectively, and pairing each segment of the outer contour image with each segment of the inner contour image to obtain N image combinations, each of which includes a segment of the outer contour image and a segment of the inner contour image; Calculating the pixel density ratio of a segment of the outer contour image to a segment of the inner contour image in each of the image combinations to obtain N pixel density ratios, where the pixel density ratio is the ratio of the pixel proportion of the segment of the outer contour image to the segment of the inner contour image in each of the image combinations, and the pixel proportion is the ratio of the pixel points to the total pixel points; The grayscale image quality corresponding to the tissue mask image is calculated according to the pixel density ratio.

12. The method according to claim 1, characterized in that The step of performing cell segmentation on the registered image to obtain a cell mask image comprises: Inputting the registered image into the image segmentation model to obtain a segmentation result of each pixel in the registered image, wherein the segmentation result includes pixel points belonging to cells; The pixel points belonging to cells or the pixel points of cell outlines in the segmentation results are determined to belong to cells, and the segmentation result of the registered image is output to obtain the cell mask image.

13. The method according to claim 1, wherein The step of performing cell segmentation on the registered image to obtain a cell mask image comprises: Preprocessing the gene expression visualization image to obtain a preprocessed image; Binarizing the preprocessed image to obtain an initial mask image; According to the initial mask image, a watershed algorithm based on distance transformation is used to segment the connected domain where cells are adhered, so as to obtain the segmented cell mask image.

14. The method according to claim 1, wherein The step of performing cell segmentation on the registered image to obtain a cell mask image comprises: Determining, based on the stained image, the labeled regions and cell areas corresponding to each tissue type of the target object; Determining the area of ​​a to-be-processed sub-region contained in each of the annotated regions based on the cell area, wherein the area of ​​the to-be-processed sub-region is positively correlated with the cell area, and the to-be-processed sub-region is used to represent the minimum processing unit of the corresponding annotated region; According to the area of ​​each of the sub-regions to be processed, the corresponding marked region is gridded to obtain grid coordinates corresponding to each of the marked regions; Data corresponding to the gridded coordinates are extracted from the gene expression visualization image and superimposed onto the gridded coordinates to obtain the cell treatment result of the target object.

15. The method according to claim 1, wherein The determining, based on the cell mask image, of cells to which background gene molecules of unknown cells belong in the gene expression visualization image belong comprises: Based on the cell mask image, each gene expression feature of the gene expression visualization image is determined as a cell gene expression feature located inside the cell and a background gene expression feature located outside the cell; For each of the cell gene expression features, a probability distribution model is fitted based on the spatial position and the signal value of the gene expression feature; Determining, based on the probability distribution model, a probability value of a target background gene expression feature located within a preset range of a target cell belonging to the target cell; Outputting a correction result according to the probability value to correct the background gene expression characteristics.

16. The method according to claim 1, wherein The determining, based on the cell mask image, of cells to which background gene molecules of unknown cells belong in the gene expression visualization image belong comprises: Determining the initial gene molecules belonging to the target cells and the background gene molecules of the unknown cells according to the gene expression visualization image; Obtaining coordinate information of the initial gene molecule; Determining the center point coordinates of the target cell according to the coordinate information; Constructing a Voronoi diagram according to the center point coordinates and the coordinate information; The cells to which the background gene molecules within the Voronoi diagram belong are determined.

17. The method according to claim 1, wherein The determining, based on the cell mask image, of cells to which background gene molecules of unknown cells belong in the gene expression visualization image belong comprises: Determining the initial number of gene molecules belonging to the target cell in the gene expression visualization image and the initial contour area of ​​the target region occupied by the gene molecules belonging to the target cell; determining the gene molecule density of the target cell according to the number of molecules and the initial contour area; Determine the number of background gene molecules within a preset region of the target region and the area of ​​the preset region; Determining the background gene molecule density according to the number of background gene molecules and the area of ​​the region; The cell to which the background gene molecule belongs is determined according to the gene molecule density and the background gene molecule density.

18. The method according to claim 1, wherein The determining, based on the cell mask image, of cells to which background gene molecules of unknown cells belong in the gene expression visualization image belong comprises: determining a target region occupied by gene molecules belonging to target cells in the gene expression visualization image; Determine the spatial coordinates and gene expression characteristic signal values ​​of background gene molecules of unknown cells within a preset calibration range of the target area; Inputting the spatial coordinates and the gene expression characteristic signal values ​​into a probability distribution model to obtain a probability score of the background gene molecule belonging to the target cell; the probability distribution model is fitted based on the spatial coordinates and gene expression characteristic signal values ​​of the gene molecule sample belonging to the target cell; The cell to which the background gene molecule belongs is determined according to the probability score.

19. The method according to claim 1, wherein The cell markings include: The cells are stained with ssDNA staining solution and cell boundary fluorescent staining solution, wherein the ssDNA staining solution includes a buffer and an ssDNA reagent, and the cell boundary fluorescent staining solution includes a fluorescent dye.

20. The method according to claim 5 or 6, characterized in that The method further comprises: Acquiring a staining image or a gene expression visualization image to be tested; and Input the staining image or gene expression visualization image to be tested into a directed target detection network to obtain a target detection result, The directed target detection network includes: The backbone layer is configured to perform feature extraction on the staining image or gene expression visualization image to be tested to obtain multiple feature maps at different scales; A Neck layer is configured to receive the multiple feature maps from the Backbone layer and perform cross-scale fusion on the multiple feature maps to obtain multiple fused feature maps; and The detection head layer is configured to predict the multiple fused images to obtain a target detection result about the staining image or gene expression visualization image to be tested, The target detection result includes: target frame coordinate prediction information, target prediction information and angle prediction information.

21. The method according to claim 5 or 6, characterized in that The dyeing image plane where the dyeing image is located includes a first direction and a second direction perpendicular to the first direction, and the dyeing image includes a plurality of first straight track lines transverse to the second direction and a plurality of second straight track lines transverse to the first direction. The method for determining the coordinates of points in the stained image includes: Sampling steps include: Apply a first sampling strip to perform equally spaced sampling in the first direction on the stained image along the first direction to determine a first candidate point set, wherein for each equally spaced sampling in the first direction, perform the following sub-steps: Accumulating and summing pixel values ​​of pixels within the sampling area of ​​the stained image selected by the first sampling strip during the first direction equally spaced sampling to obtain a series of first pixel value sums; and Determine the extreme points of the series of first pixel value sums, and for each determined extreme point: determine a corresponding first candidate point obtained by the first-direction equally-interval sampling, wherein the coordinates of the corresponding first candidate point in the second direction are the coordinates of the second-direction position corresponding to the extreme point in the second direction, and the coordinates of the corresponding first candidate point in the first direction are determined based on the position of the first sampling strip in the first direction during the first-direction equally-interval sampling. The first candidate point set is formed by first candidate points obtained by sampling in the first direction at equal intervals each time; Apply a second sampling strip to perform equally spaced sampling in the second direction on the stained image along the second direction to determine a second set of candidate points, wherein for each equally spaced sampling in the second direction, perform the following sub-steps: Accumulating and summing pixel values ​​of pixels within the sampling area of ​​the stained image selected by the second sampling strip during the second direction equally spaced sampling to obtain a series of second pixel value sums; and Determine the extreme points of the series of second pixel value sums, and for each determined extreme point: determine a corresponding second candidate point obtained by the second-direction equally-interval sampling, where the coordinates of the corresponding second candidate point in the first direction are the coordinates of the first-direction position corresponding to the extreme point in the first direction, and the coordinates of the corresponding second candidate point in the second direction are determined based on the position of the second sampling strip in the second direction during the second-direction equally-interval sampling. The second candidate point set is formed by the second candidate points obtained by sampling in the second direction at equal intervals each time; The grouping steps include: dividing the first candidate points in the first candidate point set into a plurality of first candidate point groups; and dividing the second candidate points in the second candidate point set into a plurality of second candidate point groups; The straight line fitting steps include: Performing straight line fitting on the first candidate point in each first candidate point group to obtain a corresponding first straight line analytical expression; and Performing straight line fitting on the second candidate points in each second candidate point group to obtain a corresponding second straight line analytical expression; and Coordinate calculation steps include: Calculate the coordinates of the intersection of each first straight line analytical expression and each second straight line analytical expression.

22. The method according to claim 19, wherein After labeling the cells to obtain a spatial component molecular expression matrix with cell labels, the method further includes: Acquiring a first image of a biological sample, the first image including stained cell boundaries of the biological sample; acquiring a second image of the biological sample, the second image comprising stained genetic material of the biological sample; Obtaining gene expression information of the biological sample; registering the second image with the first image to obtain a registered image of the second image and the first image; Registering the second image with the gene expression level information to obtain a registered image of the second image and the gene expression level information; Acquire a registered image of the first image and the gene expression amount information according to a registered image of the second image and the first image, and a registered image of the second image and the gene expression amount information; Acquiring cell boundaries of the biological sample according to the first image and the registered image of the gene expression information; The gene expression information of the cells of the biological sample is obtained according to the cell boundaries based on the first image and the registered image of the gene expression information.

23. A spatial omics single-cell data acquisition device, characterized in that: include: an acquisition unit, configured to acquire a stained image of the biological sample and a visualized image of gene expression of the biological sample; a registration unit, configured to register the stained image with the gene expression visualization image to obtain a registered image; a first segmentation unit, configured to perform cell segmentation on the registered image to obtain a cell mask image; a correction unit, configured to determine, based on the cell mask image, to which the background gene molecules of unknown to which cells belong in the gene expression visualization image belong; The labeling unit is used to label the cells to obtain a spatial component molecular expression matrix with cell labels.

24. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 22.

25. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-22.

26. A computer program product, characterized in that A computer program is included which, when executed by a processor, implements the method according to any one of claims 1 to 22.

Images