Scanning image correction method, apparatus and device

Abstract

The application provides a scanning image correction method, device and equipment, relates to the technical field of spatial transcriptome sequencing, and comprises the following steps: in the state that a tissue slice is attached to a biochip, a plurality of field scanning images of the biochip are collected, the biochip comprises a substrate pattern, the substrate pattern comprises a plurality of key points, and there is an overlapping area between any adjacent field scanning images; the plurality of field scanning images are fused according to the overlapping area between any adjacent field scanning images, so as to obtain a full-width scanning image of the biochip; the first position information of the plurality of key points in the full-width scanning image is determined according to the substrate pattern; and the full-width scanning image is corrected according to the first position information and the substrate pattern, so as to obtain a target full-width scanning image. The scheme of the application realizes accurate alignment of gene expression information captured by the biochip and a region represented by the full-width scanning image.

Description

Technical Field

[0001] This application relates to the field of spatial transcriptome sequencing technology, and in particular to a scanning image correction method, apparatus, and device. Background Technology

[0002] In multicellular organisms, gene expression in a single cell occurs strictly in a specific temporal and spatial order; that is, gene expression is time-specific and space-specific.

[0003] For spatial specificity, current methods typically employ spatial transcriptome sequencing to perform in situ expression and histological analysis on tissue sections on biochips, locating gene expression information to the original spatial location of the gene. After obtaining gene expression information at each spatial location, further analysis is often required, comparing it with high-resolution tissue staining or fluorescence images.

[0004] However, the full-frame scan images acquired by current scanning instruments are more or less misaligned or overlapping, which causes the gene expression information captured by the biochip and the region represented by the full-frame scan image to be misaligned, which has an adverse effect on the accuracy of spatial location data. Summary of the Invention

[0005] This application provides a scanning image correction method, apparatus, and device to solve the problem that the gene expression information captured by current biochips and the regions represented by full-frame scan images cannot be aligned.

[0006] In a first aspect, this application provides a scanned image correction method, comprising:

[0007] With tissue slices attached to the biochip, multiple field-of-view scan images of the biochip are acquired. The biochip includes a substrate pattern, which includes multiple key points. There is an overlapping area between any adjacent field-of-view scan images.

[0008] Based on the overlapping area between any adjacent field-of-view scan images, the multiple field-of-view scan images are fused to obtain a full-frame scan image of the biochip.

[0009] Based on the substrate pattern, determine the first position information of the plurality of key points in the full-frame scan image;

[0010] Based on the first position information and the substrate pattern, the full-frame scan image is corrected to obtain the target full-frame scan image.

[0011] In one possible implementation, the step of fusing the multiple field-of-view scan images based on the overlapping area between any adjacent field-of-view scan images to obtain a full-frame scan image of the biochip includes:

[0012] The relative positions between any two adjacent visual field scan images are determined based on the overlapping area between them.

[0013] The multiple field-of-view scan images are fused based on their relative positions to obtain the full-frame scan image.

[0014] In one possible implementation, determining the relative position between any two adjacent visual field scan images based on the overlapping area between them includes:

[0015] For any first field-of-view scan image among the multiple field-of-view scan images, determine a second field-of-view scan image that is adjacent to the first field-of-view scan image;

[0016] Based on the overlapping area between the first field-of-view scan image and the second field-of-view scan image, a two-dimensional window sliding process is performed on the first field-of-view scan image and the second field-of-view scan image to determine the matching parameters between the first field-of-view scan image and the second field-of-view scan image;

[0017] The relative positions between the first field-of-view scan image and the second field-of-view scan image are determined based on the matching parameters.

[0018] In one possible implementation, the step of fusing the multiple field-of-view scan images based on the relative positions between any adjacent field-of-view scan images to obtain the full-frame scan image includes:

[0019] Among the plurality of field-of-view scan images, the position of a reference field-of-view scan image is determined, wherein the reference field-of-view scan image is any one of the plurality of field-of-view scan images;

[0020] Based on the position of the reference field-of-view scan image and the relative position between any adjacent field-of-view scan images, the positions of other field-of-view scan images are determined. The other field-of-view scan images are the field-of-view scan images other than the reference field-of-view scan image among the multiple field-of-view scan images.

[0021] Based on the position of the reference field-of-view scan image and the positions of the other field-of-view scan images, the multiple field-of-view scan images are fused to obtain the full-frame scan image.

[0022] In one possible implementation, determining the first position information of the plurality of key points in the full-frame scan image based on the substrate pattern includes:

[0023] Determine the key point recognition model corresponding to the substrate pattern;

[0024] The full-frame scan image is input into the key point recognition model to obtain the first location information;

[0025] The key point recognition model is trained based on multiple sets of training samples. Each set of training samples includes a full-frame scan image of the sample and the corresponding label information. The full-frame scan image of the sample includes the substrate pattern, and the label information includes the position information of multiple key points in the full-frame scan image of the sample.

[0026] In one possible implementation, the step of correcting the full-frame scan image based on the first position information and the substrate pattern to obtain the target full-frame scan image includes:

[0027] Based on the substrate pattern and the size of the full-frame scan image, determine the second position information of the plurality of key points in the full-frame scan image;

[0028] Based on the first location information and the second location information, the full-frame scan image is corrected to obtain the target full-frame scan image.

[0029] In one possible implementation, the step of performing correction processing on the full-frame scan image based on the first location information and the second location information to obtain the target full-frame scan image includes:

[0030] Based on the first location information and the second location information, determine the correction transformation matrix;

[0031] Based on the correction transformation matrix, the full-frame scan image is subjected to mapping transformation processing to obtain the target full-frame scan image.

[0032] Secondly, this application provides a scanned image correction apparatus, comprising:

[0033] The acquisition module is used to acquire multiple field-of-view scan images of the biochip while the tissue slices are attached to the biochip. The biochip includes a substrate pattern, which includes multiple key points. There is an overlapping area between any two adjacent field-of-view scan images.

[0034] The processing module is used to fuse the multiple field-of-view scan images according to the overlapping area between any adjacent field-of-view scan images to obtain the full-frame scan image of the biochip.

[0035] The determining module is used to determine the first position information of the plurality of key points in the full-frame scan image based on the substrate pattern;

[0036] The correction module is used to perform correction processing on the full-frame scan image based on the first position information and the substrate pattern to obtain the target full-frame scan image.

[0037] In one possible implementation, the processing module is specifically used for:

[0038] The relative positions between any two adjacent visual field scan images are determined based on the overlapping area between them.

[0039] The multiple field-of-view scan images are fused based on their relative positions to obtain the full-frame scan image.

[0040] In one possible implementation, the processing module is specifically used for:

[0041] For any first field-of-view scan image among the multiple field-of-view scan images, determine a second field-of-view scan image that is adjacent to the first field-of-view scan image;

[0042] Based on the overlapping area between the first field-of-view scan image and the second field-of-view scan image, a two-dimensional window sliding process is performed on the first field-of-view scan image and the second field-of-view scan image to determine the matching parameters between the first field-of-view scan image and the second field-of-view scan image;

[0043] The relative positions between the first field-of-view scan image and the second field-of-view scan image are determined based on the matching parameters.

[0044] In one possible implementation, the processing module is specifically used for:

[0045] Among the plurality of field-of-view scan images, the position of a reference field-of-view scan image is determined, wherein the reference field-of-view scan image is any one of the plurality of field-of-view scan images;

[0046] Based on the position of the reference field-of-view scan image and the relative position between any adjacent field-of-view scan images, the positions of other field-of-view scan images are determined. The other field-of-view scan images are the field-of-view scan images other than the reference field-of-view scan image among the multiple field-of-view scan images.

[0047] Based on the position of the reference field-of-view scan image and the positions of the other field-of-view scan images, the multiple field-of-view scan images are fused to obtain the full-frame scan image.

[0048] In one possible implementation, the determining module is specifically used for:

[0049] Determine the key point recognition model corresponding to the substrate pattern;

[0050] The full-frame scan image is input into the key point recognition model to obtain the first location information;

[0051] The key point recognition model is trained based on multiple sets of training samples. Each set of training samples includes a full-frame scan image of the sample and the corresponding label information. The full-frame scan image of the sample includes the substrate pattern, and the label information includes the position information of multiple key points in the full-frame scan image of the sample.

[0052] In one possible implementation, the correction module is specifically used for:

[0053] Based on the substrate pattern and the size of the full-frame scan image, determine the second position information of the plurality of key points in the full-frame scan image;

[0054] Based on the first location information and the second location information, the full-frame scan image is corrected to obtain the target full-frame scan image.

[0055] In one possible implementation, the correction module is specifically used for:

[0056] Based on the first location information and the second location information, determine the correction transformation matrix;

[0057] Based on the correction transformation matrix, the full-frame scan image is subjected to mapping transformation processing to obtain the target full-frame scan image.

[0058] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the scan image correction method as described in any of the first aspects.

[0059] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the scan image correction method as described in any of the first aspects.

[0060] The scanning image correction method, apparatus, and device provided in this application acquire multiple field-of-view scanning images of a biochip with tissue slices attached to it. The biochip includes a substrate pattern containing multiple key points, and overlapping areas exist between any adjacent field-of-view scanning images. Then, based on the overlapping areas between any adjacent field-of-view scanning images, the multiple field-of-view scanning images are fused to obtain a full-frame scanning image of the biochip, achieving image stitching. Further, based on the substrate pattern, the first position information of multiple key points in the full-frame scanning image is determined. Then, based on the first position information and the substrate pattern, the full-frame scanning image is corrected to obtain a target full-frame scanning image. By using the first and second position information of multiple key points in the full-frame scanning image, image errors can be corrected, achieving accurate alignment between the gene expression information captured by the biochip and the region represented by the full-frame scanning image. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 A schematic flowchart illustrating the scanned image correction method provided in an embodiment of this application;

[0063] Figure 2 A schematic diagram of a substrate pattern provided for an embodiment of this application;

[0064] Figure 3 A schematic diagram of the field-of-view scanning image provided in an embodiment of this application;

[0065] Figure 4 A schematic diagram of a field-of-view scanning image provided in an embodiment of this application;

[0066] Figure 5 A schematic diagram of the fusion process provided in the embodiments of this application;

[0067] Figure 6 Schematic diagrams of several key points provided in the embodiments of this application;

[0068] Figure 7 A schematic diagram illustrating the fusion processing of field-of-view scan images provided in an embodiment of this application;

[0069] Figure 8 This is a schematic flowchart of the correction process provided in the embodiments of this application;

[0070] Figure 9 A schematic diagram of a target full-frame scan image provided in an embodiment of this application;

[0071] Figure 10 An enlarged schematic diagram of the target full-frame scan image after overlaying a substrate pattern, provided in an embodiment of this application;

[0072] Figure 11 This is a schematic diagram of the structure of the scanned image correction device provided in the embodiments of this application;

[0073] Figure 12 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0075] For single cells, if it is necessary to explore the heterogeneity between cells, the cells are usually dissociated into single-cell suspensions, and then single-cell library construction is achieved using single-cell separation techniques, such as micropores, microplates, droplets, etc.

[0076] Single-cell library construction using techniques such as single-cell isolation results in the loss of cellular spatial information within the tissue. This spatial information is crucial for practical research, particularly in understanding cell fate mechanisms and cell lineage. Therefore, developing spatial transcriptomics technology to preserve cellular spatial information is essential for studying such cell states.

[0077] In multicellular organisms, gene expression in a single cell occurs strictly in a specific temporal and spatial order; that is, gene expression is time-specific and space-specific.

[0078] Regarding time specificity, single-cell transcriptome sequencing technology can be used to analyze cell types and gene expression patterns over time by collecting samples from different time points.

[0079] However, obtaining the corresponding spatial information of cells is relatively difficult in terms of spatial specificity. Conventional transcriptome sequencing and single-cell transcriptome sequencing are both insufficient to reconstruct the original spatial information of the cells, while in situ hybridization technology is difficult to achieve high-throughput detection. Therefore, spatial transcriptome sequencing technology has emerged. Spatial transcriptome sequencing technology locates gene expression information to the original spatial location of genes by performing in situ expression analysis and histological analysis on tissue sections on biochips.

[0080] After obtaining gene expression information at each spatial location, further analysis is often required by comparing it with high-resolution tissue staining images or fluorescence images. Specifically, the tissue on the biochip is first scanned using a scanning instrument to obtain the corresponding full-frame scan image, and then the full-frame scan image is aligned with the gene expression information captured on the biochip.

[0081] However, current scanning instruments acquire full-frame scan images that are more or less misaligned or overlapping, causing misalignment between the gene expression information captured by the biochip and the region represented by the full-frame scan image. Especially in high-resolution biochips, even slight misalignments can have a significant impact on the accuracy of the entire data.

[0082] Based on this, embodiments of this application provide a scanning image correction method to achieve alignment between the gene expression information captured by the biochip and the region represented by the full-frame scan image. The solutions of embodiments of this application are described below with reference to the accompanying drawings.

[0083] Figure 1 This is a flowchart illustrating the scanned image correction method provided in the embodiments of this application, as shown below. Figure 1 As shown, it includes:

[0084] S11, with the tissue slice attached to the biochip, acquire multiple field-of-view scan images of the biochip. The biochip includes a substrate pattern, which includes multiple key points. There is an overlapping area between any adjacent field-of-view scan images.

[0085] Tissue sections are slides made from biological tissues, such as pathological tissue sections. After obtaining the tissue sections, they are attached to the biochip according to a professional slicing procedure, then treated with appropriate reagents, and the tissue sections are fixed for scanning.

[0086] Biochips are devices used to capture gene expression information in cells. They have specific substrate patterns, which can be the same or different on different biochips. Substrate patterns include, but are not limited to, circular arrays, square arrays, etc.

[0087] Figure 2A schematic diagram of a substrate pattern provided in an embodiment of this application, such as... Figure 2 As shown, the biochip includes a substrate pattern, in Figure 2 In the example, region 21 of the biochip is magnified as shown in region 22, and a sub-region 23 of region 22 is magnified as shown in sub-region 24. Figure 2 As can be seen, the biochip includes alternating horizontal and vertical lines, which together form the substrate pattern on the biochip.

[0088] The substrate pattern includes multiple key points, the number of which can be set as needed, and the position of these key points within the substrate pattern can also be set as required. For example, in... Figure 2 The example illustrates using the intersections of alternating horizontal and vertical lines as key points in the substrate pattern, such as... Figure 2 Points A, B, C, and D in the diagram are all key points in the substrate pattern.

[0089] Biochips contain multiple biological barcodes, also known as barcoads, DNA barcodes, etc. Figure 2 In the example of sub-region 24, each small dot represents a bio-barcode in the biochip.

[0090] The function of biocodes is to capture gene expression information of cells. In this embodiment, the biocodes on the biochip are used to capture the gene expression information of each cell in a tissue slice. Before the tissue slice is attached to the biochip, the biocodes on the biochip have already captured the gene expression information of each cell during tissue switching.

[0091] After the tissue slices are attached to and fixed onto the biochip, multiple field-of-view scan images of the biochip can be acquired. The fields of view of different field-of-view scan images are different, and the field of view of any field-of-view scan image is a local area of ​​the biochip. The fields of view of multiple field-of-view scan images together constitute the global area of ​​the entire biochip, and there are overlapping areas between any adjacent field-of-view scan images.

[0092] Specifically, after the tissue sections are attached to and fixed onto the biochip, a specialized scanner scans the tissue sections to obtain multiple field-of-view (FOV) images. During the scanning process, the scanner's position changes accordingly, and the field of view of the resulting FPV images changes accordingly. Optionally, the FPV images are bright-field digital microscopy images.

[0093] The following is combined Figure 3 An exemplary procedure for scanning tissue sections using a scanner is provided.

[0094] Figure 3 This is a schematic diagram of the field-of-view scanning image provided in an embodiment of this application, as shown below. Figure 3 As shown, it includes a biochip 31 and a scanner (not shown in the figure).

[0095] Initially, the scanner scans a local region 311 in the biochip 31 to obtain a visual field scan image corresponding to local region 311. Then, the scanner moves its position to scan a local region 312 in the biochip 31 to obtain a visual field scan image corresponding to local region 312, and so on. Local regions 311 and 312 are adjacent, and there is an overlapping area 300 between the visual field scan images corresponding to local regions 311 and 312.

[0096] The scanning order of the scanner can be preset. The scanner scans different local areas in the biochip 31 according to the preset scanning order to obtain corresponding field-of-view scan images.

[0097] S12, based on the overlapping area between any adjacent field-of-view scan images, multiple field-of-view scan images are fused to obtain a full-frame scan image of the biochip.

[0098] Figure 4 A schematic diagram of the field-of-view scanning image provided in the embodiments of this application, such as... Figure 4 As shown, multiple field-of-view scan images can be obtained after scanning with a scanner. These multiple field-of-view scan images can then be rearranged according to the scanning order of the scanner and numbered according to their positions so that the positions of the field-of-view scan images correspond to their actual spatial positions on the biochip.

[0099] For any two adjacent visual field scan images among multiple visual field scan images, since there is an overlapping area between the two visual field scan images, the two visual field scan images can be fused based on the overlapping area to obtain a fused image.

[0100] The fusion process can be found in [reference]. Figure 5 .

[0101] Figure 5 A schematic diagram of the fusion process provided in the embodiments of this application, such as Figure 5 As shown, the fusion process between four field-of-view scan images is illustrated. These four field-of-view scan images are image 51, image 52, image 53, and image 54.

[0102] Among them, images 51 and 52 are adjacent, with overlapping areas on the right side of image 51 and the left side of image 52; images 51 and 53 are adjacent, with overlapping areas on the bottom edge of image 51 and the top edge of image 53; images 53 and 54 are adjacent, with overlapping areas on the right side of image 53 and the left side of image 54; images 52 and 54 are adjacent, with overlapping areas on the bottom edge of image 52 and the top edge of image 54. Each overlapping area is... Figure 5 The diagram is shown in the image.

[0103] The fusion process involves combining scanned images from different fields of view into a single image based on overlapping regions. After fusion, the original overlapping regions are merged into one. For example, in... Figure 5 In the process, images 51, 52, 53 and 54 are fused together to obtain image 55.

[0104] For each adjacent field-of-view scan image in these multiple field-of-view scan images, the above-described method can be used for fusion processing. After all the field-of-view scan images have been fused, the corresponding full-frame scan image can be obtained. The field of view of the full-frame scan image is the global area of ​​the biochip, including complete tissue slices.

[0105] S13, Based on the substrate pattern, determine the first position information of multiple key points in the full-frame scan image.

[0106] In one possible implementation, for the obtained full-frame scan image, the position information of the substrate pattern in the full-frame scan image can be identified, and then the position information of multiple key points relative to the substrate pattern can be combined to determine the first position information of multiple key points in the full-frame scan image.

[0107] In one possible implementation, the first location information of multiple key points in a full-frame scan image can also be determined using a key point recognition model.

[0108] Specifically, the key point recognition model corresponding to the substrate pattern is first determined. This key point recognition model is pre-trained using multiple sets of training samples. Each set of training samples includes a full-frame scan image of the sample and the corresponding label information.

[0109] A full-frame scan image of the sample is obtained by fusing tissue slices scanned by a scanner. The tissue slices are attached to a biochip, which includes the substrate pattern. Therefore, the full-frame scan image also includes this substrate pattern. This substrate pattern includes multiple key points, and the label information includes the location information of these key points in the full-frame scan image. The label information can be in the form of an image, data, or both. The image form refers to an image obtained by labeling the key points in the full-frame scan image, while the data form refers to the coordinate data used to label the locations of multiple key points in the full-frame scan image.

[0110] Figure 6 A schematic diagram of several key points provided for embodiments of this application, such as... Figure 6 As shown, several key points on the substrate pattern are illustrated. Figure 6 In this context, the key point is the intersection of the lines, that is... Figure 6 The black dots in the image.

[0111] exist Figure 6 The example shown is an image obtained by annotating a full-frame scanned image of a sample (a part of the image). It can be used directly as the label information of the full-frame scanned image of the sample, or the coordinate information of multiple key points can be determined based on the annotated image and used as the label information of the full-frame scanned image of the sample. This application does not limit this.

[0112] S14, based on the first position information and the substrate pattern, the full-frame scan image is corrected to obtain the target full-frame scan image.

[0113] The first positional information refers to the location information of multiple key points actually determined on the full-frame scan image. The substrate pattern includes multiple key points; based on the design of the substrate pattern, it is possible to determine which locations contain key points, the number of key points, and other information. Based on the information of multiple key points fed back from the substrate pattern design, and the first positional information of these key points in the full-frame scan image, the full-frame scan image is corrected to obtain the target full-frame scan image. Since the target full-frame scan image is obtained by correcting the full-frame scan image based on the first positional information and the substrate pattern, the target full-frame scan image and the biochip are aligned. Therefore, accurate alignment of the gene expression information captured by the biochip with the region represented by the full-frame scan image is achieved.

[0114] Based on any of the above embodiments, the solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0115] Figure 7This is a schematic diagram of the fusion processing of field-of-view scan images provided in an embodiment of this application, as shown below. Figure 7 As shown, it includes:

[0116] S71, determine the relative position between any two adjacent field-of-view scan images based on the overlapping area between any two adjacent field-of-view scan images.

[0117] For any first field-of-view scan image among multiple field-of-view scan images, a second field-of-view scan image adjacent to the first field-of-view scan image is first determined. The number of second field-of-view scan images adjacent to the first field-of-view scan image may be one or more. In one possible implementation, the second field-of-view scan image may be all field-of-view scan images adjacent to the first field-of-view scan image, or it may be a portion of the field-of-view scan images adjacent to the first field-of-view scan image, such as field-of-view scan images located to the left and / or above the first field-of-view scan image. This application does not limit this specific implementation.

[0118] After determining the second field-of-view scan image, a two-dimensional window sliding process is performed on the first and second field-of-view scan images based on the overlapping area between them to determine the matching parameters between the first and second field-of-view scan images. These matching parameters indicate the optimal matching position between the first and second field-of-view scan images.

[0119] The process of sliding a two-dimensional window can be, for example, by first overlapping the first and second visual field scan images, and then sliding the second visual field scan image according to a preset step size, causing a change in the relative positions of the first and second visual field scan images. Next, a matching process is performed on the portions of the first and second visual field scan images that are in the same area (i.e., the overlapping region), determining the similarity between the portions of the first and second visual field scan images in that region. If the similarity exceeds a preset threshold, this matching position is determined as the optimal matching position between the first and second visual field scan images. Based on the current positions of the first and second visual field scan images, their relative positions can be determined.

[0120] Relative positions can be indicated in the form of relative coordinates. For example, the upper left corner of the first field-of-view scan image can be used as a reference, and the relative coordinates of the upper left corner of the second field-of-view scan image relative to the upper left corner of the first field-of-view scan image can be determined based on the relative position between the first and second field-of-view scan images. Once the position of the first field-of-view scan image is determined, the position of the second field-of-view scan image can be determined based on these relative coordinates. For example, the upper right corner of the first field-of-view scan image can also be used as a reference, and the relative coordinates of the upper left corner of the second field-of-view scan image relative to the upper right corner of the first field-of-view scan image can be determined based on the relative position between the first and second field-of-view scan images; or the lower left corner of the first field-of-view scan image can be used as a reference, and the relative coordinates of the lower left corner of the second field-of-view scan image relative to the lower left corner of the first field-of-view scan image can be determined based on the relative position between the first and second field-of-view scan images, and so on. This application does not limit the specific implementation of these methods.

[0121] S72, based on the relative positions between any two adjacent field-of-view scan images, fuses multiple field-of-view scan images to obtain a full-frame scan image.

[0122] In a multi-field-of-view (Field-of-view) scan, the position of a reference Field-of-view (Field-of-view) scan is first determined. The reference Field-of-view (Field-of-view) scan can be any one of the multiple Field-of-view (Field-of-view) scans, such as the first, second, third, etc. The position of the reference Field-of-view (Field-of-view) scan can also be represented by its coordinates. These coordinates can be the coordinates of a point on the reference Field-of-view (Field-of-view) scan, such as the coordinates of the upper left corner, upper right corner, lower left corner, lower right corner, or the midpoint, etc.

[0123] After determining the position of the reference field-of-view scan image, since the relative positions between any two adjacent field-of-view scan images have already been determined, the positions of other field-of-view scan images can be determined based on the position of the reference field-of-view scan image and the relative positions between any two adjacent field-of-view scan images. The positions of other field-of-view scan images can also be represented by the coordinates of other field-of-view scan images.

[0124] Taking the first visual field scan image as the reference image as an example, the visual field scan images adjacent to the first visual field scan image are the second and fifth visual field scan images. The second visual field scan image is adjacent to the third visual field scan image, and the fifth visual field scan image is adjacent to the fourth and sixth visual field scan images. After determining the absolute coordinates of the first visual field scan image, the absolute coordinates of the second visual field scan image can be determined based on the relative positional relationship between the first and second visual field scan images. The absolute coordinates of the fifth visual field scan image can be determined based on the relative positional relationship between the first and fifth visual field scan images.

[0125] Furthermore, after determining the absolute coordinates of the second visual field scan image, the absolute coordinates of the third visual field scan image can be determined based on the relative positional relationship between the second and third visual field scan images. Similarly, after determining the absolute coordinates of the fifth visual field scan image, the absolute coordinates of the fourth visual field scan image can be determined based on the relative positional relationship between the fifth and fourth visual field scan images. The absolute coordinates of the sixth visual field scan image can be determined based on the relative positional relationship between the fifth and sixth visual field scan images. Thus, the absolute coordinates of all visual field scan images can be determined.

[0126] In summary, after determining the position of the reference field-of-view scan image, the positions of other field-of-view scan images can be determined by combining the relative positions between any adjacent field-of-view scan images. Then, based on the positions of the reference field-of-view scan image and the positions of the other reference field-of-view scan images, the multiple field-of-view scan images are fused to obtain the full-frame scan image.

[0127] After obtaining the full-frame scan image, the first position information of multiple key points in the full-frame scan image is determined according to the substrate pattern. For the specific implementation process, please refer to the relevant introduction in S13, which will not be repeated here.

[0128] After obtaining the first position information of multiple key points in the full-frame scan image, the full-frame scan image is corrected based on the first position information and the substrate pattern on the biochip to obtain the target full-frame scan image. The following section combines... Figure 8 Let me introduce it.

[0129] Figure 8 This is a schematic diagram of the correction process provided in the embodiments of this application, such as... Figure 8 As shown, it includes:

[0130] S81 determines the second position information of multiple key points in the full-frame scan image based on the substrate pattern and the size of the full-frame scan image.

[0131] The first position information is the position information of multiple key points obtained by identifying and processing the actual full-frame scan image. It should be noted that due to various reasons such as distortion during the scanning process of the scanner and errors in the fusion process, there is a certain error between the obtained full-frame scan image and the theoretical full-frame scan image of the biochip. Therefore, the first position information and the second position information are not completely the same, so correction processing is required.

[0132] Based on the substrate pattern on the biochip and the size of the full-frame scan image, the second position information of multiple key points in the full-frame scan image can be determined. This second position information represents the theoretical positions of the key points in the full-frame scan image. Specifically, the full-frame scan image is scaled or otherwise processed to match its size with the biochip size, based on the substrate pattern design and the degree of scaling (which can be represented by a matrix). Then, the second position information of the key points in the full-frame scan image is determined based on the substrate pattern design and the degree of scaling (which can be represented by a matrix). For example, if the full-frame scan image needs to be scaled or otherwise processed to obtain a matching image for the biochip, and this processed full-frame scan image is denoted as image A, and the substrate pattern design indicates that point B on the biochip is a key point, then point C on image A can be determined based on the position of point B on the biochip. The position of point C on image A is the same as the position of point B on the biochip. Since image A is obtained by scaling and other processing of a full-frame scan image, the position of point C on the full-frame scan image can be determined by reversing image A to obtain the original full-frame scan image, which is the second position information of the key point.

[0133] S82, based on the first position information and the second position information, perform correction processing on the full-frame scan image to obtain the target full-frame scan image.

[0134] After obtaining the first and second position information of multiple key points, a correction transformation matrix can be determined based on the first and second position information. The correction transformation matrix is ​​used to realize the transformation between the first and second position information.

[0135] Then, based on the correction transformation matrix, the full-frame scan image is subjected to a mapping transformation to obtain the final error-free full-frame scan image. The correction transformation matrix indicates the transformation from first position information to second position information. The first position information is the position information of multiple key points determined based on the fused full-frame scan image, and the second position information is the theoretical position information of multiple key points in the full-frame scan image. The transformation from the first position information to the second position information achieves the correction of multiple key points. The correction transformation matrix obtained based on the first and second position information represents the transformation parameters between the first and second position information. Therefore, by performing a mapping transformation on the full-frame scan image based on the correction transformation matrix, the full-frame scan image can be corrected to obtain the corrected target full-frame scan image. The mapping transformation on the full-frame scan image based on the correction transformation matrix essentially involves multiplying the pixel matrix in the full-frame scan image by the correction transformation matrix. This may include scaling or translating the full-frame scan image, etc., which are not limited in this embodiment.

[0136] Figure 9 This is a schematic diagram of the target full-frame scan image provided in an embodiment of this application. Figure 10 This is a magnified schematic diagram of the target full-frame scan image after overlaying a substrate pattern, as provided in an embodiment of this application. Figure 9 As shown, after correction processing, a complete full-frame scan image of the target can be obtained. Figure 10 As shown, to evaluate the accuracy of the correction, a biochip substrate pattern was superimposed on the obtained full-frame scan image of the target. Figure 10 The example on the left is an image obtained without using the scanning image correction method of the embodiments of this application. As can be seen, without correction, there will be ghosting or errors in the image, resulting in low accuracy. Figure 10 The example on the right is an image obtained using the scanning image correction method of this application. As can be seen, when the correction is performed using this method, the images will completely overlap without any ghosting, resulting in high accuracy.

[0137] The scanning image correction method provided in this application involves acquiring multiple field-of-view scanning images of a biochip with tissue slices attached to it. The biochip includes a substrate pattern containing multiple key points, and overlapping areas exist between any adjacent field-of-view scanning images. Then, based on the overlapping areas between any adjacent field-of-view scanning images, the multiple field-of-view scanning images are fused to obtain a full-frame scanning image of the biochip, achieving image stitching. Further, based on the substrate pattern, the first position information of multiple key points in the full-frame scanning image is determined. Then, based on the first position information and the substrate pattern, the full-frame scanning image is corrected to obtain a target full-frame scanning image. By using the first and second position information of multiple key points in the full-frame scanning image, image errors can be corrected, achieving accurate alignment between the gene expression information captured by the biochip and the region represented by the full-frame scanning image, thus meeting subsequent high-precision alignment requirements.

[0138] Figure 11 This is a schematic diagram of the structure of the scanned image correction device provided in the embodiments of this application, as shown below. Figure 11 As shown, it includes:

[0139] The acquisition module 111 is used to acquire multiple field-of-view scan images of the biochip while the tissue slice is attached to the biochip. The biochip includes a substrate pattern, and the substrate pattern includes multiple key points. There is an overlapping area between any adjacent field-of-view scan images.

[0140] The processing module 112 is used to perform fusion processing on the multiple field-of-view scan images based on the overlapping area between any adjacent field-of-view scan images to obtain a full-frame scan image of the biochip.

[0141] The determining module 113 is used to determine the first position information of the plurality of key points in the full-frame scan image based on the substrate pattern;

[0142] The correction module 114 is used to perform correction processing on the full-frame scan image based on the first position information and the substrate pattern to obtain the target full-frame scan image.

[0143] In one possible implementation, the processing module 112 is specifically used for:

[0144] The relative positions between any two adjacent visual field scan images are determined based on the overlapping area between them.

[0145] The multiple field-of-view scan images are fused based on their relative positions to obtain the full-frame scan image.

[0146] In one possible implementation, the processing module 112 is specifically used for:

[0147] For any first field-of-view scan image among the multiple field-of-view scan images, determine a second field-of-view scan image that is adjacent to the first field-of-view scan image;

[0148] Based on the overlapping area between the first field-of-view scan image and the second field-of-view scan image, a two-dimensional window sliding process is performed on the first field-of-view scan image and the second field-of-view scan image to determine the matching parameters between the first field-of-view scan image and the second field-of-view scan image;

[0149] The relative positions between the first field-of-view scan image and the second field-of-view scan image are determined based on the matching parameters.

[0150] In one possible implementation, the processing module 112 is specifically used for:

[0151] Among the plurality of field-of-view scan images, the position of a reference field-of-view scan image is determined, wherein the reference field-of-view scan image is any one of the plurality of field-of-view scan images;

[0152] Based on the position of the reference field-of-view scan image and the relative position between any adjacent field-of-view scan images, the positions of other field-of-view scan images are determined. The other field-of-view scan images are the field-of-view scan images other than the reference field-of-view scan image among the multiple field-of-view scan images.

[0153] Based on the position of the reference field-of-view scan image and the positions of the other field-of-view scan images, the multiple field-of-view scan images are fused to obtain the full-frame scan image.

[0154] In one possible implementation, the determining module 113 is specifically used for:

[0155] Determine the key point recognition model corresponding to the substrate pattern;

[0156] The full-frame scan image is input into the key point recognition model to obtain the first location information;

[0157] The key point recognition model is trained based on multiple sets of training samples. Each set of training samples includes a full-frame scan image of the sample and the corresponding label information. The full-frame scan image of the sample includes the substrate pattern, and the label information includes the position information of multiple key points in the full-frame scan image of the sample.

[0158] In one possible implementation, the correction module 114 is specifically used for:

[0159] Based on the substrate pattern and the size of the full-frame scan image, determine the second position information of the plurality of key points in the full-frame scan image;

[0160] Based on the first location information and the second location information, the full-frame scan image is corrected to obtain the target full-frame scan image.

[0161] In one possible implementation, the correction module 114 is specifically used for:

[0162] Based on the first location information and the second location information, determine the correction transformation matrix;

[0163] Based on the correction transformation matrix, the full-frame scan image is subjected to mapping transformation processing to obtain the target full-frame scan image.

[0164] The scanning image correction device provided in this application embodiment is used to execute the above method embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0165] Figure 12 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 12 As shown, the electronic device may include a processor 1210, a communications interface 1220, a memory 1230, and a communication bus 1240, wherein the processor 1210, the communications interface 1220, and the memory 1230 communicate with each other via the communication bus 1240. The processor 1210 can call logic instructions in the memory 1230 to execute a scan image correction method. This method includes: acquiring multiple field-of-view scan images of the biochip while a tissue slice is attached to the biochip; the biochip includes a substrate pattern, the substrate pattern including multiple key points, and overlapping areas exist between any adjacent field-of-view scan images; fusing the multiple field-of-view scan images according to the overlapping areas between any adjacent field-of-view scan images to obtain a full-frame scan image of the biochip; determining first position information of the multiple key points in the full-frame scan image based on the substrate pattern; and correcting the full-frame scan image based on the first position information and the substrate pattern to obtain a target full-frame scan image.

[0166] Furthermore, the logical instructions in the aforementioned memory 1230 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0167] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the scanning image correction method provided in the above embodiments. The method includes: acquiring multiple field-of-view scanning images of the biochip while a tissue slice is attached to the biochip, the biochip including a substrate pattern, the substrate pattern including multiple key points, and overlapping areas between any adjacent field-of-view scanning images; fusing the multiple field-of-view scanning images according to the overlapping areas between any adjacent field-of-view scanning images to obtain a full-frame scanning image of the biochip; determining first position information of the multiple key points in the full-frame scanning image according to the substrate pattern; and correcting the full-frame scanning image according to the first position information and the substrate pattern to obtain a target full-frame scanning image.

[0168] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program performs the scanning image correction method provided in the above embodiments. The method includes: acquiring multiple field-of-view scanning images of the biochip while a tissue slice is attached to the biochip, the biochip including a substrate pattern, the substrate pattern including multiple key points, and overlapping areas between any adjacent field-of-view scanning images; fusing the multiple field-of-view scanning images according to the overlapping areas between any adjacent field-of-view scanning images to obtain a full-frame scanning image of the biochip; determining first position information of the multiple key points in the full-frame scanning image according to the substrate pattern; and correcting the full-frame scanning image according to the first position information and the substrate pattern to obtain a target full-frame scanning image.

[0169] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0170] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for correcting scanned images, characterized in that, include: With tissue slices attached to the biochip, multiple field-of-view scan images of the biochip are acquired. The biochip includes a substrate pattern, which includes multiple key points. There is an overlapping area between any adjacent field-of-view scan images. Based on the overlapping area between any two adjacent field-of-view scan images, the multiple field-of-view scan images are fused to obtain a full-frame scan image of the biochip. Based on the substrate pattern, determine the first position information of the plurality of key points in the full-frame scan image; Based on the first position information and the substrate pattern, the full-frame scan image is corrected to obtain the target full-frame scan image; The step of correcting the full-frame scan image based on the first position information and the substrate pattern to obtain the target full-frame scan image includes: Based on the substrate pattern and the size of the full-frame scan image, determine the second position information of the plurality of key points in the full-frame scan image; Based on the first location information and the second location information, the full-frame scan image is corrected to obtain the target full-frame scan image.

2. The method according to claim 1, characterized in that, The step of fusing multiple field-of-view scan images based on the overlapping area between any adjacent field-of-view scan images to obtain a full-frame scan image of the biochip includes: The relative positions between any two adjacent visual field scan images are determined based on the overlapping area between them. The multiple field-of-view scan images are fused based on their relative positions to obtain the full-frame scan image.

3. The method according to claim 2, characterized in that, Determining the relative position between any two adjacent visual field scan images based on the overlapping area between them includes: For any first field-of-view scan image among the multiple field-of-view scan images, determine a second field-of-view scan image that is adjacent to the first field-of-view scan image; Based on the overlapping area between the first field-of-view scan image and the second field-of-view scan image, a two-dimensional window sliding process is performed on the first field-of-view scan image and the second field-of-view scan image to determine the matching parameters between the first field-of-view scan image and the second field-of-view scan image; The relative positions between the first field-of-view scan image and the second field-of-view scan image are determined based on the matching parameters.

4. The method according to claim 2, characterized in that, The step of fusing the multiple field-of-view scan images based on the relative positions between any adjacent field-of-view scan images to obtain the full-frame scan image includes: Among the plurality of field-of-view scan images, the position of a reference field-of-view scan image is determined, wherein the reference field-of-view scan image is any one of the plurality of field-of-view scan images; Based on the position of the reference field-of-view scan image and the relative position between any adjacent field-of-view scan images, the positions of other field-of-view scan images are determined. The other field-of-view scan images are the field-of-view scan images other than the reference field-of-view scan image among the multiple field-of-view scan images. Based on the position of the reference field-of-view scan image and the positions of the other field-of-view scan images, the multiple field-of-view scan images are fused to obtain the full-frame scan image.

5. The method according to any one of claims 1-4, characterized in that, The step of determining the first position information of the plurality of key points in the full-frame scan image based on the substrate pattern includes: Determine the key point recognition model corresponding to the substrate pattern; The full-frame scan image is input into the key point recognition model to obtain the first location information; The key point recognition model is trained based on multiple sets of training samples. Each set of training samples includes a full-frame scan image of the sample and the corresponding label information. The full-frame scan image of the sample includes the substrate pattern, and the label information includes the position information of multiple key points in the full-frame scan image of the sample.

6. The method according to claim 1, characterized in that, The step of correcting the full-frame scan image based on the first location information and the second location information to obtain the target full-frame scan image includes: Based on the first location information and the second location information, determine the correction transformation matrix; Based on the correction transformation matrix, the full-frame scan image is subjected to mapping transformation processing to obtain the target full-frame scan image.

7. A scanned image correction device, characterized in that, include: The acquisition module is used to acquire multiple field-of-view scan images of the biochip while the tissue slices are attached to the biochip. The biochip includes a substrate pattern, which includes multiple key points. There is an overlapping area between any two adjacent field-of-view scan images. The processing module is used to fuse the multiple field-of-view scan images according to the overlapping area between any adjacent field-of-view scan images to obtain the full-frame scan image of the biochip. The determining module is used to determine the first position information of the plurality of key points in the full-frame scan image based on the substrate pattern; The correction module is used to perform correction processing on the full-frame scan image based on the first position information and the substrate pattern to obtain the target full-frame scan image; The correction module is specifically used for: Based on the substrate pattern and the size of the full-frame scan image, determine the second position information of the plurality of key points in the full-frame scan image; Based on the first location information and the second location information, the full-frame scan image is corrected to obtain the target full-frame scan image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the scanned image correction method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the scanned image correction method as described in any one of claims 1 to 6.

Images