Method and apparatus for stitching slide images

Abstract

The application discloses a slide image splicing method and device, and belongs to the technical field of image processing. The method comprises the following steps: acquiring an offset of a previous field of view image relative to a reference point of a panoramic canvas, determining an offset of a current field of view image relative to the reference point of the panoramic canvas based on the offset, if the current field of view image is the last field of view image in a target row where the current field of view image is located, calculating splicing positions of each field of view image in the target row in the panoramic canvas respectively based on offsets of all field of view images in the target row relative to the reference point of the panoramic canvas, wherein there is an overlapping area between every two adjacent field of view images in the target row, performing fusion processing on each overlapping area in the target row based on the splicing positions to obtain a fused splicing row, and forming a slide image based on the fused splicing row. Through the technical scheme, the quality of the slide image obtained by splicing can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for stitching together glass slide images. Background Technology

[0002] A microscopy slide scanner scans a slide with a high-magnification objective lens, which can obtain local images of multiple fields of view. By stitching together multiple local images, a complete and high-resolution slide image can be obtained.

[0003] However, the relevant technology stitches together a local image before the stitching position is determined. Although the method has a high image generation speed, it may cause the problem of misalignment of local images.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a method and device for stitching slide images, aiming to solve the problem of misalignment in the stitching of local images in slide images.

[0006] To achieve the above objectives, this application provides a method for stitching together slide images, the method comprising the following steps: Obtain the offset of the previous view image relative to the reference point of the panoramic canvas, and determine the offset of the current view image relative to the reference point of the panoramic canvas based on the offset of the previous view image relative to the reference point of the panoramic canvas. If the current field of view image is the last field of view image in the target row where the current field of view image is located, then based on the offset of all field of view images in the target row relative to the reference point of the panoramic canvas, calculate the stitching position of each field of view image in the target row in the panoramic canvas, wherein there is an overlapping area between every two adjacent field of view images in the target row. Based on the splicing position, the overlapping regions existing in the target row are merged to obtain a merged spliced ​​row; Based on the fused and stitched rows, a slide image is formed.

[0007] In one embodiment, obtaining the offset of the previous view image relative to a reference point of the panoramic canvas, and determining the offset of the current view image relative to a reference point of the panoramic canvas based on the offset of the previous view image relative to the reference point of the panoramic canvas includes: Perform template matching on the current field of view image to determine the relative offset of the current field of view image relative to the reference point of the previous field of view image; The offset of the current view image relative to the reference point of the panoramic canvas is determined based on the offset of the previous view image relative to the reference point of the panoramic canvas and the relative offset of the current view image relative to the reference point of the previous view image.

[0008] In one embodiment, performing template matching on the current field-of-view image to determine the relative offset of the current field-of-view image relative to a reference point of the previous field-of-view image includes: Using the preset overlapping region of the previous field of view image as a template, template matching is performed on the current field of view image according to the template; If the template matching result is successful, the current field of view image is added to the set of fields of view where the template matching is successful. When the template matching result is successful, the area of ​​the current field of view image used for template matching is determined as the overlapping area. Based on the overlapping area, the relative offset of the current field of view image relative to the reference point of the previous field of view image is determined. If the template matching result fails, the relative offset of the current field of view image relative to the reference point of the previous field of view image is determined to be zero.

[0009] In one embodiment, calculating the stitching position of each view image in the target row on the panoramic canvas based on the offset of all view images in the target row relative to a reference point of the panoramic canvas includes: Based on the template matching results, seed fields of view are determined for all fields of view images in the target row where the current field of view image is located; simultaneously, Based on the seed field of view and the offset of all field of view images in the target row relative to the reference point of the panoramic canvas, calculate the stitching position of each field of view image in the target row in the panoramic canvas.

[0010] In one embodiment, the method for stitching the slide images further includes: Based on the stitching positions of all view images in the target row where the current view image is located in the panoramic canvas, all view images in the target row are stitched together in the panoramic canvas to obtain an initial stitching row; Based on the template matching results, calculate the qualified threshold of the field of view image corresponding to the field of view set that successfully matches the template; Based on the qualified threshold of the visual field image, unqualified visual field images in the initial stitching row are removed.

[0011] In one embodiment, the method for stitching the slide images further includes: Perform flat-field correction and deconvolution on the current field-of-view image; If the current field of view image is the last field of view image in the target row where the current field of view image is located, and the current field of view image has completed the flat field correction and the deconvolution, the step of calculating the stitching position of each field of view image in the target row in the panoramic canvas based on the offset of all field of view images in the target row relative to the reference point of the panoramic canvas is executed. If the current field-of-view image is the field-of-view image preceding the last field-of-view image in the target row in the scanning direction, then based on the offset of the current field-of-view image relative to the reference point of the panoramic canvas, the offset of the next field-of-view image relative to the reference point of the panoramic canvas is determined, and flat field correction and deconvolution are performed on the next field-of-view image.

[0012] In one embodiment, the step of merging the overlapping regions in the target row based on the splicing position to obtain the merged spliced ​​row includes: Based on the stitching positions of each view image in the target row where the current view image is located in the panoramic canvas, the overlapping region of each view image in the target row is obtained, wherein each overlapping region contains a superimposed layer and an overlay layer. Calculate the weight of each pixel in each of the overlapping regions; Based on the weights, weighted fusion is performed on each pixel in each of the overlapping regions to obtain the fused field-view image corresponding to each field-view image in the target row; The fused stitched row is obtained based on the fused field-of-view image and the stitching position.

[0013] In one embodiment, after forming the slide image based on the fused stitching rows, the method further includes: Slice the glass slide image to obtain tiles; Each of the aforementioned tiles is written into a slice file to store the slide image.

[0014] In one embodiment, writing each of the tiles into a slice file to store the slide image includes: The scanning coordinates of each tile in the slide image are obtained, and the scanning direction of each tile is determined based on the scanning coordinates of each tile in the slide image. Based on the scanning direction and scanning coordinates of each tile, the write index of each tile to be written into the slice file is determined, and each tile is written into the slice file according to the write index of each tile to be written into the slice file to store the slide image.

[0015] In addition, to achieve the above objectives, this application also provides a slide image stitching device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the slide image stitching method described above.

[0016] The present application proposes one or more technical solutions, which have at least the following technical effects: By calculating the offset of each field of view image relative to a reference point of the panoramic canvas, and using a row composed of multiple field of view images as a processing unit, the stitching position of each field of view image in the panoramic canvas is calculated based on the offset of each field of view image in the row relative to the reference point of the panoramic canvas, wherein there is an overlapping area between each pair of adjacent field of view images; then, according to the stitching position of each field of view image in the row, the overlapping areas in the row are fused to obtain a fused stitched row; finally, a slide image is formed based on the fused stitched row. Before stitching the field of view images, the field of view images calculate their own stitching positions in the panoramic canvas based on their own offsets relative to the reference point of the panoramic canvas, and then obtain a stitched row based on the stitching position to form a slide image. This fundamentally avoids the stitching misalignment problem caused by the stitching position of the field of view image not being determined when the field of view image is stitched, ensuring that the stitched slide image is misaligned, thereby improving the stitching quality of the slide image. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the first embodiment of the slide image stitching method of this application; Figure 2 This is a flowchart illustrating the second embodiment of the slide image stitching method of this application; Figure 3 A simplified flowchart illustrating the method for stitching slide images in this application is provided; Figure 4 This is a schematic diagram of the structure of the slide image splicing device of this application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application. To better understand the above technical solutions, a detailed description of the above technical solutions will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0022] A microscopic slide scanner can obtain a local image by scanning a slide with a high-magnification objective lens. This local image is only a part of the slide image. By stitching together multiple local images, a complete and high-resolution slide image can be obtained.

[0023] However, this technique stitches together a local image before its stitching position is determined. Although this method produces slide images quickly, it may result in misalignment of the stitched images. If the slide image is used in medical practice, this misalignment could affect the interpretation of digital pathology.

[0024] To address the aforementioned problems, this application provides a method for stitching slide images. The main solution is as follows: First, obtain the offset of the previous field-of-view image relative to a reference point on the panoramic canvas. Then, based on this offset, determine the offset of the current field-of-view image relative to the reference point on the panoramic canvas. If the current field-of-view image is the last field-of-view image in the target row, calculate the stitching position of each field-of-view image in the target row on the panoramic canvas based on the offsets of all field-of-view images in the target row relative to the reference point on the panoramic canvas, wherein every two adjacent field-of-view images in the target row have an overlapping area. Based on this stitching position, perform fusion processing on each overlapping area in the target row to obtain a fused stitched row. Finally, form a slide image based on this fused stitched row.

[0025] This application calculates the offset of each field-of-view image relative to a reference point on the panoramic canvas. A row, composed of multiple field-of-view images, is used as a processing unit. Based on the offset of each field-of-view image in the row relative to the reference point on the panoramic canvas, the stitching position of each field-of-view image in the panoramic canvas is calculated, where every two adjacent field-of-view images have an overlapping area. Then, based on the stitching positions of each field-of-view image in the row, the overlapping areas in the row are fused to obtain a fused stitched row. Finally, a slide image is formed based on this fused stitched row. Before stitching the field-of-view images, each field-of-view image calculates its own stitching position on the panoramic canvas based on its offset relative to the reference point. The resulting stitching row forms the slide image, fundamentally avoiding the stitching misalignment problem caused by the undetermined stitching position of the field-of-view image during stitching. This ensures that the stitched slide image is misaligned, thereby improving the stitching quality of the slide image.

[0026] To better understand the above technical solutions, exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0027] It should be noted that the execution subject of this embodiment can be a slide image stitching device. The slide image stitching device can be a computing service device with data processing, network communication and program running functions, such as a computer, cloud server, etc., or an electronic device that can realize the above functions, or a device for acquiring slide images. This embodiment does not specifically limit it.

[0028] The stitching of the slide images in this application requires the following process: Initialization: Initialize the number of grids, field of view image size, tile size, panoramic canvas size, panoramic canvas background, and row level labels; acquire the flat field calibration map, and calculate the flat field correction matrix based on the flat field calibration map; initialize the offset of the first field of view image acquired from a slide image to zero relative to the reference point of the panoramic canvas, where the offset includes horizontal and vertical offsets. Calculate offset and enhance the field of view image: Obtain the previous field of view image and the current field of view image. Using the preset overlap area of ​​the previous field of view image as a template, perform template matching on the current field of view image to determine the overlap area between the previous and current field of view images. Based on the overlap area, determine the relative offset of the current field of view image relative to the reference point of the previous field of view image. Based on the relative offset, calculate the offset of the current field of view image relative to the reference point of the panoramic canvas, and perform flat field correction and deconvolution on the current field of view image to enhance it. For field of view images with successful template matching, add them to the set of successfully matched fields of view. Calculate the stitching position for the entire row of field-of-view images: If offset calculation and field-of-view image enhancement have been performed on all field-of-view images in a row, then the seed field of view for all field-of-view images in the row is determined based on the template matching results of all field-of-view images in the row; based on the seed field of view and the offset of all field-of-view images in the row relative to the reference point of the panoramic canvas, the stitching position of all field-of-view images in the row in the panoramic canvas is calculated respectively, and based on the stitching position, all field-of-view images in the row are stitched in the panoramic canvas to obtain the initial stitching row; Remove unqualified visual field images: Based on the set of visual fields that have successfully matched the template, calculate the qualified threshold of the visual field images, and based on the qualified threshold of the visual field images, remove unqualified visual field images from the initial stitching row; Weighted fusion of overlapping regions: For the initial stitching row of the visual field images that have been removed from the unqualified ones, the weight of each pixel in the overlapping region of each visual field image is calculated. The overlapping region includes the overlaid layer and the superimposed layer. Based on the weight, weighted fusion is performed on each pixel in the overlapping region to obtain the fused stitching row. Writing to the slide image: Based on the fused stitching rows, a slide image is formed. The slide image is sliced ​​to obtain tiles. According to the position of each tile in the slide image, the scanning direction and scanning coordinates of each tile are determined. Based on the scanning direction and scanning coordinates, the write index of each tile is determined and written to the slice file. According to the write index, each tile is written to the slice file to store the slide image.

[0029] Based on this, embodiments of this application provide a method for stitching together slide images, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the slide image stitching method of this application. In this embodiment, the slide image stitching method includes steps S10 to S40: It should be noted that a series of initializations are required before starting any steps, including: First, initializing the number of grids, field-of-view image size, tile size, panoramic canvas size, panoramic canvas background, and row-level markers; Second, denoising the uniformly illuminated image acquired by the user in an unloaded field of view with the same illumination intensity, exposure time, and objective magnification as the actual scan, to obtain a flat-field calibration map, and calculating the flat-field correction matrix based on this map, where an unloaded field of view refers to a field of view without a slide or a slide-blank field of view; Third, for each slide image, the offset of the first field-of-view image acquired is initialized to zero, where the offset of the field-of-view image is relative to the upper left corner of the panoramic canvas, and includes both horizontal and vertical offsets.

[0030] The panoramic canvas contains multiple field-of-view images. The grid number indicates the number of field-of-view images in the horizontal and vertical directions, as well as the total number of field-of-view images. The field-of-view image size indicates the number of pixels in the horizontal and vertical directions of a single field-of-view image. The tile size indicates the number of pixels in the horizontal and vertical directions of a single tile. During writing, the panoramic canvas can be sliced ​​into multiple tiles to be stored as slice files. During reading, multiple slice files are read to obtain the slide image. The panoramic canvas size can be obtained based on the grid number and the field-of-view image size. When creating a panoramic canvas, all its pixels can be initialized to a preset background color. This serves two purposes: First, as an unfilled marker, the canvas's color value is used to distinguish blank areas from overlapping areas during weighted fusion of overlapping regions. Therefore, this preset background color should be a color value that rarely appears in typical slide images. Second, in areas of the field of view that have not yet been stitched or have been removed, it prevents pure black holes or pure white blocks from appearing in the sliced ​​output. This gives these areas a dark gray background similar to the empty areas of the slide, making it easier to distinguish these areas from the actual slide image when observing it. Row-level flags are used to indicate the offset calculation and enhancement status of a row of the field of view, determining whether stitching can be performed on that row.

[0031] Optionally, by grid number Field of view image size Tile size For example, the panoramic canvas size can be calculated as follows: This indicates that within the panoramic canvas, the horizontal direction includes... A field-of-view image, containing vertically... Total visual field images, A visual field image; representing the horizontal direction containing within that visual field image. Pixels, vertically containing Each pixel represents the number of pixels contained in the horizontal direction within that tile. Pixels, vertically containing 1 pixel.

[0032] Preferably, the preset background color of the panoramic canvas can be rgb(4,4,4), where rgb(·) is a function describing color values, and its three parameters represent the red channel, green channel, and blue channel, respectively. Optionally, the preset background color of the panoramic canvas can also be other color values ​​or grayscale values.

[0033] Optionally, the row-level flag may include two flags, one for indicating the offset calculation of a row of field-view images. ,in, Indicates the first Line; and a flag used to indicate the enhancement status of a line of visual field images. For example, row-level flags and It can be initialized to 0. If the offsets of all rows of the field of view have been calculated and enhanced, then the row-level flag is set. and All values ​​are updated to 1, indicating that the row can be concatenated.

[0034] Denoising is performed to remove noise interference from a uniformly illuminated image acquired with the same illumination intensity, exposure time, and objective magnification as the actual scan, resulting in a flat-field calibration map. The grayscale value of each pixel in this map reflects the actual illumination intensity at that location. Denoising methods can include: acquiring a single uniformly illuminated image and performing Gaussian smoothing to suppress noise; or acquiring multiple uniformly illuminated images and averaging them pixel-by-pixel to suppress noise, resulting in a flat-field calibration map. Let this flat-field calibration map be... ,Depend on The flat-field grayscale matrix formed by the grayscale values ​​of each pixel is: Then the flat field correction matrix It can be calculated as follows: .

[0035] in, Representatives passed The average of all elements is the sum of the elements in the expression. Matrices of the same dimension. For example, when passing through... When the average of all elements in the matrix is ​​180, then For the combination of elements with 180 Matrices of the same dimension. If If the value of a certain element in the matrix is ​​120, then the value of the corresponding element in the flat field correction matrix is ​​1.5. Values ​​greater than 1 indicate... If the pixel of this element is too dark, the pixel at the corresponding position in the field-of-view image acquired in the future with the illumination intensity, exposure time, and objective magnification used in the formal scanning configuration may be too dark, requiring multiplication by 1.5 to increase brightness; if If the value of a certain element in the matrix is ​​240, then the value of the corresponding element in the flat field correction matrix is ​​0.75. A value less than 1 indicates... If the pixel of this element is too bright, the pixel at the corresponding position in the field-of-view image acquired in the future with the formal scanning configuration may also be too bright, requiring multiplication by 0.75 to reduce brightness; if If the value of a certain element in the matrix is ​​180, then the value of the corresponding element in the flat field correction matrix is ​​1, where 1 represents... The pixel brightness of this element is moderate. In future field-of-view images acquired with a formal scanning configuration, the pixel at the corresponding position of this element may also be moderate, so it is multiplied by 1 to maintain brightness. Each element in the flat field correction matrix represents the correction factor of the pixel at the corresponding position in the field-of-view image relative to the average brightness. Multiplying the field-of-view image pixel by pixel with the flat field correction matrix can reduce the bright and dark stripes caused by uneven illumination in the field-of-view image.

[0036] Step S10: Obtain the offset of the previous view image relative to the reference point of the panoramic canvas, and determine the offset of the current view image relative to the reference point of the panoramic canvas based on the offset of the previous view image relative to the reference point of the panoramic canvas. The current field-of-view image refers to the scanned field-of-view image whose offset relative to the reference point of the panoramic canvas is to be calculated. The previous field-of-view image refers to the previous scanned field-of-view image whose offset relative to the reference point of the panoramic canvas has been calculated, relative to the current field-of-view image. Optionally, scanning can be performed using a microscope scanner. Within the same row, the previous field-of-view image specifically refers to the field-of-view image preceding the current field-of-view image in the scanning direction. If the current field-of-view image is the first image of a row, then its previous field-of-view image refers to the field-of-view image in the same column as the current field-of-view image in the previous row. The scanning direction is determined by the row number: the scanning direction for row 0 is from left to right, the scanning direction for subsequent odd-numbered rows is from right to left, and the scanning direction for even-numbered rows is from left to right. If the scanning direction is from right to left, the field-of-view image immediately to the right of the current field-of-view image is the previous field-of-view image; if the scanning direction is from left to right, the field-of-view image immediately to the left of the current field-of-view image is the previous field-of-view image.

[0037] The offset of the current field-of-view image relative to the reference point of the panoramic canvas is calculated based on the offset of the previous field-of-view image relative to the reference point of the panoramic canvas and the relative offset of the current field-of-view image relative to the reference point of the previous field-of-view image. Therefore, this offset may contain cumulative offset errors caused by the offset of the previous field-of-view image relative to the reference point of the panoramic canvas in the scanning direction. The reference point of the previous field-of-view image refers to its upper left corner. The offset of the field-of-view image relative to the reference point of the panoramic canvas is used to calculate the stitching position of the field-of-view image within the panoramic canvas; this stitching position is the location required for the field-of-view image to be written into the panoramic canvas.

[0038] It should be noted that for each field-of-view image, its offset relative to the reference point of the panoramic canvas needs to be calculated. The purpose of this is to calculate the corresponding stitching position in the panoramic canvas based on the offset of the field-of-view image relative to the reference point before stitching the field-of-view images. The field-of-view images are then stitched according to this position, thereby ensuring that there are no misalignments after the field-of-view images are stitched together.

[0039] Step S20: If the current field of view image is the last field of view image in the target row where the current field of view image is located, then based on the offset of all field of view images in the target row relative to the reference point of the panoramic canvas, calculate the stitching position of each field of view image in the target row in the panoramic canvas. The target row containing the current field-of-view image typically contains multiple field-of-view images. Only after all field-of-view images in the target row have had their offsets relative to the reference point of the panoramic canvas calculated—that is, when the current field-of-view image, whose offset relative to the reference point of the panoramic canvas has been calculated, is the last field-of-view image in the target row—can the stitching positions of each field-of-view image in the panoramic canvas be calculated based on their offsets relative to the reference point of the panoramic canvas. This is used for stitching subsequent field-of-view images to avoid misalignment issues in the stitched results.

[0040] In this target row, there is an overlapping area between every two adjacent field-of-view images. This overlapping area is a reserved area in the design of slide image scanning. This reserved overlapping area serves two purposes: First, by comparing the overlapping areas of two field-of-view images, it can be determined whether the two field-of-view images are adjacent, thereby determining the relative position of the two field-of-view images; Second, there may be certain errors in the slide image scanning process. If the reserved overlapping area is not designed, there may be unscanned areas or unwanted duplicate scans between the two scanned field-of-view images. After stitching the two field-of-view images, the stitched result is that the edges of the two field-of-view images cannot be aligned. However, after designing the reserved overlapping area, the errors in the scanning process can be further eliminated by comparing the two field-of-view images. By fusing the overlapping area, an edge-aligned and complete field-of-view image can be obtained.

[0041] The stitching position includes horizontal and vertical stitching positions, which are calculated based on the stitching position of the seed view that has been determined and the offset of the view image relative to the reference point of the panoramic canvas. This avoids the cumulative offset error that exists in the offset. At the same time, it indicates the canvas coordinates required when the view image is written to the panoramic canvas, rather than just the offset.

[0042] Step S30: Based on the splicing position, perform fusion processing on each of the overlapping regions existing in the target row to obtain a fused spliced ​​row; After determining the stitching positions of each view image in the panoramic canvas, the positions of the overlapping regions between the view images can be precisely determined, thus obtaining each overlapping region. Each overlapping region contains both the overlaid layer and the superimposed layer. The pixels of the overlaid layer and the superimposed layer are identical within the overlapping region. The visual effect of the overlapping region formed by the overlaid layer and the superimposed layer may be that the overlapping region is darker in color compared to the non-overlapping region, resulting in inconsistent display effects with the non-overlapping region, leading to bright and dark stripes. To ensure that the overlapping region formed by the overlaid layer and the superimposed layer displays the same effect as the non-overlapping region, a weighted fusion process is needed to obtain a fused view image. Based on the stitching positions of this fused view image and its corresponding view image in the panoramic canvas, a fused stitching row can be obtained. The fused view image includes both the non-overlapping region in its corresponding view image and the weighted overlapping region in its corresponding view image.

[0043] Step S40: Based on the fused stitching rows, form a slide image.

[0044] A slide is a thin, transparent piece of material used for observation under a microscope, primarily to hold samples for observation. Slide images are obtained using a microscope scanner and can be either images of the sample with a sample placed on it, or blank images of the slide without a sample. Slide images typically require very high resolution; therefore, the size of a single slide image can reach gigabytes. To meet the high-resolution requirement, a partial image is usually obtained by scanning with a microscope scanner. This partial image is called the field of view image. These field of view images are then stitched together to obtain the slide image. For storage, the slide image needs to be sliced ​​into tiles to create a slice file. The complete slide image can be retrieved by reading and stitching these slice files. This is because a single slide image is too large to be directly stored or retrieved.

[0045] In this embodiment, by calculating the offset of each field-of-view image relative to a reference point on the panoramic canvas, and using a row composed of multiple field-of-view images as a processing unit, the stitching position of each field-of-view image in the panoramic canvas is calculated based on its offset relative to the reference point of the panoramic canvas. Each pair of adjacent field-of-view images has an overlapping area. Then, based on the stitching positions of each field-of-view image in the row, the overlapping areas in the row are fused to obtain a fused stitched row. Finally, a slide image is formed based on this fused stitched row. Before stitching the field-of-view images, each field-of-view image calculates its own stitching position in the panoramic canvas based on its offset relative to the reference point of the panoramic canvas. The resulting stitching row forms the slide image, fundamentally avoiding the stitching misalignment problem caused by the undetermined stitching position of the field-of-view image during stitching. This ensures that the stitched slide image is misaligned, thereby improving the stitching quality of the slide image.

[0046] In one feasible implementation, step S10 may include steps S11-S12: Step S11: Perform template matching on the current field of view image to determine the relative offset of the current field of view image relative to the reference point of the previous field of view image; Template matching is a localization algorithm based on image similarity. It determines the position in the search image where the target image has the highest similarity by sliding the target image across the search image and calculating the similarity between the target image and sub-regions at various locations within the search image. Once the position with the highest similarity is determined, the overlapping region between the search and target images can be identified. This overlapping region can have three scenarios: First, the target image is after its starting position in the search image, in which case the overlapping region is larger than the target image and is defined as extending from the starting position in the search image to the ending position of the target image within the search image; second, the target image is before its starting position in the search image, in which case the overlapping region is smaller than the target image and is defined as extending from the starting position in the search image to the ending position of the target image within the search image; third, the target image is at its starting position in the search image, in which case the overlapping region is identical to the target image. The target image can be a pre-defined overlapping region of the previous visual field image, i.e., the template, and the search image can be a pre-defined search window of the current visual field image. Based on the source of the previous visual field image, template matching can be divided into horizontal matching and vertical matching: if the previous visual field image is the visual field image immediately to the left or right of the current visual field image, then the template matching is horizontal matching; if the previous visual field image is the visual field image in the same column as the current visual field image in the previous row, then the template matching is vertical matching.

[0047] Furthermore, since the size of the visual field image is fixed, and based on the defined overlap area between the previous and current visual field images, the relative offset of the current visual field image relative to the reference point of the previous visual field image can be determined. Taking the horizontal direction as an example, if the width of the visual field image is... The width of the determined overlapping region is Then the relative offset of the current field of view image with respect to the reference point of the previous field of view image is: .

[0048] Step S12: Based on the offset of the previous view image relative to the reference point of the panoramic canvas and the relative offset of the current view image relative to the reference point of the previous view image, determine the offset of the current view image relative to the reference point of the panoramic canvas. The offset of the current view image relative to the reference point of the panoramic canvas is obtained by adding the offset of the previous view image relative to the reference point of the panoramic canvas to the relative offset of the current view image relative to the reference point of the previous view image.

[0049] In this embodiment, by performing template matching on the current field-of-view image, the overlapping area between the current and previous field-of-view images can be determined. Based on this overlapping area, the relative offset of the current field-of-view image relative to the reference point of the previous field-of-view image can be calculated. Then, based on the offset of the previous field-of-view image relative to the reference point of the panoramic canvas and this relative offset, the offset of the current field-of-view image relative to the reference point of the panoramic canvas can be calculated. The template matching can obtain a true overlapping area, thereby eliminating the errors generated during the scanning of the field-of-view images. Therefore, the relative offset of the current field-of-view image relative to the reference point of the previous field-of-view image determined based on the overlapping area, and the offset of the current field-of-view image relative to the reference point of the panoramic canvas calculated based on the relative offset, also do not contain the errors generated during the scanning of the field-of-view images. This allows the determined offsets of each field-of-view image relative to the reference point of the panoramic canvas to be used to calculate the stitching position of each field-of-view image in the panoramic canvas, so as to determine the stitching position of each field-of-view image before stitching, ensuring that the stitching result is misaligned and improving the stitching quality of the slide images.

[0050] In one feasible implementation, step S11 may include steps S111 to S113: Step S111: Using the preset overlapping region of the previous field of view image as a template, perform template matching on the current field of view image according to the template; In one optional approach, the template matching algorithm is as follows: using a preset overlapping region of the previous visual field image as a template, a preset search window is extracted in the current visual field image. The template slides pixel by pixel within the preset search window. Each time it slides, the similarity between the template and a sub-region image within the preset search window is calculated. The position of the sub-region image with the highest similarity is the position of the template within the preset search window, and this position and the highest similarity are recorded as position 1 and highest similarity 1, respectively. Then, using the sub-region image with the highest similarity as a template, a preset search window is extracted from the previous visual field image, and the template and the preset search window undergo the same operation. Similarly, the location and highest similarity of the new sub-region image with the highest similarity are recorded as location 2 and highest similarity 2, respectively. Then, the distance between location 1 and location 2 is calculated, and it is determined whether the distance is less than a preset distance threshold. If the distance is greater than or equal to the preset distance threshold, the template matching result is determined to be a failure. If the distance is less than the preset distance threshold, further judgment is made. It is further determined whether the highest similarity 1 and highest similarity 2 are greater than the preset similarity threshold. If they are greater than or equal to the preset similarity threshold, the template matching result is determined to be a success. Otherwise, the template matching result is determined to be a failure.

[0051] Preferably, when performing horizontal template matching, the template can extract the right edge region from the visual field image immediately adjacent to the left side of the current visual field image. The width of this region can be 10% of the width of the visual field image immediately adjacent to the left side, and the height of this region can be 50% of the height of the visual field image immediately adjacent to the left side. This region is vertically centered in the visual field image immediately adjacent to the left side. When performing vertical template matching, the template can extract the lower edge region from the visual field image immediately adjacent to the upper side of the current visual field image. The height of this region can be 10% of the height of the visual field image immediately adjacent to the upper side, and the width of this region can be 50% of the width of the visual field image immediately adjacent to the upper side. This region is horizontally centered in the visual field image immediately adjacent to the upper side.

[0052] Preferably, when performing horizontal template matching, the preset search window can extract the left edge region from the current field of view image. The width of this region can be 60% of the width of the current field of view image, the height of this region can be 50% of the height of the current field of view image, and this region is vertically centered in the current field of view image. When performing vertical template matching, the preset search window can extract the upper edge region from the current field of view image. The height of this region can be 60% of the height of the current field of view image, the width of this region can be 50% of the width of the current field of view image, and this region is horizontally centered in the current field of view image.

[0053] Alternatively, the similarity calculation algorithm used for template matching can be normalized cross-correlation coefficient matching.

[0054] Optionally, the preset distance threshold can be 2 pixels, and the preset similarity threshold can be 0.98.

[0055] Step S112: If the template matching result is successful, the current field of view image is added to the set of fields of view where the template matching is successful. When the template matching result is successful, the area of ​​the current field of view image used for template matching is determined as the overlapping area. Based on the overlapping area, the relative offset of the current field of view image relative to the reference point of the previous field of view image is determined. The set of successfully matched fields of view records the field of view images that were successfully matched by the template. The field of view images contain overlapping and non-overlapping areas. The set of successfully matched fields of view is used to calculate the qualified threshold of the field of view images. The qualified threshold of the field of view images is used to remove unqualified field of view images from the field of view images that have been stitched according to the stitching position. The unqualified field of view images may be out of focus, have stains and / or blanks.

[0056] Step S113: If the template matching result is a failure, determine that the relative offset of the current field of view image relative to the reference point of the previous field of view image is zero.

[0057] Optionally, template matching may fail in the following situations: the visual field image participating in the template matching is a blank visual field image; the visual field image participating in the template matching has low similarity due to low image quality; or other situations besides the above two. In this case, the relative offset of the current visual field image with respect to the reference point of the previous visual field image is set to zero, so that the current visual field image can be forcibly written in later.

[0058] In this embodiment, a template matching algorithm is used to calculate the similarity between the previous and current field-of-view images. Based on the similarity calculation result, the template matching result can be determined. According to the template matching result, the relative offset of the current field-of-view image relative to the reference point of the previous field-of-view image can be determined. This relative offset can be used to calculate the offset of the current field-of-view image relative to the reference point of the panoramic canvas. Based on this offset, the stitching position of the current field-of-view image in the panoramic canvas can be calculated. This allows for determining the stitching position of each field-of-view image before stitching, ensuring that the stitching result is misaligned and improving the stitching quality of the slide images.

[0059] In another feasible implementation, steps S13 to S15 may be included after step S10: Step S13: Perform flat-field correction and deconvolution on the current field-of-view image; The purpose of planar correction is to achieve uniform brightness within the field of view image and consistent brightness between different field of view images. Deconvolution aims to compensate for the blurring caused by optical point diffusion, thereby improving the clarity of the field of view image. Optical point diffusion is a physical phenomenon. Due to the wave nature of light and the limitations of lens aperture, light rays cannot be perfectly converged into an infinitesimally small point. When the light from this point passes through the microscope objective, it is diffused or blurred into a circular spot of light, brightest in the center and gradually darkening outwards, with alternating bright and dark rings. This phenomenon of turning a point into a spot is called optical point diffusion. After planar correction and deconvolution, the stitched field of view images will not show bright and dark stripes, and there will be no blurred areas.

[0060] Step S14: If the current field of view image is the last field of view image in the target row where the current field of view image is located, and the current field of view image has completed the flat field correction and the deconvolution, execute the step of calculating the stitching position of each field of view image in the target row in the panoramic canvas based on the offset of all field of view images in the target row relative to the reference point of the panoramic canvas. Preferably, for a field-of-view image to be able to calculate its stitching position, three conditions must be met: first, the offset of the field-of-view image relative to the reference point of the panoramic canvas has been calculated; second, the field-of-view image has undergone flat-field correction and deconvolution; and third, all field-of-view images in the target row containing the field-of-view image have met the above two conditions. It is understood that the second condition is optional.

[0061] Step S15: If the current field of view image is the field of view image before the last field of view image in the target row in the scanning direction, then based on the offset of the current field of view image relative to the reference point of the panoramic canvas, determine the offset of the next field of view image relative to the reference point of the panoramic canvas, and perform flat field correction and deconvolution on the next field of view image.

[0062] It should be noted that if the current field of view image is not the last field of view image in the target row, then it is necessary to continue to calculate the offset of each of the remaining field of view images in the target row relative to the reference point of the panoramic canvas, and perform flat field correction and deconvolution on the remaining field of view images, until the offset of all field of view images in the target row relative to the reference point of the panoramic canvas has been calculated and flat field correction and deconvolution have been completed, before the stitching position can be calculated for the field of view images in the target row.

[0063] In this embodiment, after calculating the offset of the current field-of-view image relative to the reference point of the panoramic canvas, flat-field correction and deconvolution are used to further constrain the calculation of the stitching position of the current field-of-view image in the panoramic canvas. This ensures that the current field-of-view image can only be calculated in the panoramic canvas after meeting three conditions. This ensures that when stitching the field-of-view images, the field-of-view image has an offset relative to the reference point of the panoramic canvas and has already completed flat-field correction and deconvolution. This results in a field-of-view image stitching result without misalignment, bright or dark stripes, and / or blurring, thereby improving the stitching quality of the slide images.

[0064] The above are merely two feasible implementations of step S10 provided in this embodiment. This embodiment does not specifically limit the implementation of step S30. Furthermore, the above examples are only for understanding the steps and are not intended to limit the implementation of each step.

[0065] In one feasible implementation, step S20 may include steps S21-S22: Step S21: Based on the template matching result, determine the seed field of view of all field of view images in the target row where the current field of view image is located; A seed field of view is a reference field of view image used to help determine the stitching position of the current field of view image within the panoramic canvas. During the process of determining the offset of the field of view image relative to the reference point of the panoramic canvas, template matching is performed on the field of view image, and the template matching result is obtained. Based on this result, the seed field of view can be determined. Template matching results include success and failure, each corresponding to different seed field of view selection methods. For a successful template matching result, the seed field of view for the current field of view image is determined as follows: for the current field of view image, the first field of view image whose corresponding stitching position in the panoramic canvas has been determined and whose own template matching has been completed is selected as the seed field of view, using the field of view images immediately adjacent to it on its left, right, top, and bottom sides as the determination order. In the case where the template matching result fails, the seed field of view of the current field of view is determined as follows: in the target row where the current field of view is located, select the middle field of view image that has completed horizontal matching as the seed field of view, and use the seed field of view to determine the corresponding stitching positions of the current field of view image and the other field of view images in the target row that have not yet been determined in the panoramic canvas.

[0066] In one alternative approach, if the template matching result of the current field-of-view image is successful, the scanning direction of the target row where the current field-of-view image is located is from left to right, and the field-of-view image immediately to its left has already determined its corresponding stitching position in the panoramic canvas and completed its own template matching, then the seed field-of-view image of the current field-of-view image is determined to be the field-of-view image immediately to its left. If the template matching result of the current field-of-view image fails, and three field-of-view images in the target row where the current field-of-view image is located have already completed horizontal matching, then the field-of-view image located in the middle of the three field-of-view images is selected as the seed field-of-view.

[0067] Step S22: Based on the offset of all view images in the target row relative to the reference point of the panoramic canvas, calculate the stitching position of each view image in the target row in the panoramic canvas.

[0068] The calculation of the stitching position of the field-of-view image in the panoramic canvas follows a vertically-first, downward expansion strategy, i.e., row-first, progressing row by row from top to bottom. If the template matching result of the current field-of-view image is successful, the stitching position of the current field-of-view image in the panoramic canvas is calculated based on the seed field of view of the current field-of-view image and the offset relative to the reference point of the panoramic canvas. Specifically, taking the stitching position of the seed field of view of the current field-of-view image as a reference, the difference between the offset of the current field-of-view image and the offset of the seed field of view of the current field-of-view image relative to the reference point of the panoramic canvas is superimposed. This offset difference considers eliminating the overlapping area between the current field-of-view image and the seed field of view of the current field-of-view image and removing the out-of-bounds correction amount of the current field-of-view image. The out-of-bounds correction amount can be a forced correction value that forcibly pulls the stitching position back into the grid position of the field-of-view image when the cumulative error of the offset of the field-of-view image relative to the reference point of the panoramic canvas causes the stitching position of the field-of-view image to be calculated out of bounds. It is dynamically calculated based on the grid position of the field-of-view image. If template matching for the current field-of-view image fails, the stitching position of that image within the panoramic canvas is calculated based on preset grid positions and the field-of-view image size. The grid positions within the panoramic canvas are determined by their arrangement within the canvas. For example, when the number of grids is... When, the grid position of the last grid in the first row is .

[0069] In one alternative approach, if the current field of view image When the template matching result is successful, the stitching position of the current field-of-view image in the panoramic canvas is... It can be calculated as, .

[0070] in, Represents the grid position as The field of view image, Represents the grid position as The seed field of view can be , This represents the stitching position of the view image at a certain grid location within the panoramic canvas. and These represent the horizontal and vertical offsets of the view image at a specific grid location relative to a reference point on the panoramic canvas. This describes a method for calculating the stitching position of the view image within the panoramic canvas. If the current view image... The seed field of view is the field of view immediately to the left of the current field of view image. At that time, the horizontal stitching position of the current field-of-view image in the panoramic canvas can be calculated as follows: The vertical splicing position can be calculated as follows: The out-of-bounds correction amount can be calculated as follows: .

[0071] Furthermore, after determining the stitching position of the current view image in the panoramic canvas, seed propagation can be performed. Specifically, using the current view image as the seed view, the adjacent view images in the four directions (up, down, left, and right) whose offsets have been calculated but whose corresponding stitching positions in the panoramic canvas have not yet been determined are recursively checked. The stitching positions of these adjacent view images in the panoramic canvas are then determined, and this process is recursively propagated outwards level by level, that is, the stitching positions of the adjacent view images in the panoramic canvas are determined level by level for the view images whose corresponding stitching positions in the panoramic canvas have been determined. An upper limit for the recursion depth can be set. This limits the depth of propagation and prevents infinite recursion in abnormal situations.

[0072] In another alternative approach, if the current field of view image When the template matching result fails, the horizontal stitching position of the current field-of-view image in the panoramic canvas is... It can be calculated as Vertical splicing position It can be calculated as In this case, seed dispersal is no longer carried out.

[0073] In this embodiment, a seed field of view can be determined for the field of view image based on the template matching result. Based on the seed field of view and the offset of the field of view image relative to the reference point of the panoramic canvas, the corresponding stitching position of the field of view image in the panoramic canvas can be determined. This ensures that the corresponding stitching position in the panoramic canvas is determined when the field of view image is stitched, thereby ensuring that the stitching result is without misalignment and improving the stitching quality of the slide image.

[0074] In one feasible implementation, after step S22, steps S221 to S223 may also be included: Step S221: Based on the stitching positions of all view images in the target row where the current view image is located in the panoramic canvas, stitch all view images in the target row in the panoramic canvas to obtain an initial stitching row; Step S222: Based on the template matching result, calculate the qualified threshold of the field of view image corresponding to the field of view set that successfully matches the template. The set of successfully matched fields of view stores images of fields of view where the template matching result is successful. Based on the set of successfully matched fields of view, the average gray standard deviation of the overlapping region when the template matching result is successful can be calculated. for: .

[0075] in, The set of views that successfully matched the template. The number of field-of-view images that successfully matched the template. For visual field images The result of template matching is the standard deviation of gray levels in the overlapping region when successful. This is calculated by taking the standard deviation of the gray values ​​of all pixels in the overlapping region, representing the texture contrast of that region. Based on this average gray level standard deviation, the acceptable threshold for the visual field image can be obtained. for: .

[0076] Optionally, In practice, the selection range can be [0.3, 0.7]; preferably, The value can be 0.5. Under the same microscope scanner and scanning conditions, the sample to be scanned contains normal tissue, blank areas, and localized defocusing. Compare the slide image without removal, the slide image after field-of-view image removal using a fixed threshold, and the image obtained by... The slide images after threshold-based removal of visual field images are evaluated using metrics including subjective scores for graying / contamination, recall rate of items to be removed, false removal rate, and the number of holes in the grid after forced filling. The experimental results are as follows: The threshold-based rejection method has a higher recall rate and a lower false rejection rate, and the number of holes in the grid is zero after forced filling. It should be noted that... Sensitivity is: Easily missed and rejected. Mistakenly removed rising, A compromise is preferable. Therefore, As the default value.

[0077] Step S223: Based on the qualified threshold of the visual field image, remove unqualified visual field images from the initial stitching row.

[0078] In one alternative approach, based on a visual field image qualification threshold, unqualified visual field images in the initial stitching row can be removed using the following method: .

[0079] in, The standard deviation of gray levels in the visual field image reflects its texture contrast. When... If the image value is less than the acceptable threshold, it indicates that the image is out of focus, blank, and / or has stains. Therefore, the image is considered unacceptable and is discarded.

[0080] In another alternative approach, for viewpoint images for which the offset relative to the reference point of the panoramic canvas has been calculated, but whose corresponding stitching position in the panoramic canvas cannot be determined due to reasons such as missing seed views, and which have not been culled, the grid position of the viewpoint image can be used. The system forcibly calculates the stitching position of the current field-of-view image in the panoramic canvas and forcibly fills in the current field-of-view image with that stitching position. The horizontal stitching position of the current field-of-view image in the panoramic canvas is then determined. It can be calculated as Vertical splicing position It can be calculated as When stitching the field-of-view image to the panoramic canvas based on the stitching position, only pixels in the panoramic canvas that still have the preset background color are stitched together. Existing pixels are not overwritten, and no weighted fusion is performed. This ensures the integrity of the stitched slide image on the panoramic canvas while avoiding damage to the stitched content. It should be noted that the rejected field-of-view image is not included in the forced calculation of its corresponding stitching position in the panoramic canvas, and its corresponding grid retains the preset background color of the panoramic canvas.

[0081] In this embodiment, by using a qualified field-of-view image threshold, field-of-view images with issues such as defocusing, blank areas, and / or stains can be removed from the stitching rows. For field-of-view images where the offset relative to the reference point of the panoramic canvas has been calculated but the corresponding stitching position in the panoramic canvas cannot be determined due to reasons such as missing seed fields of view, and which have not been removed, the stitching position of the field-of-view image in the panoramic canvas is forcibly determined. This ensures that when stitching the field-of-view image, the stitching is completed with a suitable field-of-view image based on the determined stitching position, thus ensuring that the stitching result is free from problems such as misalignment, defocusing, blank areas, and / or stains, thereby improving the stitching quality of the slide images.

[0082] The above is merely one feasible implementation of step S20 provided in this embodiment, and this embodiment does not specifically limit the specific implementation of step S20. Furthermore, the above examples are only for understanding the steps and are not intended to limit the implementation of each step.

[0083] In one feasible implementation, step S30 may include steps S31 to S34: Step S31: Based on the stitching positions of each view image in the target row where the current view image is located in the panoramic canvas, obtain the overlapping area of ​​each view image in the target row; The overlapping area originates from the following: when planning the slide image scanning path, the stage step distance between adjacent field-of-view images is designed to be smaller than the image size, forming a pre-reserved overlapping area. This overlapping area is then formed by superimposing the measurement error of the offset of the field-of-view image relative to the reference point of the panoramic canvas. Therefore, the overlapping area is actively reserved in the design. Optionally, the width of the pre-reserved overlapping area can be... The height of the overlapping area can be The width and height of the overlapping area may take values ​​within the preset overlapping area.

[0084] Specifically, based on the stitching positions of each field-of-view image in the target row of the current field-of-view image within the panoramic canvas, the overlap region between the two field-of-view images can be determined by judging the consistency of their stitching positions. The overlapping layer within the overlap region refers to the overlapping area of ​​the two field-of-view images that have already been stitched together using their corresponding stitching positions in the panoramic canvas. The superimposed layer within the overlap region refers to the overlapping area of ​​the two field-of-view images that will later need to be stitched together using their corresponding stitching positions in the panoramic canvas.

[0085] Step S32: Calculate the weight of each pixel in each of the overlapping regions; For overlapping regions, it is necessary to calculate the weight of each pixel for both the overlaid and superimposed layers.

[0086] In one alternative approach, the weight of each pixel in the overlay layer is calculated. Taking a pixel with coordinates (x, y) in the overlay layer as an example, where the pixel is not located at or outside the edge of the overlapping region: First, the horizontal weight of this pixel is calculated. ,in, This is the horizontal distance of the pixel from the closer edge of the overlapping area (left or right). Next, calculate the vertical weight of the pixel. ,in, This is the vertical distance of the pixel from the closer edge of the overlapping area's upper and lower edges. Finally, calculate the weight of the pixel. Weights based on pixels with coordinates (x, y) in the overlay layer. The weight of the pixel with coordinates (x, y) in the stacked layers can be determined as follows: If a pixel in the overlay layer is located at the edge of the overlapping region, then the pixel's... Take the weight corresponding to the edge direction: If the pixel coordinates are (x, 0), then the weight of the pixel is the horizontal direction weight. If the pixel coordinates are (0, y), then the weight of that pixel is the vertical weight. If a pixel in the overlay layer is located outside the overlapping region, then That is, the overlay layer directly covers the overlaid layer.

[0087] In another alternative approach, the horizontal weights can also be calculated as follows: The vertical weight can also be calculated as .

[0088] Step S33: Based on the weights, perform weighted fusion on each pixel in each of the overlapping regions to obtain the fused field-view image corresponding to each field-view image in the target row; This process involves weighted fusion of each pixel within the overlapping region. Essentially, it merges the overlaid and superimposed layers within the overlapping region to avoid significant differences in display quality compared to non-overlapping areas. The pixel with coordinates (x, y) in the superimposed layer is considered as an example. and the pixel with coordinates (x,y) in the overlay For example, fusion of field of view pixels It can be calculated as: .

[0089] For each field-of-view image in the target row, its corresponding fused field-of-view image can be obtained based on multiple fused pixels.

[0090] Step S34: Based on the fused field-of-view image and the stitching position, the fused stitching row is obtained.

[0091] Specifically, based on the stitching positions of each field-of-view image in the panoramic canvas, the fused field-of-view images corresponding to each field-of-view image can be stitched together to obtain a fused stitching row.

[0092] In this embodiment, the weights of the overlaid and superimposed layers in the overlapping region are calculated separately, and then the weights are applied to the overlaid and superimposed layers in the overlapping region to fuse the overlaid and superimposed layers in the overlapping region, thereby obtaining a fused field-of-view image. This makes the display effect of the overlapping region similar to that of the non-overlapping region, without problems such as bright and dark stripes. Based on the stitching position of the field-of-view image in the panoramic canvas, a fused stitching row can be obtained. Based on the fused stitching row, the slide image can be obtained. This ensures that the stitching effect of the field-of-view image in the slide image is natural and smooth, while also ensuring that there are no misalignments in the slide image stitching, thereby improving the stitching quality of the slide image.

[0093] The above is merely one feasible implementation of step S30 provided in this embodiment, and this embodiment does not specifically limit the specific implementation of step S30. Furthermore, the above examples are only for understanding the steps and are not intended to limit the implementation of each step.

[0094] Based on the same inventive concept, this application also provides a second embodiment, referring to... Figure 2 , Figure 2 This is a schematic flowchart of a second embodiment of the slide image stitching method of this application. In this embodiment, after step S40, the slide image stitching method further includes steps S50-S60: Step S50: Slice the glass slide image to obtain tiles; Optionally, the slide image may originate from a panoramic canvas or from multiple local canvases. When the slide image comes from a panoramic canvas, the image within that panoramic canvas is the slide image. When the slide image comes from multiple local canvases, these local canvases need to be joined end-to-end horizontally to form a panoramic canvas, and the image within that panoramic canvas is the slide image. These multiple local canvases correspond to multiple scans of local regions of the slide image. For example, partitioned scanning.

[0095] Slicing refers to the process of spatially dividing the panoramic canvas into a series of fixed-size, non-overlapping rectangular image sub-blocks. This process does not involve lossy compression or format conversion; it only performs spatial cropping operations on pixel data.

[0096] In this context, a tile refers to a rectangular image sub-block obtained by slicing a panoramic canvas image. Each tile contains complete pixel information of the area it covers, has a fixed pixel width and height, and a unique spatial index identifier used to identify its row and column position in the panoramic canvas. The tile size can be the same as the view image size, or it can be a standard tile size, such as 256×256 or 512×512.

[0097] Step S60: Write each of the tiles into a slice file to store the slide image.

[0098] Among them, the slide file refers to the physical file used to store slide image data, that is, it can store tiles. Its internal structure follows the common or standard format specifications in the field of digital pathology, and supports random reading and dynamic loading of tiles through spatial indexing.

[0099] In this process, tiles need to be written to the slice file according to the tile index. The purpose is to ensure that the spatial arrangement of the slice file is consistent with the result obtained from the physical scan, so as to ensure that there will be no skipping of slice files when reading the slice file according to the tile index.

[0100] In this embodiment, the slide image, which is stitched together based on the stitching position of the field of view image in the panoramic canvas, is sliced ​​into tiles. The tiles are written into the slice file through the tile index. This ensures that the spatial arrangement of the tiles read when reading the slice file is consistent with the result obtained by physical scanning. At the same time, the slide image composed of the read tiles is without misalignment, ensuring that there are no problems such as skipped tiles or misalignment when reading the slide.

[0101] In one feasible implementation, step S60 may include steps S61-S62: Step S61: Obtain the scanning coordinates of each tile in the slide image, and determine the scanning direction of each tile based on the scanning coordinates of each tile in the slide image; The scan coordinates of the tile in the slide image refer to the coordinates determined based on the scanning order of each field of view when the slide image is scanned, including scan row coordinates and scan column coordinates. By obtaining the scan row coordinates of the tile in the slide image, the scanning direction of the tile can be determined: even-numbered rows of tiles are scanned from left to right, while odd-numbered rows of tiles are scanned from right to left because the stage moves from right to left during scanning. Even-numbered rows include the tile in row 0.

[0102] In one alternative approach, taking a tile with scanning coordinates (i,j) in a slide image as an example, if i is an even-numbered row, the scanning direction is from left to right; if i is an odd-numbered row, the scanning direction is from right to left.

[0103] Step S62: Based on the scanning direction and scanning coordinates of each tile, determine the write index of each tile to be written into the slice file, and write each tile into the slice file according to the write index of each tile to store the slide image.

[0104] In one alternative approach, the write index of a tile to the slice file can be determined based on the tile's scan direction and scan coordinates: the write index for even-numbered rows of tiles is determined by arranging the scan column coordinates from left to right, while for odd-numbered rows of tiles, since the stage moves from right to left during scanning, the scan column coordinates are used... Mirror image This serves as the write index. Subsequently, the tiles are written to the slice file according to this write index, and stored as a slide image.

[0105] In this embodiment, the scanning coordinates are mapped to tile indices. Based on these tile indices, the tiles are written into the slide file. This ensures that the spatial arrangement of the tiles read when reading the slide file is consistent with the result obtained from the physical scan. At the same time, the slide image composed of the read tiles is free of misalignment, ensuring that there are no problems such as skipped tiles or misalignment when reading the slide.

[0106] The above is merely one feasible implementation of step S60 provided in this embodiment, and this embodiment does not specifically limit the specific implementation of step S60. Furthermore, the above examples are only for understanding the steps and are not intended to limit the implementation of each step.

[0107] Since the system described in Embodiment 3 of this application is a system used to implement the method of Embodiment 1 of this application, those skilled in the art can understand the specific structure and variations of the system based on the method described in Embodiment 1 of this application, and therefore will not be described again here. All systems used in the method of Embodiment 1 of this application fall within the scope of protection of this application.

[0108] For example, to help understand the implementation process of the slide image stitching method obtained by combining Embodiment 2 with Embodiment 1 above, please refer to... Figure 3 , Figure 3 A simplified flowchart of the slide image stitching method of this application is provided, specifically: Initialization: Initialize the number of grids, field of view image size, tile size, panoramic canvas size, panoramic canvas background, and row level labels; acquire the flat field calibration map, and calculate the flat field correction matrix based on the flat field calibration map; initialize the offset of the first field of view image acquired from a slide image to zero relative to the reference point of the panoramic canvas, where the offset includes horizontal and vertical offsets. Calculate offset and enhance the field of view image: Obtain the previous field of view image and the current field of view image. Using the preset overlap area of ​​the previous field of view image as a template, perform template matching on the current field of view image to determine the overlap area between the previous and current field of view images. Based on the overlap area, determine the relative offset of the current field of view image relative to the reference point of the previous field of view image. Based on the relative offset, calculate the offset of the current field of view image relative to the reference point of the panoramic canvas, and perform flat field correction and deconvolution on the current field of view image to enhance it. For field of view images with successful template matching, add them to the set of successfully matched fields of view. Calculate the stitching position for the entire row of field-of-view images: If offset calculation and field-of-view image enhancement have been performed on all field-of-view images in a row, then the seed field of view for all field-of-view images in the row is determined based on the template matching results of all field-of-view images in the row; based on the seed field of view and the offset of all field-of-view images in the row relative to the reference point of the panoramic canvas, the stitching position of all field-of-view images in the row in the panoramic canvas is calculated respectively, and based on the stitching position, all field-of-view images in the row are stitched in the panoramic canvas to obtain the initial stitching row; Remove unqualified visual field images: Based on the set of visual fields that have successfully matched the template, calculate the qualified threshold of the visual field images, and based on the qualified threshold of the visual field images, remove unqualified visual field images from the initial stitching row; Weighted fusion of overlapping regions: For the initial stitching row of the visual field images that have been removed from the unqualified ones, the weight of each pixel in the overlapping region of each visual field image is calculated. The overlapping region includes the overlaid layer and the superimposed layer. Based on the weight, weighted fusion is performed on each pixel in the overlapping region to obtain the fused stitching row. Writing to the slide image: Based on the fused stitching rows, a slide image is formed. The slide image is sliced ​​to obtain tiles. According to the position of each tile in the slide image, the scanning direction and scanning coordinates of each tile are determined. Based on the scanning direction and scanning coordinates, the write index of each tile is determined and written to the slice file. According to the write index, each tile is written to the slice file to store the slide image.

[0109] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the splicing method of the slide images in this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0110] Based on the same inventive concept, this application provides a slide image stitching device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the slide image stitching method in the above embodiments.

[0111] The following is for reference. Figure 4 The diagram illustrates a structural schematic of a slide image stitching device suitable for implementing embodiments of this application. The slide image stitching device in the embodiments of this application may include, but is not limited to, computing service devices with data processing, network communication, and program execution functions, such as computers and cloud servers. Figure 4 The illustrated slide image stitching device is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0112] like Figure 4As shown, the slide image stitching device may include a processing unit 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the slide image stitching device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the slide image stitching device to communicate wirelessly or wiredly with other devices to exchange data. Although slide image stitching devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0113] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0114] The slide image stitching device provided in this application, employing the slide image stitching method described in the above embodiments, can solve the technical problem of stitching misalignment in local image stitching of slide images. Compared with the prior art, the beneficial effects of the slide image stitching device provided in this application are the same as those of the slide image stitching method provided in the above embodiments, and other technical features of this slide image stitching device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0115] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0116] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for stitching together slide images, characterized in that, The method includes: Obtain the offset of the previous view image relative to the reference point of the panoramic canvas, and determine the offset of the current view image relative to the reference point of the panoramic canvas based on the offset of the previous view image relative to the reference point of the panoramic canvas. If the current field of view image is the last field of view image in the target row where the current field of view image is located, and the current field of view image has completed the flat field correction and the deconvolution, then based on the offset of all field of view images in the target row relative to the reference point of the panoramic canvas, calculate the stitching position of each field of view image in the target row in the panoramic canvas, wherein there is an overlapping area between every two adjacent field of view images in the target row. Based on the splicing position, the overlapping regions existing in the target row are merged to obtain a merged spliced ​​row; Based on the fused and stitched rows, a slide image is formed; The method further includes: if the current field-of-view image is the field-of-view image preceding the last field-of-view image in the target row in the scanning direction, then based on the offset of the current field-of-view image relative to the reference point of the panoramic canvas, determining the offset of the next field-of-view image relative to the reference point of the panoramic canvas, and performing flat field correction and deconvolution on the next field-of-view image.

2. The method for stitching slide images as described in claim 1, characterized in that, The step of obtaining the offset of the previous view image relative to the reference point of the panoramic canvas, and determining the offset of the current view image relative to the reference point of the panoramic canvas based on the offset of the previous view image relative to the reference point of the panoramic canvas, includes: Perform template matching on the current field of view image to determine the relative offset of the current field of view image relative to the reference point of the previous field of view image; The offset of the current view image relative to the reference point of the panoramic canvas is determined based on the offset of the previous view image relative to the reference point of the panoramic canvas and the relative offset of the current view image relative to the reference point of the previous view image.

3. The method for stitching slide images as described in claim 2, characterized in that, The step of performing template matching on the current field-of-view image to determine the relative offset of the current field-of-view image relative to the reference point of the previous field-of-view image includes: Using the preset overlapping region of the previous field of view image as a template, template matching is performed on the current field of view image according to the template; If the template matching result is successful, the current field of view image is added to the set of fields of view where the template matching is successful. When the template matching result is successful, the area of ​​the current field of view image used for template matching is determined as the overlapping area. Based on the overlapping area, the relative offset of the current field of view image relative to the reference point of the previous field of view image is determined. If the template matching result fails, the relative offset of the current field of view image relative to the reference point of the previous field of view image is determined to be zero.

4. The method for stitching slide images as described in claim 3, characterized in that, The step of calculating the stitching position of each view image in the target row in the panoramic canvas based on the offset of all view images in the target row relative to the reference point of the panoramic canvas includes: Based on the template matching results, seed fields of view are determined for all fields of view images in the target row where the current field of view image is located; simultaneously, Based on the seed field of view and the offset of all field of view images in the target row relative to the reference point of the panoramic canvas, calculate the stitching position of each field of view image in the target row in the panoramic canvas.

5. The method for stitching glass slide images as described in claim 3, characterized in that, The method further includes: Based on the stitching positions of all view images in the target row where the current view image is located in the panoramic canvas, all view images in the target row are stitched together in the panoramic canvas to obtain an initial stitching row; Based on the template matching results, calculate the qualified threshold of the field of view image corresponding to the field of view set that successfully matches the template; Based on the qualified threshold of the visual field image, unqualified visual field images in the initial stitching row are removed.

6. The method for stitching slide images as described in claim 1, characterized in that, The process of fusing the overlapping regions in the target row based on the splicing position to obtain the fused spliced ​​row includes: Based on the stitching positions of each view image in the target row where the current view image is located in the panoramic canvas, the overlapping region of each view image in the target row is obtained, wherein each overlapping region contains a superimposed layer and an overlay layer. Calculate the weight of each pixel in each of the overlapping regions; Based on the weights, weighted fusion is performed on each pixel in each of the overlapping regions to obtain the fused field-view image corresponding to each field-view image in the target row; The fused stitched row is obtained based on the fused field-of-view image and the stitching position.

7. The method for stitching slide images as described in claim 1, characterized in that, After forming the slide image based on the fused stitched rows, the process further includes: Slice the glass slide image to obtain tiles; Each of the aforementioned tiles is written into a slice file to store the slide image.

8. The method for stitching slide images as described in claim 7, characterized in that, The step of writing each of the tiles into a slice file to store the slide image includes: The scanning coordinates of each tile in the slide image are obtained, and the scanning direction of each tile is determined based on the scanning coordinates of each tile in the slide image. Based on the scanning direction and scanning coordinates of each tile, the write index of each tile to be written into the slice file is determined, and each tile is written into the slice file according to the write index of each tile to store the slide image.

9. A device for stitching together slide images, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the method for stitching slide images as described in any one of claims 1 to 8.

Images