Methods and equipment for stitching together microscopic scanning images of glass slides and aquatic organisms.

Abstract

This disclosure provides a method and apparatus for stitching microscopic scanned images of a glass slide and aquatic organisms. The glass slide includes a slide body and a grid of marker lines. The grid of marker lines includes multiple first marker lines and multiple second marker lines with different grayscale characteristics. The spacing between adjacent first marker lines and the spacing between adjacent second marker lines are both less than a threshold distance. During the microscopic image stitching process, at least two stitching references can be determined based on the first and second marker lines. Based on these at least two stitching references, the stitching relationship between two adjacent microscopic images can be determined, thus enabling accurate and rapid stitching of the microscopic images based on the stitching references.

Description

Technical Field

[0001] This disclosure relates to the field of microscopic observation technology, specifically to a method and apparatus for stitching together microscopic scanning images of glass slides and aquatic organisms. Background Technology

[0002] Microscopic examination of aquatic organisms is a crucial step in aquatic ecosystem assessment. Because aquatic organisms are small in size and have complex taxonomic characteristics, microscopic examination requires capturing magnified images of different locations on a slide using a microscopic scanning imaging system. These magnified images are then stitched together to form a unified image, which is subsequently analyzed, identified, and its biomass is statistically analyzed.

[0003] In the aforementioned operation, stitching together the magnified microscopic images to form a unified image is the most challenging aspect. Specifically, because the aquatic organisms are randomly positioned and vary in density on the slide, it can be difficult to find reference points from adjacent magnified microscopic images for correct stitching, resulting in poor image stitching quality. To overcome this problem, existing technologies can only maximize the overlap of the magnified microscopic images. However, increasing the overlap increases the time required to capture images per unit slide and the number of magnified microscopic images acquired, further increasing the resource consumption and workload in the subsequent image stitching process. Summary of the Invention

[0004] To address the problem of excessively high costs associated with stitching microscopic images due to the lack of reference objects, embodiments of this disclosure provide a method and computing device for stitching microscopic scanned images of glass slides and aquatic organisms.

[0005] In a first aspect, embodiments of this disclosure provide a glass slide, comprising: a glass slide body, and a mesh marking line disposed in the microscopic observation area of ​​the glass slide body;

[0006] The mesh marking lines include multiple first marking lines that penetrate the microscopic observation area along a first direction, and second marking lines that penetrate the microscopic observation area along a second direction, wherein the first direction is different from the second direction;

[0007] At least adjacent first marker lines are configured to have different light processing characteristics, resulting in different line grayscale values ​​after microscopic imaging under the same lighting conditions; at least adjacent second marker lines are configured to have different light processing characteristics, resulting in different line grayscale values ​​after microscopic imaging under the same lighting conditions.

[0008] The spacing between adjacent first marker lines and the spacing between adjacent second marker lines are both less than a threshold distance; the threshold distance is the distance between at least two first marker lines and at least two second marker lines in the microscopic images to be stitched together after microscopic imaging.

[0009] Optionally, each of the first marker lines has different light processing characteristics, and the grayscale of each first marker line changes sequentially after microscopic imaging; and / or,

[0010] Each of the second marker lines has different light processing characteristics, and the grayscale of each second marker line changes sequentially after microscopic imaging.

[0011] Optionally, the first marking line and / or the second marking line are etching lines formed by etching in the microscopic observation area; at least adjacent first marking lines have different etching depths and / or etching widths; and / or at least adjacent second marking lines have different etching depths and / or etching widths.

[0012] or,

[0013] The first and / or the second marker lines are deposition lines formed on the microscopic observation area; at least adjacent first marker lines have different deposition thicknesses and / or different light absorption characteristics; and / or, at least adjacent second marker lines have different deposition thicknesses and / or different light absorption characteristics.

[0014] Optionally, each of the first marking lines is arranged at equal intervals on the glass slide body;

[0015] And / or, each of the second marking lines is disposed at equal intervals on the glass slide body.

[0016] Secondly, embodiments of this disclosure provide a method for stitching together microscopic scanning images of aquatic organisms, including:

[0017] The first and second marker lines in each microscopic image to be stitched are identified, and the position coordinates of the stitching reference in the microscopic image to be stitched are determined based on the positions of the identified first and second marker lines. The two-dimensional position markers of each stitching reference are determined according to the pixel grayscale of the first and second marker lines. The microscopic images to be stitched are obtained by a microscopic imaging system from samples prepared based on the glass slides as described above.

[0018] Based on the premise that the stitching reference pixels with the same two-dimensional position identifier in adjacent microscopic images to be stitched coincide, the adjacent microscopic images to be stitched are stitched and fused to obtain a stitched microscopic image.

[0019] Optionally, before performing stitching and fusion processing on adjacent microscopic images to be stitched, the method further includes: determining the number of pixels between two stitching references in the microscopic images to be stitched;

[0020] The scaling factor is determined based on the number of interval pixels and the corresponding number of standard pixels; wherein the number of standard pixels is determined by performing microscopic imaging on the two stitching references at a standard magnification, or by performing microscopic imaging on a similar reference that has the same distance from the two stitching references.

[0021] The corresponding microscopic images to be stitched are scaled according to the stated scaling factor to determine the scaled microscopic images; or...

[0022] Alternatively, the local pixel regions of the microscopic image to be stitched, including the two stitching references, can be scaled according to the scaling factor to obtain a locally scaled image; and the partially scaled images can be stitched together to obtain a scaled microscopic image.

[0023] The process of stitching and fusing adjacent microscopic images to be stitched together includes: stitching and fusing scaled microscopic images corresponding to adjacent images to be stitched together.

[0024] Optionally, before performing stitching and fusion processing on the scaled microscopic images corresponding to adjacent images to be stitched, the method further includes:

[0025] Based on the position coordinates of the stitching reference in each image to be stitched, the corresponding scaled microscopic image is rotated and corrected.

[0026] The process of stitching and merging adjacent scaled images to be stitched includes: stitching and merging scaled microscopic images that have undergone rotation correction.

[0027] Alternatively, based on the position coordinates of the stitching reference in each image to be stitched, rotation correction can be performed on each image to be stitched.

[0028] The step of scaling the corresponding microscopic image to be stitched according to the scaling factor to determine the scaled microscopic image includes: scaling the corresponding microscopic image to be stitched after rotation correction according to the scaling factor to determine the scaled microscopic image.

[0029] Optionally, based on the premise that the stitching reference pixels with the same two-dimensional positional identifiers in adjacent microscopic images to be stitched coincide, the adjacent microscopic images to be stitched are subjected to stitching and fusion processing to obtain a stitched microscopic image, including:

[0030] Based on the pixel overlap in adjacent images to be stitched, the pixels to be fused and pixels not to be fused in adjacent images to be stitched are determined. The pixels to be fused are the pixels in the adjacent images to be stitched that represent the same imaging object.

[0031] Calculate the fusion grayscale of the pixels to be merged, and use it as the pixel grayscale of the merged pixel;

[0032] The pixels that do not need to be merged and the pixels that need to be merged are stitched together according to the positional relationship between them to obtain the stitched microscopic image.

[0033] Optionally, the splicing reference is the intersection point of the first and second identification lines;

[0034] The step of determining the position coordinates of the stitching reference in the microscopic image to be stitched based on the position coordinates of the first and second identified marker lines includes: determining the position coordinates of the intersection point of the lines based on the position coordinates of the first and second identified marker lines or the coordinates of the lines they are on.

[0035] The step of determining the two-dimensional position identifier of each splicing reference based on the pixel grayscale of the first identifier line and the second identifier line includes: determining the first dimension position identifier based on the pixel grayscale of the first identifier line forming the intersection point of the lines, and determining the second dimension position identifier based on the pixel grayscale of the second identifier line forming the intersection point of the lines.

[0036] The first-dimensional position identifier and the second-dimensional position identifier are combined to obtain the two-dimensional position identifier of the line intersection point.

[0037] Thirdly, embodiments of this disclosure provide a computing device, characterized in that it includes a processor and a memory, the memory being used to store a computer program; when the computer program is loaded by the processor, it causes the processor to execute the stitching method for microscopic scanning images of aquatic organisms as described above.

[0038] Using the glass slides provided in the embodiments of this disclosure for microscopic imaging, at least two stitching references can be determined based on the first and second marker lines, and the pixel coordinates of two adjacent microscopic images to be stitched can be determined. Based on the pixel coordinates of the aforementioned at least two stitching references, the stitching relationship between the two adjacent microscopic images to be stitched can be determined, and thus accurate and rapid stitching of the microscopic images to be stitched can be achieved based on the stitching references. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0040] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort, wherein...

[0041] Figure 1 This is a schematic diagram of the structure of the glass slide provided in the embodiments of this disclosure;

[0042] Figure 2 This is a top view of the microscopic imaging area;

[0043] Figure 3 This is a flowchart of a method for stitching together microscopic scanned images of aquatic organisms provided in an embodiment of this disclosure;

[0044] Figure 4 It is a microscopic image to be stitched together, collected during the actual implementation process;

[0045] Figure 5 It is another microscopic image to be stitched together, collected during the actual implementation process;

[0046] Figure 6 It is by Figure 4 and Figure 5 A microscopic image formed by stitching together two microscopic images to be stitched together;

[0047] Figure 7 This is a schematic diagram of the microscopic image stitching device provided in the embodiments of this disclosure;

[0048] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this disclosure. Detailed Implementation

[0049] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0050] The term "comprising" and its variations as used herein are open-ended inclusion, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0051] To address the high costs associated with existing microbial microscopic image stitching methods due to the lack of reference objects, this disclosure provides a novel glass slide for microscopic imaging of microorganisms, and a method for stitching aquatic organism microscopic scanning images based on the aforementioned glass slide.

[0052] Figure 1 This is a schematic diagram of the structure of a glass slide provided in an embodiment of this disclosure. For example... Figure 1 As shown, the glass slide 100 provided in this embodiment is a glass slide, which includes a glass slide body 101, on which a microscopic imaging region 102 is provided. The microscopic imaging region 102 is the region that holds the water sample to achieve subsequent microscopic imaging.

[0053] Figure 2 This is a top view of the microscopic imaging area. For example... Figure 2 As shown, the microscopic imaging region 102 is provided with multiple first marker lines 1021 and second marker lines 1022, wherein... Figure 2 The horizontal line that runs through the microscopic imaging area 102 is the first marker line 1021, and the vertical line that runs through the microscopic imaging area 102 is the second marker line 1022. The black line at the edge is not the first marker line 1021 or the second marker line 1022, but the edge line of the microscopic imaging area 102 (which can be used to indicate the start or end of microscopic imaging).

[0054] like Figure 2 As shown, each of the first marker lines 1021 is configured to have different pixel grayscale values ​​when performing microscopic imaging under the same lighting conditions, and similarly, each of the second marker lines 1022 is also configured to have different pixel grayscale values ​​when performing microscopic imaging under the same lighting conditions. The fact that each of the first marker lines 1021 and each of the second marker lines 1022 is configured to have different pixel grayscale values ​​when performing microscopic imaging ensures that even if the adjacency relationship of the images to be stitched cannot be determined, it is still possible to stitch together all the microscopic images to be stitched.

[0055] In other embodiments, adjacent first marker lines may be configured to have different pixel grayscale values ​​during microscopic imaging, and adjacent second marker lines 1022 may be configured to have different pixel grayscale values ​​during microscopic imaging. In this case, it is necessary to reasonably set the movement step size during the microscopic imaging process and record the adjacency relationship of each microscopic image to be stitched together to ensure that image stitching can be achieved.

[0056] Furthermore, in this embodiment, the spacing between adjacent first marker lines 1021 and the spacing between adjacent second marker lines 1022 are both less than a threshold distance. The aforementioned threshold distance is a distance that ensures all microscopic images to be stitched together after microscopic imaging have at least two first marker lines 1021 and two marker lines 1022. The threshold distance can be determined based on the actual imaging area size corresponding to the microscopic imaging field of view.

[0057] The preceding section only specifies the spacing between adjacent first marker lines 1021 and adjacent second marker lines 1022; it does not require the spacing between the first marker lines 1021 to be the same as the spacing between the second marker lines. Furthermore, it does not require that the spacing between each adjacent first marker line 1021 be the same, nor does it require that the spacing between each adjacent first marker line 1021 be the same. In other words, in practical applications, the spacing between adjacent first marker lines 1021 only needs to be less than a threshold distance, and is not required to be the same; similarly, in practical applications, the spacing between adjacent second marker lines 1022 only needs to be less than a threshold distance, and is not required to be the same.

[0058] However, considering ease of use (consistency of reference distance), in practical applications, the spacing between adjacent first marker lines 1021 is set to be the same, and the spacing between adjacent second marker lines 1022 is also set to be the same. Furthermore, as... Figure 2 As shown, in some embodiments, the spacing between the first marker line 1021 and the second marker line 1022 is set to be the same. When the spacing between the first marker line 1021 and the second marker line 1022 is set to be the same, the magnification of each image to be stitched (or different regions of the images to be stitched) can be determined based on the actual number of pixels of the first marker line 1021 or the second marker line 1022 in the image.

[0059] See Figure 2 In specific implementation, the first marker line 1021 and the second marker line 1022 are set as solid lines with continuous imaging features. In other embodiments, the first marker line 1021 and the second marker line 1022 can also be set as discontinuous lines (as long as two first marker lines 1021 and two second marker lines 1022 can be imaged simultaneously in a single microscopic imaging).

[0060] like Figure 2 As shown, in this embodiment of the present disclosure, the first marker line 1021 and the second marker line 1022 are set to be perpendicular to each other. In other embodiments, the first marker line 1021 and the second marker line 1022 can also be set to other intersection angles, which does not affect the subsequent determination of the splicing reference based on it, and the splicing of adjacent images to be spliced ​​based on the splicing reference.

[0061] As analyzed above, in this embodiment of the present disclosure, the slide 100 is provided with a first identifier line 1021 and a second identifier line 1022, and the identifier lines to be stitched formed by microscopic imaging each have two first identifier lines 1021 and two identifier lines 1022, so that at least two stitching references for two microscopic images to be stitched can be determined based on the first identifier lines 1021 and the second identifier lines 1022. Based on the aforementioned at least two stitching references, the stitching relationship between two adjacent microscopic images to be stitched can also be determined, and thus the rapid and accurate stitching of microscopic images to be stitched can be achieved based on the stitching references.

[0062] In some embodiments, the aforementioned first marking line 1021 and second marking line 1022 are both etching lines disposed on the glass body. Specifically, the etching lines can be formed by laser etching. To ensure that adjacent first marking lines 1021 have different pixel grayscale values ​​under the same illumination, and to ensure that adjacent second marking lines 1022 have different pixel grayscale values ​​under the same illumination, the etching depth or etching width of adjacent first marking lines 1021 and adjacent second marking lines 1022 are different. More preferably, to ensure that all first marking lines 1021 and all second marking lines 1022 have different pixel grayscale values ​​during microscopic imaging, the etching depth of each first marking line 1021 can be set to be different but the etching width to be the same, and the etching depth of each second marking line 1022 can be set to be different but the etching width to be the same.

[0063] It should be noted here that the aforementioned etching width is much smaller than the size of the microorganisms in the prepared water sample, so as to avoid interfering with the observation of the microorganism morphology (this is also the reason why the first marker line 1021 and the second marker line 1022 are not set to have varying etching widths).

[0064] In other embodiments, the first marker line 1021 and the second marker line 1022 can also be deposition lines disposed on the glass body. These deposition lines can be formed using a coating process or a photoresist coating process used in the semiconductor field. Specifically, to ensure different pixel grayscale values ​​during microscopic imaging under the same illumination conditions, at least adjacent first marker lines 1021 must have different deposition thicknesses or different light absorption characteristics, and at least adjacent second marker lines 1022 must have different deposition thicknesses or different light absorption characteristics. The aforementioned differences in light absorption characteristics can be achieved by changing the type of deposit material or the surface shape of the deposit.

[0065] like Figure 2As shown, in this embodiment of the present disclosure, the first marking line 1021 and the second marking line 1022 are set as straight line segments. In other embodiments, the first marking line 1021 and the second marking line 1022 are set as arcs, wavy lines or other types of line segments, which does not hinder the realization of their marking function.

[0066] The spacing between the first marker line 1021 and the second marker line 1022 was mentioned earlier. In practical applications, the spacing must be set to ensure that adjacent microscopic images can be stitched together, while also considering imaging efficiency and microscope focal length adjustment. Taking all these factors into account, the spacing is set to 1 / 3 to 1 / 2 of the side length of the imaging field of view at standard magnification.

[0067] As mentioned earlier, slide 100 is a glass slide. In practice, slide 100 is not limited to being a glass slide; it can also be a coverslip.

[0068] In addition to providing the aforementioned slide structure, this disclosure also provides a method for stitching together microscopic scanning images of aquatic organisms. The images to be stitched together by this method are obtained by capturing samples prepared using the aforementioned slide as a carrier using a microscopic imaging system.

[0069] Figure 3 This is a flowchart of a method for stitching together microscopic scanning images of aquatic organisms provided in an embodiment of this disclosure. Figure 3 As shown, the stitching method for microscopic scanning images of aquatic organisms includes S110-S120.

[0070] S110: Identify the first and second marker lines in the microscopic image to be stitched, determine the position coordinates of at least two stitching references based on the pixel coordinates of the identified first and second marker lines, and determine the two-dimensional relative position markers of each stitching reference based on the pixel grayscale of the first and second marker lines.

[0071] In a specific embodiment, image processing algorithms such as Hough transform can be used to process the pixel content in the stitched microscopic image to determine the first and second marker lines. As analyzed earlier, since the first and second marker lines are straight lines in conventional applications, the pixels containing the first and second marker lines can be easily determined using Hough transform.

[0072] In practical applications, during the microscopic imaging process, the microscopic imaging system and the slide move horizontally relative to each other or vertically in the same direction. Therefore, after the slide is relatively fixed, the pointing directions of the first and second marker lines do not change much during the microscopic imaging process. Thus, the first and second marker lines and their position coordinates in the microscopic image to be stitched can be identified relatively accurately.

[0073] After identifying the position coordinates of the first and second marker lines in the microscopic image, the pixel grayscale values ​​of the pixels within the aforementioned coordinate regions are extracted to determine the pixel grayscale values ​​of the first and second marker lines. Based on a pre-defined correspondence between pixel grayscale values ​​and position markers, the corresponding line position markers can be determined according to the pixel grayscale values ​​of the first and second marker lines. These line position markers are used to indicate which first marker line or which second marker line in the slide it is.

[0074] In this embodiment, the stitching reference is a reference jointly determined by the first and second marker lines, used to determine the image stitching baseline. Specifically, the stitching reference can be the intersection point of the first and second marker lines, or it can be the first and second marker lines themselves. Considering the ease of processing when using points as references, the aforementioned intersection point is often used as the stitching reference in practical applications.

[0075] In practice, after determining the pixel coordinates of the first and second marker lines, the linear expressions for the first and second marker lines can also be determined. Based on the linear expressions of the first and second marker lines, the position coordinates of the splicing reference can also be determined.

[0076] After determining the line positions of the first and second marker lines as described above, the two-dimensional relative position of the intersection point determined by the first and second marker lines can also be determined. Specifically, the two-dimensional relative position is obtained by combining the line position marks of the first and second marker lines, and it represents the point formed by the intersection of which first marker line and which second marker line.

[0077] In practice, in order to stitch two images together, at least two stitching references should be determined to establish the relative positional and angular relationships between the two images to be stitched together.

[0078] S120: Based on the principle that the stitching reference pixels with the same relative position markings in adjacent microscopic images to be stitched coincide, the adjacent microscopic images to be stitched are stitched and fused to obtain a stitched microscopic image.

[0079] Once the stitching reference for the microscopic images to be stitched is determined, the adjacency relationship between the microscopic images to be stitched can also be determined.

[0080] In some embodiments, the slides used have different gray levels for each of the first and second marker lines. The corresponding line position markers are absolute markers, and the two-dimensional relative position markers of each stitching parameter can be used as absolute indicators of the relative positional relationship of the images to be stitched. Therefore, adjacent microscopic images to be stitched can be determined based on the two-dimensional relative position markers of the stitching reference. Subsequently, the adjacent microscopic images to be stitched can be stitched and fused using the alignment of stitching reference pixels with the same relative position markers as a reference, to obtain the stitched microscopic image.

[0081] In other embodiments, adjacent first marker lines on the slide have different gray levels, and adjacent second marker lines have different gray levels. In this case, the adjacency relationship of each image to be stitched can be determined based on the acquisition of the images to be stitched and the movement path of the lens of the microscopic imaging system relative to the slide, thereby determining adjacent images to be stitched. Subsequently, based on the alignment of stitching reference pixels with the same relative position markers in the adjacent microscopic images to be stitched, the adjacent microscopic images to be stitched can be stitched and fused to obtain a stitched microscopic image.

[0082] Figure 4 It is a microscopic image to be stitched together, collected during the actual implementation process. Figure 5 This is another microscopic image to be stitched, acquired during the specific implementation. Using the aforementioned S110 processing, the pixel coordinates of reference point 1 and reference point 2 (reference point 1 and reference point 2 are formed by the intersection of the first and second marker lines of different gray levels) on the two microscopic images to be stitched are determined. Then, the images can be stitched together by aligning reference point 1 and reference point 2 in the two images as a reference. Figure 6 It is by Figure 4 and Figure 5 The resulting microscopic image, formed by stitching together two images, demonstrates that the method described above successfully stitched the two images together correctly. It should be noted here that... Figure 4 and Figure 5 The two microscopic images to be stitched together are two images with the same magnification.

[0083] As analyzed above, by preparing samples using the aforementioned glass slides and obtaining microscopic images to be stitched using a microscopic imaging system, the stitching reference can be determined using the first and second marker lines in the microscopic images to be stitched. Subsequently, based on the stitching reference, the correspondence between pixels in two adjacent microscopic images to be stitched can be quickly determined, thereby achieving rapid image alignment and stitching.

[0084] In some applications, when using a microscopic imaging system (especially one with image self-correction based on magnification and optical lens distortion characteristics) to acquire images of different regions of a sample (i.e., forming microscopic images of different regions to be stitched together), the microscopic imaging system automatically focuses based on the clarity of the organism within the image region, thereby capturing the clearest possible image. However, this automatic focusing results in varying degrees of geometric distortion and positioning misalignment between the different microscopic images to be stitched together. In such cases, directly stitching together two adjacent microscopic images can lead to misalignment.

[0085] To avoid the aforementioned problems, S130-S150 can also be executed before executing S120.

[0086] S130: Determine the number of pixels between the two stitching references in the microscopic images to be stitched.

[0087] This place is Figure 4 For example, determining the number of interval pixels corresponding to the target reference in the stitching reference is to determine the number of interval pixels between the aforementioned reference point 1 and reference point 2.

[0088] S140: Determine the scaling factor based on the number of interval pixels and the corresponding number of standard pixels.

[0089] S150: Scale the corresponding microscopic image to be stitched according to the scaling factor to determine the scaled microscopic image.

[0090] The number of standard pixels is determined by performing microscopic imaging on the two stitched references at a standard magnification, or by performing microscopic imaging on a similar reference at the same distance from the two stitched references.

[0091] It can be assumed that if the number of standard pixels is less than the aforementioned number of interval pixels, then the shooting... Figure 4 The image magnification at that time is greater than the standard magnification. If the number of standard pixels is greater than the aforementioned interval pixel number, then the image is captured... Figure 4 The magnification of the image at that time is less than the standard magnification.

[0092] Specifically, the scaling factor is determined based on the number of intervals and the corresponding number of standard pixels. This is achieved by dividing the number of standard pixels by the number of interval pixels, and then using the divisor as the scaling factor.

[0093] After obtaining the aforementioned scaling factor, the microscopic images to be stitched together can be scaled using this factor to determine the scaled microscopic images. It is conceivable that the scaled microscopic images of each microscopic image to be stitched together use the same image magnification, and that the number of imaging pixels for the same length is the same across all scaled microscopic images.

[0094] When performing the aforementioned S130-S150, the aforementioned S120 performs stitching processing on adjacent microscopic images to be stitched together, which is a stitching and fusion processing of scaled microscopic images.

[0095] In other applications, due to the optical imaging system, the microscopic images to be stitched together formed by microscopic imaging exhibit significant distortion in specific areas (especially edge areas). Directly using distorted images for image stitching may result in uneven transitions at the edges of aquatic organisms after stitching, making subsequent automated identification of aquatic organism types difficult. To address this issue, in some embodiments of this disclosure, before executing the aforementioned S120, the following steps S160-S190 may also be executed.

[0096] S160: Determine the number of pixels between two adjacent stitching references in the microscopic images to be stitched.

[0097] S170: Determine the scaling factor based on the number of interval pixels and the corresponding number of standard pixels.

[0098] The implementation process of S160-S170 is the same as that of S130-S140 mentioned above, and will not be repeated here. In specific implementation, the two determined critical stitching references are preferably stitching references on the same first or second marker line. In addition, in specific implementation, the number of first and second marker lines included in the microscopic image to be stitched is more than two, so that different regions of the microscopic image to be stitched receive different scaling factors, reflecting the image distortion characteristics caused by the optical imaging system.

[0099] S180: The pixel regions of the microscopic image to be stitched, including the two stitching references, are scaled according to the scaling factor to obtain a partially scaled image.

[0100] After obtaining the scaling factors corresponding to different regions of the different microscopic images to be stitched, the regions corresponding to the images to be stitched can be scaled using the aforementioned scaling factors. Specifically, the local pixel regions, including those corresponding to the two stitching references, are scaled to obtain partially scaled regions. The partially scaled regions obtained through the aforementioned scaling process can partially correct image distortion caused by the optical system's imaging.

[0101] S190: Stitch together the partially scaled images to obtain a scaled microscopic image.

[0102] After obtaining the aforementioned partially scaled images corresponding to different local pixel regions according to S170, these partially scaled images can be stitched together to form a scaled microscopic image. It should be noted that when stitching the images, the aforementioned stitching reference can be used as a basis. Furthermore, during the image stitching process, the scaled microscopic image corresponding to the central region of the microscopic image to be stitched can be used as a base, and other adjacent scaled microscopic images can be stitched sequentially. During the stitching process, for some blank areas (areas without filled pixels), their pixel grayscale can be determined by pixel resampling to avoid blank pixel areas in the resulting scaled microscopic image.

[0103] It should be noted that the prerequisite for performing the aforementioned S160-S190 is that each microscopic image to be stitched together is imaged using the same magnification during the microscopic imaging process.

[0104] When performing the aforementioned S130-S150 or S160-S190, S120 specifically involves performing stitching and fusion processing on the scaled microscopic images corresponding to adjacent images to be stitched.

[0105] In practical applications, during the process of moving the slide relative to the imaging lens of the microscopic imaging system according to a preset step size, the slide may rotate slightly. In this case, image stitching cannot be achieved by translational fusion stitching, and rotation correction processing is required. Therefore, in some embodiments, before executing the aforementioned S120, the following S200 can also be executed.

[0106] S200: Based on the position coordinates of the stitching reference in each image to be stitched, perform rotation correction on the corresponding scaled microscopic image.

[0107] Still with Figure 4 For example, the coordinates of reference point 1 are (x... 11 ,y 11 The coordinates of reference point 2 are (x...). 12 ,y 12 If the rotation angle of the image to be stitched relative to the horizontal direction is determined as θ1 = tan -1 y 12 -y 11 Scaling up microscopic images of stitched images

[0108] x 12 -x 11

[0109] Rotation correction can be performed by rotating the image by θ1 so that the line containing reference point 1 and reference point 2 is a horizontal line.

[0110] The aforementioned method can also correct each microscopic image to be stitched to a reference angle, thus adapting to the correct stitching process. When performing the aforementioned S200, S150 or S190 specifically involves stitching and fusing the scaled microscopic images that have undergone rotation correction.

[0111] The previous solution involves scaling the microscopic image to be stitched together before rotating it. In other embodiments, the microscopic image to be stitched together can be rotated and corrected, and then scaled according to the scaling factor to determine the scaled microscopic image.

[0112] The preceding text mentioned the process of stitching and fusing adjacent microscopic images to be stitched together to obtain a stitched microscopic image. This section analyzes how the stitching and fusing process is performed to obtain the stitched microscopic image. In some embodiments, the stitching and fusing process of the microscopic images to be stitched may include the following steps S121-S123.

[0113] S121: Based on the fact that the pixels in the adjacent images to be stitched coincide, determine the pixels that need to be merged and the pixels that do not need to be merged in the adjacent images to be stitched.

[0114] The pixels to be fused are those representing the same imaging object in the adjacent images to be stitched together, that is, the pixels corresponding to the overlapping imaging regions. Correspondingly, the pixels that do not need to be fused are those without overlapping imaging regions.

[0115] S122: Calculate the fusion grayscale of the pixels to be fused, and use it as the pixel grayscale of the fused pixels.

[0116] S123: The pixels that do not need to be merged and the pixels that need to be merged are stitched together according to the positional relationship between the pixels that do not need to be merged and the pixels that need to be merged, to obtain the stitched microscopic image.

[0117] For pixels that need to be merged, the mean gray level of corresponding pixels in the two images to be stitched can be calculated, and the mean value can be used as the pixel gray level of the merged pixel.

[0118] After determining the pixel grayscale of the fused pixels, a stitching process can be performed based on the original pixels corresponding to the fused pixels and the pixels that do not need to be fused, to obtain a stitched microscopic image. In specific implementation, the number of pixels that need to be translated horizontally and vertically in one of the microscopic images to be stitched can be determined according to the pixel positions of the two microscopic images to be stitched (after rotation correction and scaling). Then, the pixel coordinates of each point after stitching are determined according to the aforementioned number of translated pixels, and then it is determined which pixels need to be fused and which do not need to be fused.

[0119] In practice, some pixels may have null values ​​due to the aforementioned scaling and rotation processes. In this case, the grayscale value of the pixel can be set to the maximum brightness.

[0120] Based on the previous analysis, the pixel grayscale of each point in the stitched microscopic image can be represented by the following formula.

[0121]

[0122] Where P1(i,j) represents the pixel grayscale value in the first microscopic image to be stitched together, and P2(i,j) represents the pixel grayscale value in the second microscopic image to be stitched together. When using... In the case of a certain condition, the pixel is considered a pixel that needs to be merged; otherwise, it is considered a pixel that does not need to be merged.

[0123] In practice, a large image canvas can be established, and adjacent images to be stitched can be determined sequentially according to the aforementioned method. Then, each image to be stitched can be stitched sequentially according to the stitching order until the image of the entire microscopic imaging area 102 of the slide 100 is obtained.

[0124] In some embodiments, the microscopic images to be stitched, acquired using a microscopic imaging system, are color images. To enable rapid stitching of the images, the color image can be converted to a grayscale image; specifically, this can be achieved using I... gray x,y=0.299×R(x,y)+0.587×G(x,y)+0.114×B(x,y) to obtain the gray value of pixel (x,y).

[0125] In addition to providing the aforementioned method for stitching together microscopic scanned images of aquatic organisms, this disclosure also provides a microscopic image stitching device.

[0126] Figure 7 This is a schematic diagram of the microscopic image stitching device provided in an embodiment of this disclosure. Figure 7 As shown, the microscopic image stitching device 700 includes a stitching reference recognition unit 701 and a stitching unit 702.

[0127] The stitching reference recognition unit 701 is used to identify the first and second marker lines in each microscopic image to be stitched, and to determine the position coordinates of the stitching reference in the microscopic image to be stitched based on the positional features of the identified first and second marker lines, and to determine the two-dimensional position marker of each stitching reference according to the pixel grayscale of the first and second marker lines; wherein the microscopic image to be stitched is obtained by a microscopic imaging system from a sample prepared based on the aforementioned slide.

[0128] The stitching unit 702 is used to stitch and fuse adjacent microscopic images to obtain a stitched microscopic image based on the alignment of stitching reference pixels with the same two-dimensional position identifier in the adjacent microscopic images to be stitched.

[0129] In some embodiments, the microscopic image stitching device 700 further includes an image correction unit, which is used to determine the number of interval pixels between two stitching references in the microscopic image to be stitched; determine a scaling factor based on the number of interval pixels and the corresponding number of standard pixels; wherein the number of standard pixels is determined by performing microscopic imaging on the two stitching references at a standard magnification, or by performing microscopic imaging on a similar reference at the same distance from the two stitching references; and perform scaling processing on the corresponding microscopic image to be stitched according to the scaling factor to determine a scaled microscopic image; or, according to the scaling factor, perform scaling processing on a local pixel region of the microscopic image to be stitched, including the two stitching references, to obtain a locally scaled image; and stitch the partially scaled image to obtain a scaled microscopic image. Correspondingly, the stitching unit 702 performs stitching and fusion processing on the scaled microscopic images corresponding to adjacent images to be stitched.

[0130] In some embodiments, the image correction unit performs rotation correction on the corresponding scaled microscopic image based on the position coordinates of the stitching reference in each image to be stitched; the stitching unit 702 performs stitching and fusion processing on the scaled microscopic image after rotation correction.

[0131] Alternatively, the image correction unit performs rotation correction on each image to be stitched based on the position coordinates of the stitching reference in each image; then, it performs scaling processing on the corresponding microscopic image to be stitched after rotation correction according to the scaling factor to determine the scaled microscopic image.

[0132] During the stitching process, the stitching unit 702 uses the pixel overlap of the stitching reference in adjacent images to determine the pixels to be fused and the pixels not to be fused in the adjacent images to be stitched. The pixels to be fused are the pixels in the adjacent images to be stitched that represent the same imaging object. The fusion grayscale of the pixels to be fused is calculated and used as the pixel grayscale of the fused pixel. Then, the pixels not to be fused and the pixels to be fused are stitched together according to the positional relationship between the pixels not to be fused and the pixels to be fused to obtain the stitched microscopic image.

[0133] In some embodiments, the splicing reference is the intersection point of the first and second identifier lines; the splicing reference identification unit 701 determines the position coordinates of the line intersection point based on the pixel coordinates or the line coordinates of the identified first and second identifier lines; it determines a first-dimensional position identifier based on the pixel grayscale of the first identifier line forming the line intersection point, and determines a second-dimensional position identifier based on the pixel grayscale of the second identifier line forming the line intersection point; the first-dimensional position identifier and the second-dimensional position identifier are combined to obtain a two-dimensional position identifier of the line intersection point.

[0134] This disclosure also provides a computing device for implementing the aforementioned method. Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this disclosure. See below for details. Figure 8 It shows a schematic diagram of a structure suitable for implementing the computing device 800 in the embodiments of this disclosure. Figure 8 The computing device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0135] like Figure 8 As shown, the computing device 800 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory ROM 802 or a program loaded from a storage device 808 into a random access memory RAM 803. The RAM 803 also stores various programs and data required for the operation of the computing device 800. The processing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0136] Typically, the following devices can be connected to I / O interface 805: input devices 805 including, for example, touchscreens, touchpads, cameras, microphones, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows computing device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 A computing device 800 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0137] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by a processing device 801, it performs the functions defined in the methods of embodiments of this disclosure.

[0138] It should be noted that the computer-readable medium described above in this disclosure may be a computer-readable storage medium, a computer-readable signal medium, or any combination thereof.

[0139] Computer-readable storage media can be, for example—but not limited to—electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0140] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0141] In some implementations, clients and computing devices can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0142] The aforementioned computer-readable medium may be included in the aforementioned computing device; or it may exist independently and not assembled into the computing device.

[0143] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the tester's computer, partially on the tester's computer, as a standalone software package, partially on the tester's computer and partially on a remote computer, or entirely on a remote computer or computing device. In cases involving a remote computer, the remote computer can be connected to the tester's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0145] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not necessarily limiting in certain circumstances. The functions described above can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), etc.

[0146] The above are merely specific embodiments of this disclosure, enabling those skilled in the art to understand or implement this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A glass slide, characterized in that, include: The slide body has a grid-like marking line set in the microscopic observation area of ​​the slide body. The mesh marking lines include multiple first marking lines that penetrate the microscopic observation area along a first direction, and second marking lines that penetrate the microscopic observation area along a second direction, wherein the first direction is different from the second direction; At least adjacent first marker lines are configured to have different light processing characteristics, resulting in different line grayscale values ​​after microscopic imaging under the same lighting conditions; at least adjacent second marker lines are configured to have different light processing characteristics, resulting in different line grayscale values ​​after microscopic imaging under the same lighting conditions. The spacing between adjacent first marker lines and the spacing between adjacent second marker lines are both less than a threshold distance; the threshold distance is the distance that ensures that the microscopic images to be stitched together after microscopic imaging have at least two first marker lines and at least two second marker lines.

2. The glass slide according to claim 1, characterized in that: Each of the first marker lines has different light processing characteristics, and the grayscale of each first marker line changes sequentially after microscopic imaging; and / or, Each of the second marker lines has different light processing characteristics, and the grayscale of each second marker line changes sequentially after microscopic imaging.

3. The glass slide according to claim 1 or 2, characterized in that, The first marking line and / or the second marking line are etching lines formed by etching in the microscopic observation area; at least adjacent first marking lines have different etching depths and / or etching widths; And / or, at least the etching depth and / or etching width of adjacent second marker lines are different; or, The first and / or the second marker lines are deposition lines formed on the microscopic observation area; at least adjacent first marker lines have different deposition thicknesses and / or different light absorption characteristics; and / or, at least adjacent second marker lines have different deposition thicknesses and / or different light absorption characteristics.

4. The glass slide according to claim 1 or 2, characterized in that: Each of the first marking lines is equally spaced on the glass slide body; And / or, each of the second marking lines is disposed at equal intervals on the glass slide body.

5. A method for stitching together microscopic scanning images of aquatic organisms, characterized in that, include: The first and second marker lines in each microscopic image to be stitched are identified, and the position coordinates of the stitching reference in the microscopic image to be stitched are determined based on the positions of the identified first and second marker lines. The two-dimensional position markers of each stitching reference are determined according to the pixel grayscale of the first and second marker lines. The microscopic images to be stitched are obtained by a microscopic imaging system from samples prepared based on glass slides as described in any one of claims 1-4. Based on the premise that the stitching reference pixels with the same two-dimensional position identifier in adjacent microscopic images to be stitched coincide, the adjacent microscopic images to be stitched are stitched and fused to obtain a stitched microscopic image.

6. The splicing method according to claim 5, characterized in that, Before performing stitching and fusion processing on adjacent microscopic images to be stitched, the method further includes: determining the number of pixels between two stitching references in the microscopic images to be stitched; The scaling factor is determined based on the number of interval pixels and the corresponding number of standard pixels; wherein the number of standard pixels is determined by performing microscopic imaging on the two stitching references at a standard magnification, or by performing microscopic imaging on a similar reference that has the same distance from the two stitching references. The corresponding microscopic image to be stitched is scaled according to the scaling factor to determine the scaled microscopic image; Alternatively, the local pixel regions of the microscopic image to be stitched, including the two stitching references, can be scaled according to the scaling factor to obtain a locally scaled image; and the locally scaled image can be stitched together to obtain a scaled microscopic image. The process of stitching and fusing adjacent microscopic images to be stitched together includes: stitching and fusing scaled microscopic images corresponding to adjacent images to be stitched together.

7. The method for stitching together microscopic scanning images of aquatic organisms according to claim 6, characterized in that, Before performing stitching and fusion processing on the scaled microscopic images corresponding to adjacent images to be stitched, the method further includes: Based on the position coordinates of the stitching reference in each image to be stitched, the corresponding scaled microscopic image is rotated and corrected. The process of stitching and merging adjacent scaled images to be stitched includes: stitching and merging scaled microscopic images that have undergone rotation correction. Alternatively, based on the position coordinates of the stitching reference in each image to be stitched, rotation correction can be performed on each image to be stitched. The step of scaling the corresponding microscopic image to be stitched according to the scaling factor to determine the scaled microscopic image includes: scaling the corresponding microscopic image to be stitched after rotation correction according to the scaling factor to determine the scaled microscopic image.

8. The method for stitching together microscopic scanning images of aquatic organisms according to claim 5, characterized in that, Based on the premise that the stitching reference pixels with the same two-dimensional positional identifiers in adjacent microscopic images to be stitched coincide, the adjacent microscopic images to be stitched are stitched and fused to obtain a stitched microscopic image, including: Based on the pixel overlap in adjacent images to be stitched, the pixels to be fused and pixels not to be fused in adjacent images to be stitched are determined. The pixels to be fused are the pixels in the adjacent images to be stitched that represent the same imaging object. Calculate the fusion grayscale of the pixels to be merged, and use it as the pixel grayscale of the merged pixel; The pixels that do not need to be merged and the pixels that need to be merged are stitched together according to the positional relationship between them to obtain the stitched microscopic image.

9. The method for stitching together microscopic scanning images of aquatic organisms according to any one of claims 5-8, characterized in that, The splicing reference is the intersection point of the first and second identification lines; The step of determining the position coordinates of the stitching reference in the microscopic image to be stitched based on the positions of the first and second identified marker lines includes: determining the position coordinates of the intersection point of the lines based on the position coordinates of the first and second identified marker lines or the coordinates of the lines they are on. The step of determining the two-dimensional position markers of each stitching reference based on the pixel grayscale of the first and second marker lines includes: A first-dimensional position identifier is determined based on the pixel grayscale of the first identifier line that forms the intersection point of the lines, and a second-dimensional position identifier is determined based on the pixel grayscale of the second identifier line that forms the intersection point of the lines. The first-dimensional position identifier and the second-dimensional position identifier are combined to obtain the two-dimensional position identifier of the intersection point of the lines.

10. A computing device, characterized in that, It includes a processor and a memory, the memory being used to store a computer program; when the computer program is loaded by the processor, it causes the processor to execute the stitching method for microscopic scanning images of aquatic organisms as described in any one of claims 5-9.

Images